boost: update to 1.70.0

[michaelld (Michael Dickens)]

Boost 1.70.0 is out as of 20190412.

Prior Description, which is all likely still relevant: [[ Boost 1.69.0 is out as of 120181212.

Because the Portfile currently uses "--layout=tagged", the resulting libraries names change from the Boost 1.68.0 to include an abbreviated architecture (e.g., "x64" for Intel 64-bit; "p32" for PPC 32-bit); thus for example, the prior naming might be "libboost_system-mt.dylib", while the new naming is "libbboost_system-mt-x64.dylib". This slight change combined with a defect in how CMake works with Clang results in the FindBoost.cmake script failing to detect the new library names with significant coercion.

This issue is resolved by moving to "--layout=system", which reverts the library names to just the minimum for each component; for example here "libboost_system.dylib".

Most of the changes to this latest Boost itself seem to be compatible with the ports I've tried building with it, once the Portfile is fixed to properly detect the changed name. That said, the 10 or so ports I checked pale compared with the total number of ports that depend on Boost (some 250 in total).

Verifying that "enough" of these ports build and function properly "enough" is an arduous task. We in MacPorts have yet to determine a reasonable approach to updating Boost, so the version provided in MacPorts regularly lags the latest release. Please be patient. ]]

[michaelld (Michael Dickens)]

OK so there are some issues I'm coming upon with changes to boost::thread and boost::posix_time ... hopefully it's just slopping programming on the project's part. Or maybe the Boost folks fixed some type conversation issues. I'll work on fixing & verify back backward compatibility with (at least) boost 1.66.0.

[elfprince13 (Thomas Dickerson)]

There are also some changes in the interaction between CMake and Boost::python, which may be non-backwards compatible.

[michaelld (Michael Dickens)]

Given the ~4 months for projects to fix their issues with Boost 1.67.0, and possible more issues now with 1.68.0, let's test this release with ports that require Boost to make sure the vast majority work. I would like to get this updated Boost in place sooner rather than later, assuming compatibility is good.

[michaelld (Michael Dickens)]

Updated the patch to reset the revision to 0.

[ryandesign (Ryan Carsten Schmidt)]

I'm not working on this; go ahead with what you think is best.

[michaelld (Michael Dickens)]

Updated the patch to remove part of a patchfile incorporated into Boost master for +python3X only.

[michaelld (Michael Dickens)]

One major ABI naming change: the Boost::Python library is renamed to include the major.minor Python version but with no "." ... so, "27" for Python 2.7 and "36" for Python 3.6 (for 2.,7 was "libboost_python-mt.dylib" is now "libboost_python27-mt.dylib"). Not sure if this is a change from 1.67.0 or 1.66.0 ... but it's important since any project using CMake to find Boost must now specify "python27" or "python36" as a component to look for, not just "python" or "python2" or "python3"; guessing other build systems will be impacted as well.

[michaelld (Michael Dickens)]

The ABI name change was in Boost 1.67.0 ... so, projects might already be updating to check for it. Not sure where CMake is on this change.

[michaelld (Michael Dickens)]

CMake is aware of this change: ​https://gitlab.kitware.com/cmake/cmake/blob/master/Modules/FindBoost.cmake#L43 . One just has to use the correct module name, which now depends on which version of Boost is installed ... so, some logic needs to be in place for projects looking for any Boost Python-based ABI (e.g., "python" or "numpy"). Fun fun!

[luzpaz]

FYI 1.69.0 is latest stable

[elfprince13 (Thomas Dickerson)]

Any update on the progress on this? I'm working on a project where my collaborators are on Linux with 1.67, and constantly having to git stash/git stash pop this patch in our CMake:

-find_package(Boost 1.65 REQUIRED COMPONENTS system context python27)
+find_package(Boost 1.65 REQUIRED COMPONENTS system context python)

is...annoying.

[michaelld (Michael Dickens)]

I haven't had or taken the time to work over the boost dependencies to make sure that 90%+ of them work. There are some 250 ports that depend on Boost, so doing this check isn't trivial. It's also a challenge to know which ports don't build with the current Boost and ignore them if they still don't build.

MacPorts needs a better way to do testing / verification of such ports that are depended on so heavily; doing it manually as we currently do means that someone has to find and take the time to do all of this checking -- and, then fixing and/or issuing tickets for those that aren't easily fixable. Maybe crowdsourcing this checking is the way to go?

If you need something right now, use the patch attached to this ticket to update to Boost 1.68.0. There is no patch for 1.69.0 because the Boost folks changed the default API naming scheme to include the "ARCH" type & that wreaks havoc on CMake-based builds on macOS / OSX (but, not in general). Again, I have't had or taken the time to figure out the change to revert the "ARCH" to get back to what MacPorts desires (and what works for CMake).

[mohd-akram (Mohamed Akram)]

Boost 1.70.0 released - ​https://www.boost.org/users/history/version_1_70_0.html

[michaelld (Michael Dickens)]

Thanks for the info on Boost 1.70.0 @mohd-akram . Any incompatibilities you know of already?

[michaelld (Michael Dickens)]

OK so i've figured out how to revert library names back to before the added <ARCH> ... building using this patch does indeed result in the correct ABI naming as we currently have with 1.66.0 ... yay! One issue down. Here's the whole patch in case anyone wants to use it right now for testing (-p0)

--- boostcpp.jam.orig
+++ boostcpp.jam
@@ -163,7 +163,7 @@
             <base> <threading> <runtime> ;
     case 1.69 :
         .format-name-args =
-            <base> <threading> <runtime> <arch-and-model> ;
+            <base> <threading> <runtime> ;
     }
 }
 else if $(layout) = system

[michaelld (Michael Dickens)]

1.70.0 provides a CMake scripts for Boost detection using modern targets(any of 1.67-1.69 might have also; I don't recall seeing these being installed by default, but who knows). There are multiple scripts per Boost component .. so a -ton- of these.

I'm having (an) issue/s getting them to work; not sure what's going on yet, hence the "(an)".

This detection is kinda critical for -lots- of ports, so it needs solid testing / fixing if necessary ... that or we just don't install the CMake scripts & use the (meh) one provided by CMake itself.

Once the CMake detection is worked out, my hope is to work on a testing / verification script that iterates over all of the ports that depend on Boost && tests each in turn all the way through "install" && tallies the successes and failures in some reasonable way.

[michaelld (Michael Dickens)]

The Portfile section to get the CMake scripts installed where at least some ports do so (share/cmake), is:

pre-destroot {
    destroot.args {*}${build.args} --prefix=${destroot}${prefix} \
        --cmakedir=${destroot}${prefix}/share/cmake
    system "find ${worksrcpath} -type f -name '*.gch' -exec rm {} \\;"
}

note to --cmakedir addition. Without this tweak they are installed into lib/cmake.

Not sure which is the better location, but after a quick review of the ports I have that install into both of these, I'm actually inclines to leave the install in lib/cmake. Anyone have an opinion about this?

[cjones051073 (Chris Jones)]

Could I suggest it would be very useful to do something similar to the on going openssl 1.1.x update here, so push your changes to a WIP github PR and also start to revbump dependents as required.

​https://github.com/macports/macports-ports/pull/3822

I think that would make it much easier for others to review and test (I personally am not going to faff about downloading and installing patches by hand, but if its a branch in github thats a different story).

I would also suggest aiming for perfection here is counter productive. We should aim to fix things as far as is reasonable, but at some point just merge the changes, and deal with whatever that causes.

[michaelld (Michael Dickens)]

Agreed, Chris. I'll do a WIP PR hopefully later today.

Updates from the past week, mostly discussing the new Boost 1.70.0 with others who have also been trying it out:

Disable the new CMake build scripts provided by Boost, as they are not yet ready for prime time; just use the one provided by CMake as it should work as currently expected by other ports. I'll post that portfile tweak once I've confirmed it, per ...

I'm updating Boost on my 10.14 build system today & will start testing dependent ports as the week goes on, hopefully creating something of an automated script to do so. Others can also test dependent ports. Once we hit a reasonable threshold (80%? 90%?) of them being functional, I'd be ready to commit & work on fixing up the remainder afterwards.

[michaelld (Michael Dickens)]

OK I have a WIP PR for testing < ​https://github.com/macports/macports-ports/pull/4243 >. In my testing thus far, it works very well overall, with the primary -port- issue being the MPI variant, which fails in destroot & I'm not sure how to fix it yet. Please test / verify & report back successes / failures!

[michaelld (Michael Dickens)]

I know that the gnuradio ports currently fail because of an ASIO API change. I'm working on fixing them.

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #58511.

[michaelld (Michael Dickens)]

I've fixed gnuradio-next to work with this Boost, and have pushed the fix to an upstream PR. I need to start adding rev-bumps of compatible ports to the WIP PR listed above.

[blair (Blair Zajac)]

I started working on a PR for the boost package. Two of the patches needed work, you can find them in my local branch: ​https://github.com/blair/macports-ports/tree/boost-1.70.0 .

[michaelld (Michael Dickens)]

@blair: Thanks for your work!

Did you try the +mpi option? NP if not; just wondering since I think it was broken when I tried it with Boost 1.70.0.

Also: Looks like you deleted one (patch-random-uniform_on_sphere.hpp.diff) & removed a chunk from another (patch-boost-python3.diff) ... but that's it, yes?

[michaelld (Michael Dickens)]

In b09ef91d8de72290191eb87b0b33641b37e5a5ca/macports-ports (master):

boost: update to 1.70.0

1) tweak current patches to apply cleanly.

2) Add a patch to revert the ABI naming to what we have historically had in MacPorts: "libboost_COMPONENT-mt.dylib", where COMPONENT is the component name e.g., "system" or "thread" or the like.

Closes: #56294