boost: enable static libraries by default

[mohd-akram (Mohamed Akram)]

I was trying to build something with boost and found that the static libs aren't built by default. Is there a reason for this? I also found it very strange that the variants are confusingly called +no_single and +no_static, rather than +single and +static. I suggest renaming the variants with default variants being +static.

[ryandesign (Ryan Carsten Schmidt)]

Of course there is a reason. 😊 But the reasons don't all necessarily still apply, and the way that the problems would be solved today would be different.

Boost used to default to building all flavors of the libraries (dynamic multithreaded, dynamic single-threaded, static multithreaded, and static single-threaded). This took a very long time and people complained; that was in 2010. The no_single and no_static variants were created so users could disable the flavors they didn't need if they wanted to save build time.

It would have been more logical to have named those variants single and static and to have them do the opposite, as you suggested, but there was a longstanding bug in MacPorts base which did not preserve disabled variants across upgrades, so it was customary to name variants that disabled features with a "no_" prefix and to have users enable those variants instead. I think this bug was #2377 which was fixed in MacPorts 1.9.0 which was released a few months before the variants were added to boost, so the variant names in boost were probably just following the existing convention by that point despite the underlying reason for that convention having already been removed. It took some time before most of the "no_"-prefixed variants were removed from ports. For example, the effort to change no_x11 variants to x11 lasted from 2013 to 2015.

The addition of these variants in boost predated the existence of MacPorts 2.0.0 (​released in 2011) with the capability of fetching pre-built archives and of our buildbot infrastructure (​installed in 2011) for producing pre-built archives. For distributable ports, pre-built archives lessen the significance of long build times, but they don't eliminate it completely because users who cannot use pre-built archives are still affected.

In 2012, the no_single and no_static variants were enabled by default so that only the most-commonly-used types of libraries were built by default. This made the build take only ¼ of the time.

Using variants to control which libraries get installed is bad because MacPorts base does not offer the capability for other ports that require those libraries to depend on a variant of a port; see #126. The active_variants 1.1 portgroup implements a workaround for that but it is easily subverted by users. The correct solution would be for the different flavors of boost libraries to be made available as subports instead of variants. That way, each port can declare a dependency on the subport for whichever flavor of boost libraries it requires. The addition of the variants to the boost port predates the existence of the subport feature which was introduced in MacPorts 2.0.0. See #67116 for the similar request that the python variants should be subports for a similar reason.

See also #38406 for a request similar to yours that single-threaded libraries should be enabled by default.

All of this also predates the split of the boost port into multiple ports for different versions of boost. This doesn't really affect the solution except to the extent that it multiples the amount of work involved by the number of boost ports.

Making any change (either renaming and changing the meaning of the variants or changing the variants into subports) requires attention not only to the boost ports but also to all the ports that depend on them. For example, alembic and librets check which flavor of boost libraries is installed, and blender, deal.ii, freecad, htcondor, ledger, ompl, openvdb, osl, pybind11, usd, and vigra check which python variant of boost is installed. Changes would have to be coordinated so that all of these ports experience a smooth upgrade path.

Finally, why do you want static libraries? Their use is not recommended or customary on macOS.

[mohd-akram (Mohamed Akram)]

Thank you for the reply, that's a great trip through MacPorts history. Well, I was building a package that used Boost via CMake and it gave a cryptic error that it couldn't find Boost until I realized that it defaulted to looking for static Boost. While use of dynamic libraries isn't common on macOS, from what I've seen it is customary for MacPorts ports to provide both shared and static libraries, and this is part of the appeal of using ports, so I was surprised to find them missing. Static libraries are also useful for distributing a single binary.

[ryandesign (Ryan Carsten Schmidt)]

While it's true that ports have historically provided both static and dynamic libraries and we still try to do so when we can, it was originally probably more of a coincidence of the default behavior of most autotools build systems. These days, many projects have switched to other build systems like cmake, and not only is building both kinds of libraries not the default in cmake, it's not possible at all, unless you do two completely separate builds, which the cmake portgroups are not programmed to do. And when forced to choose, obviously we must choose to provide the dynamic libraries and not the static ones. So more and more ports are providing only dynamic libraries.

Distributing a single application bundle that contains dynamic libraries is not a problem. The dylibbundler tool, available in MacPorts, can help with that. If the goal is distributing a single executable file that does not reference any other library files, then it's true that static libraries would be needed to achieve that. But if this is the goal, why is this the goal?

Dynamic libraries are better because the OS can load in only those libraries that are actually being used for a particular run of the program. With static libraries, the OS sees no difference between library code and program code and has to load everything into memory every time.

[mohd-akram (Mohamed Akram)]

Thanks for pointing to dylibbundler, that should come in handy. Static builds are becoming more popular due to newer languages such as Go and Rust, where the expectation is a single binary that you can drop into /usr/local/bin, or anywhere else. But I think the simpler reason for me is that things should "just work" as much as possible. In this case, the project I was building used CMake and the Boost_USE_STATIC_LIBS flag that's in turn read by FindBoost.cmake, which comes with CMake. Yet, while both CMake and Boost in this case are provided by MacPorts and the project uses a standard and supported build flag, the build fails with a cryptic error that it can't "find" Boost, when really it can't find the static version. So I think if a port naturally supports building both dynamic and static libraries, and there is sanctioned use of the static versions in the ecosystem, we should support that.