build shared-mime-info for deployment target 10.14 fails on macOS Sonoma

[MStraeten (Martin Straeten)]

% sudo port install shared-mime-info
--->  Computing dependencies for shared-mime-info
--->  Fetching distfiles for shared-mime-info
--->  Verifying checksums for shared-mime-info
--->  Extracting shared-mime-info
--->  Applying patches to shared-mime-info
--->  Configuring shared-mime-info
--->  Building shared-mime-info
Error: Failed to build shared-mime-info: command execution failed

:info:build ../shared-mime-info-2.3/src/update-mime-database.cpp:809:29: error: '~directory_iterator' is unavailable: introduced in macOS 10.15
:info:build         for (auto const& dir_entry : std::filesystem::directory_iterator{dir}) {
:info:build                                    ^
:info:build /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/c++/v1/__filesystem/directory_iterator.h:80:25: note: '~directory_iterator' has been explicitly marked unavailable here
:info:build   _LIBCPP_HIDE_FROM_ABI ~directory_iterator() = default;

[ryandesign (Ryan Carsten Schmidt)]

Replying to MStraeten:

:info:build ../shared-mime-info-2.3/src/update-mime-database.cpp:809:29: error: '~directory_iterator' is unavailable: introduced in macOS 10.15
:info:build         for (auto const& dir_entry : std::filesystem::directory_iterator{dir}) {
:info:build                                    ^
:info:build /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/c++/v1/__filesystem/directory_iterator.h:80:25: note: '~directory_iterator' has been explicitly marked unavailable here
:info:build   _LIBCPP_HIDE_FROM_ABI ~directory_iterator() = default;

This is correct, to the extent that the code is using std::filesystem which is only available in the version of libc++ included in macOS 10.15 and later.

The port contains code to work around this and use a newer MacPorts libc++ on older systems, but the condition for doing that is based on ${os.major}, which is the OS version of the machine doing the building, when it should probably be based on ${macosx_deployment_target} instead.

[kencu (Ken)]

There are many many many ways in which the logic in legacysupport and it's supporting ports, such as macports-libcxx, can be shown to work inadequately when building for an older deployment target.

In general, this process should be considered to not work reliably at all at present.

In other words, if you want to build a version of the software that works back to 10.14, you would best build it on a 10.14 VM, package it up as a self-contained bundle, and then test it on that system (10.14) and newer systems to see how that would work out in practice.

That is the only way to reliably have a hope of it working.

And in particular, if you try using the libc++ from macports-libcxx on newer systems that already can support filesystem in their libc++ versions, you can expect problems -- this has basically never been tested at all, in any way, and likely will break.

[kencu (Ken)]

Of course, this is imperfect. I do not anticipate any better solution coming along during this lifetime, however. The necessary code changes to legacysupport that would allow it to use fallback implementations of functions only if they are not found at runtime are certainly possible to implement -- but the amount of recoding and rearrangement of logic to allow this is frankly overwhelming and very unlikely to ever be done, barring a sponsor.

[mascguy (Christopher Nielsen)]

Replying to kencu:

There are many many many ways in which the logic in legacysupport and it's supporting ports, such as macports-libcxx, can be shown to work inadequately when building for an older deployment target.

Yes indeed, and that's the tricky part with this: Portgroup legacysupport determines the macOS version via os.major, rather than macosx_deployment_target.

And we don't want to risk changing the latter's logic - at least not in terms of the default behavior - as it could cause unforeseen issues elsewhere. So perhaps we need to add a new opt-in flag for pg legacysupport, which enables use of macosx_deployment_target.

But per Ken's point, even with such a change in place, there could still be other areas that fail to use the latter [for determining the macOS version].

So there may not be much we can do at this point, barring a more extensive audit of the various code paths.

Thoughts?

[kencu (Ken)]

If you want to deploy down to 10.14, for example, set up a 10.14 VM in Parallels (or similar), install MacPorts on it, and build and package it there.

Then you should be good to go, I think.

[ryandesign (Ryan Carsten Schmidt)]

Replying to mascguy:

Yes indeed, and that's the tricky part with this: Portgroup legacysupport determines the macOS version via os.major, rather than macosx_deployment_target.

And we don't want to risk changing the latter's logic - at least not in terms of the default behavior - as it could cause unforeseen issues elsewhere. So perhaps we need to add a new opt-in flag for pg legacysupport, which enables use of macosx_deployment_target.

Surely requiring the addition of a flag to indicate this would be terrible. MacPorts base supports changing the deployment target. Any port or portgroup that doesn't honor this is buggy and should be fixed without the addition of flags needed to indicate whether one wants the bug to be fixed.

[kencu (Ken)]

legacysupport doesn't use os.major at all, nor are there any places where it would be useful to substitute macosx_deployment_target.

It uses __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ for it's testing, which would be properly set by setting the deployment target when building legacysupport.

The problem I see is how the replacements work ... making sure that when building something against legacysupport on a system that HAS the symbols, to have them link against and use the ones in legacysupport instead so that when deployed on a system that doesn't have the symbols, something will be found.

And this is really not right either -- you would want the legacysupport library to only use the replacement symbols when needed, and use the system symbols if they exist, which means runtime testing, not build-time testing.

IE -- a big project, to do it right, and not something I'm prepared to do, certainly.

And taking into account the fact that 99% of all open source software also does not properly handle the case of building on a new system to deploy on an old system -- that is why I say just build the damned software on a VM of your desired lowest target, and be done with it ...

[lukaso (Lukas Oberhuber)]

And taking into account the fact that 99% of all open source software also does not properly handle the case of building on a new system to deploy on an old system -- that is why I say just build the damned software on a VM of your desired lowest target, and be done with it ...

I know this is an old one, but intel VMs do not run on Apple Silicon and intel hosts are being phased out by Circle CI. If only it were so easy to find a place to run the 10.14 VM.

[kencu (Ken)]

Parallels runs Intel VMs on Apple silicon. qemu runs everything on Apple silicon (I am running Tiger PPC on a qemu VM on my M1 Mac right now).

Do you know what software is available to run a VM on your CI system?