binary package download ignores macosx_deployment_target set globally

[gallafent]

As evinced in #41589 , the binary packages appear to be built with the macports variable macosx_deployment_target set to 10.9 (or unset, and built on a 10.9 machine).

When a user has set this variable to have a different value, but is still allowing binary ports, the binary ports are downloaded even though they were built with a different value to that specified globally by the user in macports.conf. Thise leads to incompatibilities such as those mentioned above.

Possible solutions to this:

• Force buildfromsource always if macosx_deployment_target is set to other than the current system's OS version (this is very easy and has an overhead only for the user)
• Include the macosx_deployment_target as part of the request data when fetching a binary package (this is harder, and could lead to a proliferation of binary packages, one for each possible value of macosx_deployment_target

Either way, the current situation leads to ports being installed which do not honour the value set in macports.conf, which is clearly a bug.

The workaround is to add a buildfromsource always line to macports.conf then rebuild your system with sudo port -n upgrade --force installed (for example).

[gallafent]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

There are four buildslaves at this time, one each for 10.6, 10.7, 10.8, and 10.9, each building binary packages for use only on that operating system. Yes, macosx_deployment_target is unset in their macports.conf files; in general, their macports.conf files are almost completely (or likely even completely) default.

Yes, there are any number of changes the user could make on their system—from the settings in their macports.conf to the variants they use to the version of Xcode they use—which would make using the MacPorts binary packages unadvisable; you've found another one of them. If we do anything to fix this, it should be to disable binary packages when macosx_deployment_target is non-default. Or we might do nothing, since there's no particularly good or particularly well-supported reason why a user should ever change macosx_deployment_target within MacPorts.

[cooljeanius (Eric Gallager)]

Cc Me!

[gallafent]

Replying to ryandesign@…:

… there's no particularly good or particularly well-supported reason why a user should ever change macosx_deployment_target within MacPorts.

I think there are good reasons to want to build packages which can be used on older versions of Mac OS, and there's a variable defined in the Portfile reference which does appear to achieve this effect when set in the global config file with some success!

If it is really the case that this should actually not be attempted under any circumstances, I'd suggest that macosx_deployment_target should be removed as a MacPorts variable. How can it be useful in any particular portfile if it is not useful when used globally? I am not actually suggesting that this is done, though, since that would make it harder for me to do what I'm doing (!), just wondering what the use-case is that means it is still there in macports (or is its presence just historical?).

Something which catches the attempted use of binary packages when in fact the user's macports.conf means (through macosx_deployment_target or any of the other (many, as you say) configurations which would mean they “don't match”) and prevents their use (or just tells the user that she must add the buildfromsource always in order to use whichever option it is, would be useful, I think. Should I open an enhancement ticket to suggest this away from these various other reports?

[ryandesign (Ryan Carsten Schmidt)]

Replying to william@…:

I think there are good reasons to want to build packages which can be used on older versions of Mac OS, and there's a variable defined in the Portfile reference which does appear to achieve this effect when set in the global config file with some success!

If it is really the case that this should actually not be attempted under any circumstances, I'd suggest that macosx_deployment_target should be removed as a MacPorts variable.

The reason it was added as a global option is because it was requested in #19875. However it was never advertised or documented and remains an unsupported option for now. This capability is not in high demand.

How can it be useful in any particular portfile if it is not useful when used globally? I am not actually suggesting that this is done, though, since that would make it harder for me to do what I'm doing (!), just wondering what the use-case is that means it is still there in macports (or is its presence just historical?).

The variable is used in portfiles. It serves a purpose there. Certain individual ports may not be compatible with certain deployment targets or SDKs. For example, textmate2 apparently fails to build on 10.8 when the deployment target is set to 10.8, therefore the port sets "macosx_deployment_target 10.7". A very small number of ports have such problems so selective application of macosx_deployment_target is sufficient.

On the other hand, a probably much larger number of ports would have problems building for a non-default deployment target, and many ports probably ignore the setting entirely, so setting it globally to a non-default value would probably cause many problems.

Something which catches the attempted use of binary packages when in fact the user's macports.conf means (through macosx_deployment_target or any of the other (many, as you say) configurations which would mean they “don't match”) and prevents their use (or just tells the user that she must add the buildfromsource always in order to use whichever option it is, would be useful, I think. Should I open an enhancement ticket to suggest this away from these various other reports?

We already do this, for prefix, applications_dir, frameworks_dir, and portarchivetype. See archive_sites.tcl. It is not so easy to add macosx_deployment_target to this list because its default value varies by OS X version, so there is no single default value that we could put in archive_sites.tcl, like we do for the other variables.

Other macports.conf settings that would presumably influence the final product that are currently not taken into consideration when deciding whether to allow a binary download include destroot_umask, startupitem_type, amd startupitem_install.

For now, if you are changing those values, you should set "buildfromsource always" in macports.conf, as you found.

[gallafent]

Replying to ryandesign@…:

Replying to william@…:

If it is really the case that this should actually not be attempted under any circumstances, I'd suggest that macosx_deployment_target should be removed as a MacPorts variable.

The reason it was added as a global option is because it was requested in #19875. However it was never advertised or documented and remains an unsupported option for now. This capability is not in high demand.

It is documented in “man portfile” without that caveat, thus:

     macosx_deployment_target
         Value for MACOSX_DEPLOYMENT_TARGET environment variable when invoking the configure script.
         Type: optional
         Default: (current OS version)
         Example:
               macosx_deployment_target 10.4

[…]

On the other hand, a probably much larger number of ports would have problems building for a non-default deployment target, and many ports probably ignore the setting entirely, so setting it globally to a non-default value would probably cause many problems.

Understood … from my (not deep) understanding it's necessary for a port to understand how weak linking works in order to work correctly when built using an SDK for Mac OS 10.X, with a deployment target of Mac OS 10.Y, where Y < X, so ports which don't (of which I expect there are many!) may fail, where they make calls to any Mac OS system library which contains a symbol which is new or has changed between OS 10.Y and OS 10.X.

Something which catches the attempted use of binary packages when in fact the user's macports.conf means (through macosx_deployment_target or any of the other (many, as you say) configurations which would mean they “don't match”) and prevents their use (or just tells the user that she must add the buildfromsource always in order to use whichever option it is, would be useful, I think. Should I open an enhancement ticket to suggest this away from these various other reports?

We already do this, for prefix, applications_dir, frameworks_dir, and portarchivetype. See archive_sites.tcl. It is not so easy to add macosx_deployment_target to this list because its default value varies by OS X version, so there is no single default value that we could put in archive_sites.tcl, like we do for the other variables.

OK, fair enough :)

Other macports.conf settings that would presumably influence the final product that are currently not taken into consideration when deciding whether to allow a binary download include destroot_umask, startupitem_type, amd startupitem_install.

For now, if you are changing those values, you should set "buildfromsource always" in macports.conf, as you found.

Is it possible to get something added to ​http://guide.macports.org/ and/or the man pages for port / portfile / macports.conf to mention this as a way to help others out in the future? At the moment I see:

configure.macosx_deployment_target, configure.macosx_deployment_target-append, configure.macosx_deployment_target-delete

    TODO: add description

        Default: ???

        Example:

        TODO: add example

I guess this knowledge should either appear in the “buildfromsource” section of the documentation, or in that for each of the variables which can cause problems of this type when non-default.

Thanks for the detailed explanation :)

[ryandesign (Ryan Carsten Schmidt)]

Replying to william@…:

The reason it was added as a global option is because it was requested in #19875. However it was never advertised or documented and remains an unsupported option for now. This capability is not in high demand.

It is documented in “man portfile” without that caveat, thus:

     macosx_deployment_target
         Value for MACOSX_DEPLOYMENT_TARGET environment variable when invoking the configure script.
         Type: optional
         Default: (current OS version)
         Example:
               macosx_deployment_target 10.4

"man portfile" documents things that can be done in a portfile, not things that can be done in macports.conf. The caveat I mentioned above applies to the global option, not to the portfile option; the portfile option is supported and works as intended.

Is it possible to get something added to ​http://guide.macports.org/ and/or the man pages for port / portfile / macports.conf to mention this as a way to help others out in the future? At the moment I see:

configure.macosx_deployment_target, configure.macosx_deployment_target-append, configure.macosx_deployment_target-delete

    TODO: add description

        Default: ???

        Example:

        TODO: add example

That part of the documentation is out of date. The portfile option was briefly called "configure.macosx_deployment_target" before being changed to just "macosx_deployment_target" several years ago. This section documents (or fails to document) the portfile option, not the global option.

I guess this knowledge should either appear in the “buildfromsource” section of the documentation, or in that for each of the variables which can cause problems of this type when non-default.

In fact the documentation barely mentions that we have pre-compiled binaries at all, and could really use some work.