Macports should add a global "use_xcode yes" config option

[mouse07410 (Mouse)]

It would be great if /opt/local/etc/macports/macports.conf had an option such as use_xcode yes, to be able to force all of the ports installed on this system use Xcode rather than CLT.

The current behavior of turning to Xcode only when CLT is not installed is insufficient for systems that for whatever reason may need CLT, but want to use it only for certain limited cases.

[ryandesign (Ryan Carsten Schmidt)]

I still don't understand why you want this. Can you give a specific example of a problem you're having that you believe will be solved by this?

[mouse07410 (Mouse)]

One example: I want the ability to force Macports to use Xcode even when CLT is installed, without having to muck around with individual Portfiles. As you know, some ports have problems with CLT.

This global flag would allow me to keep CLT installed, and yet not impact Macports.

[ryandesign (Ryan Carsten Schmidt)]

Replying to mouse07410:

One example: I want the ability to force Macports to use Xcode even when CLT is installed,

I'm trying to understand why you want this. It should not be necessary.

As you know, some ports have problems with CLT.

No, I do not know this. The use_xcode variable exists so that ports that require Xcode can use it. If there are ports that require this to be set that haven't set it, individual bug reports should be filed for each of those ports.

[mouse07410 (Mouse)]

The use_xcode variable exists so that ports that require Xcode can use it. If there are ports that require this to be set that haven't set it, individual bug reports should be filed for each of those ports

True, but that is taking time and efforts, every time a port gets bitten by CLT incompatibility. It taxes (unnecessarily, IMHO) every port maintainer and users of that port. Plus, the time lag for discovery, time lag for discussion, time lag to incorporate the fix. All of which could be avoided if the use_xcode is added or move to (a) become global, and (b) user-controlled, rather than port maintainer-controlled.

I'm advocating for turning the control of whether to use CLT or Xcode, to user.

[kencu (Ken)]

Yes, it could be done, but it's quite a bit of work to accommodate for little value.

And it would break many prebuilt binaries for people, so they would be building everything from source if they weren't using the default CLT setup. So more complaining.

Installing the CLTs is a 2 minute effort for users, so --- not really a good tradeoff.

I know you focused on #70750 as the reason you think this would be needed, but #70750 was a once-in-a-decade thing, and it could just have easily have been an Xcode upgrade error.

[mouse07410 (Mouse)]

it would break many prebuilt binaries for people, so they would be building everything from source if they weren't using the default CLT setup

You sure? I did not realize that.

but #70750 was a once-in-a-decade thing, and it could just have easily have been an Xcode upgrade error.

Naw, in my case Xcode upgrades all succeed (though are PAINfully slow), but CLT upgrades often/usually are problematic. And have to be done via .dmg, never through Xcode-select --install.

OK, if you're correct regarding rebuilding everything from source - I'm convinced.

[kencu (Ken)]

a number of ports "bake in" the path to the SDK, and sometimes the paths to other things, into the installed build files.

Over the years we have standardized on this path to the baked-in components being into the SDK installed with the CLTs. The buildbots are building software configured that way.

It is not insurmountable to find all these baked-in spots and try to have them overrideable at runtime to the actual SDK location, or the actual location of whatever else might be baked in. But it is difficult to be sure that you've always found 100% of such instances.

Much easier to just assume that the SDKs are in the location the CLTs have put them, and go forth from there.

The actual compiler and other tools are more forgiving regarding the baked-in components, I think. But the SDK is pretty core.

[mouse07410 (Mouse)]

a number of ports "bake in" the path to the SDK, and sometimes the paths to other things, into the installed build files.

Is it truly necessary? It sounds to me like a sure path to troubles.

Over the years we have standardized on this path to the baked-in components being into the SDK installed with the CLTs.

Yeah, I understand. But wouldn't using the output of xcrun --show-sdk-path be far better?

It is not insurmountable to find all these baked-in spots and try to have them overrideable at runtime to the actual SDK location . . .

This would be the optimal solution. But I suspect it to be quite an effort. I don't think I have time or skills :-( to do that, and of course I cannot assign others to it.

Much easier to just assume that the SDKs are in the location the CLTs have put them, and go forth from there.

Yes, except that CLT tend to be broken much more often than Xcode itself (and that's quite a high bar!), and some other toolchains (Haskell, for example) can't work with it.

[ryandesign (Ryan Carsten Schmidt)]

Replying to mouse07410:

The use_xcode variable exists so that ports that require Xcode can use it. If there are ports that require this to be set that haven't set it, individual bug reports should be filed for each of those ports

True, but that is taking time and efforts, every time a port gets bitten by CLT incompatibility. It taxes (unnecessarily, IMHO) every port maintainer and users of that port. Plus, the time lag for discovery, time lag for discussion, time lag to incorporate the fix. All of which could be avoided if the use_xcode is added or move to (a) become global, and (b) user-controlled, rather than port maintainer-controlled.

I'm advocating for turning the control of whether to use CLT or Xcode, to user.

Right. What you're saying is: something in MacPorts doesn't work, and you want the freedom to fix it in such a way that only you benefit and other users don't. What I'm saying is that's not how MacPorts works. The purpose of MacPorts is that the way to build a particular software package only has to be discovered once, and that knowledge is then encoded into a portfile so that all users can successfully install it without needing to know the technical details of how it was done.

[ryandesign (Ryan Carsten Schmidt)]

Replying to mouse07410:

a number of ports "bake in" the path to the SDK, and sometimes the paths to other things, into the installed build files.

Is it truly necessary? It sounds to me like a sure path to troubles.

Quite the contrary of being necessary, we believe it is an undesirable practice. Nevertheless, many projects do this, baking not only the SDK path but often all of the other CFLAGS, CXXFLAGS, CPPFLAGS, LDFLAGS, and compiler choices that were used to build that software into the files that they install so that these choices infect anything else built by that software. See #68095 for the most recent instance I came across (yorick) but there are many others (notably perl). It is difficult for us to fix this since often this is a deliberate design decision by the designers of the software. We have tried to fix it in perl but in general our job should just be to package upstream software, not to make changes to it. It may take a considerable effort for us to convince the developers that it is a bad design and then more effort for them to redesign their ecosystem to avoid it.

Over the years we have standardized on this path to the baked-in components being into the SDK installed with the CLTs.

Yeah, I understand. But wouldn't using the output of xcrun --show-sdk-path be far better?

Unless the output of xcrun --show-sdk-path is exactly /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk (replacing 12 with your major OS version), no, that would not be better. And since we already know the exact CLT SDK path we want, there's no need to run an external program to get it, especially since xcrun does not exist on every macOS version MacPorts works on.

On my system, the output of xcrun --show-sdk-path is /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk, so no, that is not what we want baked into the badly-designed projects that bake in the SDK path, because then that software would require the full Xcode to be installed, which users have often balked at because it is so large. It also does not specify the SDK version, which means it would use the latest SDK version, and this is a frequent source of problems when compiling software not designed with an understanding of how macOS SDKs work. It is much more reliable to use the SDK version that matches the OS version. Xcode frequently only includes the latest SDK, whereas the CLT typically contains both the latest SDK and the one that matches the OS version, so this is another reason why we prefer using the CLT SDK for software that does not require Xcode.

Yes, except that CLT tend to be broken much more often than Xcode itself (and that's quite a high bar!),

I'm not aware of this.

and some other toolchains (Haskell, for example) can't work with it.

I'm not aware of this either.

[mouse07410 (Mouse)]

What you're saying is: something in MacPorts doesn't work, and you want the freedom to fix it in such a way that only you benefit and other users don't.

How does this not benefit all the users who happen to have both Xcode and CLT installed???

The purpose of MacPorts is that the way to build a particular software package only has to be discovered once, and that knowledge is then encoded into a portfile so that all users can successfully install it without needing to know the technical details of how it was done.

Meaning - you'd rather discover that a specific port fails to build with CLT and figure how to address it, even if it takes a longer time. I can understand this.

Unless the output of xcrun --show-sdk-path is exactly /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk (replacing 12 with your major OS version), no, that would not be better

Well, the main reason I made that suggestion was exactly the fact that it returns ....../MacOSX.sdk. Thus, automatically fixing the problem when Xcode include only the "next-version" SDK (e.g., on MacOS-14 it only offered MacOSX15.sdk - which worked with MacOS-14), and the CLT version of SDK (MacOSX14.x.sdk failed to work, even though it was still MacOS-14).

I'm not sure I appreciate al the implications, but off-hand I don't see why you'd rather use MacOSX12.sdk instead of MacOSX.sdk.

xcrun does not exist on every macOS version MacPorts works on

Sorry, I did not know that.

Xcode frequently only includes the latest SDK, whereas the CLT typically contains both the latest SDK and the one that matches the OS version, so this is another reason why we prefer using the CLT SDK for software that does not require Xcode

True, except that CLT often is delayed - sometimes considerably so, compared to Xcode release. And in my place timely updates (including Xcode) are mandatory (i.e., not up to me whether or when to apply them).