base: ports cannot use SDK inside of Developer dir unless using Xcode build type

[Ionic (Mihai Moldovan)]

Currently, base only allows access to the SDK directory to ports that use the Xcode build type or explicitly state that they use Xcode or the SDK directory lies outside of the Developer directory.

This behavior, though, is too restrictive.

One example of this failing badly is py35-pyqt5:

:info:configure Error: Failed to determine the detail of your Qt installation. Try again using
:info:configure the --verbose flag to see more detail about the problem.
:info:configure Querying qmake about your Qt installation...
:info:configure Determining the details of your Qt installation...
:info:configure /opt/local/libexec/qt5/bin/qmake -o cfgtest_QtCore.mk cfgtest_QtCore.pro
:info:configure Project ERROR: Could not resolve SDK Path for 'macosx10.9'

That's likely not isolated to this port, but any port using the qmake5-1.0 PG or actually even qmake(5) directly.

Base should allow access to the SDK dir if configure.sdk_version is set, regardless of whether configure.sdkroot is the default/set to an empty value (and hence relative to the Developer dir) or not.

[jmroot (Joshua Root)]

I thought this was what [6b27854906b3c19d209c50ff59a4fa78d42f3a03/macports-base] was trying to fix, but I guess more is needed.

[Ionic (Mihai Moldovan)]

This behavior might be a peculiarity of qmake(5). qmake figures out the SDK directory itself, probably in a hardcoded fashion and does not use information provided by MacPorts to do this (other than selecting the SDK version).

I've looked a bit further and noticed that portconfigure::configure_get_sdkroot returns an empty string if the SDK version passed in (which should be configure.sdk_version) matches the running macOS version and /usr/include exists, so getting the actual SDK location will turn out to be a bit difficult.

[6b27854906b3c19d209c50ff59a4fa78d42f3a03/macports-base] only set use_xcode to true if CLTs are not installed, but that's not relevant in this case.

Arguably, I guess we could tell qmake to use / as the SDK root if configure.sdkroot turns out to be empty... but do we really want this?

[jmroot (Joshua Root)]

Replying to Ionic:

[6b27854906b3c19d209c50ff59a4fa78d42f3a03/macports-base] only set use_xcode to true if CLTs are not installed, but that's not relevant in this case.

If you install both Xcode and the CLT then you won't have /Library/Developer/CommandLineTools/usr/bin/make, or at least I don't. If there is a case where that exists but the SDK inside Xcode is still used, that would be a problem.

Arguably, I guess we could tell qmake to use / as the SDK root if configure.sdkroot turns out to be empty... but do we really want this?

That would match what is done elsewhere. Using the headers in /usr/include is safer than using the SDK in Xcode because the latter could be for a newer OS version, which causes problems with build-time detection of features.

[Ionic (Mihai Moldovan)]

If you install both Xcode and the CLT then you won't have /Library/Developer/CommandLineTools/usr/bin/make, or at least I don't. If there is a case where that exists but the SDK inside Xcode is still used, that would be a problem.

That doesn't seem to hold for 10.9 for instance. It's entirely possible that such setups only exist in older systems nowadays, but I have both CLT and Xcode installed and the paths all exist.

That would match what is done elsewhere. Using the headers in /usr/include is safer than using the SDK in Xcode because the latter could be for a newer OS version, which causes problems with build-time detection of features.

Yeees, since Apple doesn't necessarily ship the needed SDK versions with all supported Xcode versions any longer. But also back then, using the provided SDKs was safer pre-SIP.

Not allowing access to the Developer dir isn't just problematic because of the SDKs, though. In #58770 I reported a build failure that I "fixed" by setting use_xcode to true in the port, but that workaround is wrong and stupid. Nothing within scons or gpsd actually uses xcrun - the call is a side-effect of the shims in /usr/bin (for clang for instance). In theory, we'd have to allow access to the Developer dir every time we use the system compiler as well.

[satraul (Satryaji Aulia)]

Replying to Ionic:

If you install both Xcode and the CLT then you won't have /Library/Developer/CommandLineTools/usr/bin/make, or at least I don't. If there is a case where that exists but the SDK inside Xcode is still used, that would be a problem.

That doesn't seem to hold for 10.9 for instance. It's entirely possible that such setups only exist in older systems nowadays, but I have both CLT and Xcode installed and the paths all exist.

Yes, the existence of /usr/lib/libxcselect.dylib (xcode-select) indicates macOS >= 10.9 which we assume is the earliest macOS version that moved CLT from / to /Library/Developer/CommandLineTools/.

For < 10.9 we agreed that using CLT alone isn't sufficient enough so we always set use_xcode yes.

Not allowing access to the Developer dir isn't just problematic because of the SDKs, though. In #58770 I reported a build failure that I "fixed" by setting use_xcode to true in the port, but that workaround is wrong and stupid. Nothing within scons or gpsd actually uses xcrun - the call is a side-effect of the shims in /usr/bin (for clang for instance). In theory, we'd have to allow access to the Developer dir every time we use the system compiler as well.

Thank you for the discovery, this is actually part of my project right now for MacPorts. Around 2 days ago, we discovered the qmake problem on Qt5, and planned a solution similar to #58770. And yes, the solution is a bad workaround.

As you said, qmake does find the SDK on it's own by using xcrun here: ​https://code.qt.io/cgit/qt/qtbase.git/tree/configure#n235 with $sdk resolving to configure.sdk_version.

I just found out that there are many ports (~150) that depend on qmake, so I think we should consider a larger solution. Ccing my project mentors.

The #58770 problem needs to be discussed seperately too. There may be more ports that call shims without MacPorts' environment variables. First with ports using system -W, now with scons (~26 ports).

[mouse07410 (Mouse)]

There's one problem. I have Xcode-10.3 installed with CLT, but it did not install include files into /use/include.

[Ionic (Mihai Moldovan)]

Yes, the existence of /usr/lib/libxcselect.dylib (xcode-select) indicates macOS >= 10.9 which we assume is the earliest macOS version that moved CLT from / to /Library/Developer/CommandLineTools/.

Which is fine... I mean, we do want to get rid of needing the full Xcode suite, hence your GSoC project.

The current approach fails if both CLT and Xcode are installed, though, at least for older OS versions.

Let me quote from #58770:

:info:build /usr/bin/clang -arch x86_64 -o bits.os -c -Os -O2 bits.c
:info:build xcrun: error: invalid active developer path (/Applications/Xcode.app/Contents/Developer), missing xcrun at: /Applications/Xcode.app/Contents/Developer/usr/bin/xcrun

What essentially happened here, as far as I can tell, is that the system clang binary was invoked, which... is actually a shim at /usr/bin/clang. This shim uses libxcselect.dylib, which then probably calls xcrun to get the actual tool location.

With Xcode installed, xcode-select -p returns a Developer dir path of /Applications/Xcode.app/Contents/Developer. Since tracemode hides that, xcrun won't be able to see the xcrun binary bundled in there, assume that that path is wrong and fail to execute tools.

I see two viable options:

1. Always allow access to the Xcode-based Developer directory if it exists. That probably wouldn't be too bad since it's what base has always done so far. If that directory doesn't exist, xcode-select -p SHOULD use /Library/Developer/CommandLineTools anyway so whitelisting a non-existent path shouldn't cause trouble.
2. Override the Developer dir path via the DEVELOPER_DIR environment variable if we don't want to use Xcode (i.e., if use_xcode is no). Example:

   root@nopileos~#  xcrun --sdk macosx10.9 -f clang
   /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
   root@nopileos~#  DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun --sdk macosx10.9 -f clang
   /Library/Developer/CommandLineTools/usr/bin/clang
   

Option two would mimic your original intention more closely, but requires deeper changes to base code since we'd now always have to set DEVELOPER_DIR if using Xcode is not explicitly requested.

---

I just found out that there are many ports (~150) that depend on qmake, so I think we should consider a larger solution. Ccing my project mentors.

Well, ports that use qmake(5) usually also use the qmake(5)-1.0 PortGroups, so it would be easy to just change the two as a workaround. There are some ports that don't use the PGs, but they probably should do so anyway.

This said, we probably don't want to use yet another workaround anyway. Such a workaround would essentially use option one and be put into every port (or PortGroup) that uses the system clang compiler, which we also do by default most of the time. Not really a good option.

---

There's one problem. I have Xcode-10.3 installed with CLT, but it did not install include files into /use/include.

Uh, that sounds really, really odd. Not having /usr/include would essentially make CLT useless. Xcode bundles SDKs anyway which include the command line tools, but a standalone CLT package without the proper header files would defeat the whole point of this exercise.

Do you have the shims in /usr/bin (e.g., /usr/bin/clang)?

Did they move the headers around by any chance? /Library/Developer/CommandLineTools/usr/include exists traditionally (on my 10.9 machine at least), but that only ships some C++ headers as far as I can tell. Not sure why they wouldn't have been installed in /usr/include as well together with all the other stuff, but I guess it's because they are specifically used by clang. /usr/include/c++ includes a 4.2.1 directory, which smells like Apple-GCC at first, but:

root@nopileos~#  DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun g++ --version 
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix

Funnily enough, clang seems to be lying:

root@nopileos~#  DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun g++ -E -x c++ - -v < /dev/null
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix
 "/Library/Developer/CommandLineTools/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.9.0 -E -disable-free -disable-llvm-verifier -main-file-name - -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu core2 -target-linker-version 241.9 -v -resource-dir /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.0 -stdlib=libc++ -fdeprecated-macro -fdebug-compilation-dir /var/root -ferror-limit 19 -fmessage-length 224 -stack-protector 1 -mstackrealign -fblocks -fobjc-runtime=macosx-10.9.0 -fencode-extended-block-signature -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -vectorize-slp -o - -x c++ -
clang -cc1 version 6.0 based upon LLVM 3.5svn default target x86_64-apple-darwin13.4.0
ignoring nonexistent directory "/usr/include/c++/v1"
#include "..." search starts here:
#include <...> search starts here:
 /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1
 /usr/local/include
 /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.0/include
 /Library/Developer/CommandLineTools/usr/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 175 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

I also noticed that one should better not specify --sdk when overriding DEVELOPER_DIR to the CLT location, because more fun stuff seems to happen in that case:

root@nopileos~#  DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun --sdk macosx10.9 g++ --version                 
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=macosx10.9/usr/include/c++/4.2.1
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix

... which, again, is a lie:

root@nopileos~#  DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun --sdk macosx10.9 g++ -E -x c++ - -v < /dev/null
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix
 "/Library/Developer/CommandLineTools/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.9.0 -E -disable-free -disable-llvm-verifier -main-file-name - -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu core2 -target-linker-version 241.9 -v -resource-dir /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.0 -stdlib=libc++ -fdeprecated-macro -fdebug-compilation-dir /var/root -ferror-limit 19 -fmessage-length 224 -stack-protector 1 -mstackrealign -fblocks -fobjc-runtime=macosx-10.9.0 -fencode-extended-block-signature -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -vectorize-slp -o - -x c++ -
clang -cc1 version 6.0 based upon LLVM 3.5svn default target x86_64-apple-darwin13.4.0
ignoring nonexistent directory "/usr/include/c++/v1"
#include "..." search starts here:
#include <...> search starts here:
 /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1
 /usr/local/include
 /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.0/include
 /Library/Developer/CommandLineTools/usr/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 175 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

Also note how clang doesn't show the "Configured with:" line at all when invoking it as "clang++", so that output can probably be mostly ignored:

root@nopileos~#  DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun --sdk clang++ --version
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix

Sometimes it feels like all of this stuff is held together by band aid.

[satraul (Satryaji Aulia)]

Replying to Ionic:

2. Override the Developer dir path via the DEVELOPER_DIR environment variable if we don't want to use Xcode (i.e., if use_xcode is no). Example:

This is what we're implementing now with the new variable configure.developer_dir which is set to CLT when Xcode isn't requested. So we're now actually exporting DEVELOPER_DIR to an extent: [31e0c5619836ebfc23c812b2d9f65d3c98938efa/macports-base]

But it turns out that wasn't enough because things like build { system -W /usr/bin/clang } was ignoring this change, so it was solved with this: [e09517b2ebf0ca18cd7b6e66ac3ffafba48296b3/macports-base]

So with that change, I expected that it would solve this problem too: when qmake runs xcrun to find SDK, it should have DEVELOPER_DIR set thus finding CLT's SDK. I'm not sure why this isn't the case.

---

I just found out that there are many ports (~150) that depend on qmake, so I think we should consider a larger solution. Ccing my project mentors.

Well, ports that use qmake(5) usually also use the qmake(5)-1.0 PortGroups, so it would be easy to just change the two as a workaround. There are some ports that don't use the PGs, but they probably should do so anyway.

This said, we probably don't want to use yet another workaround anyway. Such a workaround would essentially use option one and be put into every port (or PortGroup) that uses the system clang compiler, which we also do by default most of the time. Not really a good option.

Right, it's nice to know that we still have that workaround just in case.

Thank you for the explanations, I've understood a bit more (or less) about what's happening.

[neverpanic (Clemens Lang)]

FWIW, I just did a trace mode build of py37-pyqt5 on 10.12 with Xcode 9.2 and the CLTs installed with the following patch applied to the qt5-qtbase Portfile, and it passed:

aqua/qt5/Portfile

@@ -987 +987 @@
                 if { ![file exists ${sdks_dir}/MacOSX${configure.sdk_version}.sdk] } {
                     configure.sdk_version
                 }
+                if {${developer_dir} ne ${configure.developer_dir} && ![file exists ${configure.developer_dir}/SDKs/MacOSX${configure.sdk_version}.sdk]} {
+                    configure.sdk_version
+                }
             }
 
             # respect configure.sdk_version

If that doesn't work on 10.9, we should fix what's wrong there, but in general, our approach seems to work.

[jmroot (Joshua Root)]

Replying to Ionic:

There's one problem. I have Xcode-10.3 installed with CLT, but it did not install include files into /use/include.

Uh, that sounds really, really odd. Not having /usr/include would essentially make CLT useless. Xcode bundles SDKs anyway which include the command line tools, but a standalone CLT package without the proper header files would defeat the whole point of this exercise.

That's actually normal with Xcode 10+. MacPorts has been able to use the SDK installed by the CLTs for some time.

[Satryaji Aulia <satraul@…>]

In dffe6b4ea1a59396ace818378594893dc78ceafd/macports-ports (master):

qt5: account for configure.developer_dir

References #58779

[Ionic (Mihai Moldovan)]

Wait, that change will always clear ${configure.sdk_version}! Did you mean to use ${developer_dir} (or ${configure.developer_dir}) instead of ${configure.sdkroot}? The sdkroot variable already contains the SDK path.

[Ionic (Mihai Moldovan)]

Neither change won't fix builds on 10.9 and below anyway. 10.9 for instance can at most use the qt58 ports, newer versions are not supported. Clearing ${configure.sdk_version} is also only done when using Xcode 7+, which is only available on 10.10 and higher.

Currently, building the port still fails, because:

• ${configure.sdkroot} is set to an empty string (because ${configure.sdk_version} matches the running OS version and /usr/include exists)
• access to the directory containing SDKs (Xcode) is denied
• qmake can't resolve the "macosx" SDK location

CLT on older platforms do not ship any SDK, so we're forced to either use the SDKs provided by Xcode or no SDK/sysroot at all.

If I clear ${configure.sdk_version}, this happens:

• ${configure.sdkroot} is still empty, because it tries to go down the !use_xcode route, executes env DEVELOPER_DIR=${cltpath} xcrun --sdk macosx${sdk_version} --show-sdk-path 2> /dev/null which essentially means calling env DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun --sdk macosx --show-sdk-path 2> /dev/null. This outputs an empty string and does NOT return an error code != 0, so base thinks the command succeeded and takes the empty string as the SDK path...
• same as above

Sadly, qmake-based ports always want to use an SDK. I've tried hacking around setting CONFIG-=sdk, but that only led to yet another build error due to Xcode being unusable. It also tried to use a compiler symlink in ${prefix} opportunistically, so that's not a great idea anyway.

I don't think we can phase out the Xcode dependency on systems with Xcode 6 and below, since Xcode is the only package providing SDKs there. Disallowing the usage of SDKs (on older system) would solve that problem, but likely create a myriad of new ones (see qmake-based ports). Also, technically, trace mode will need to always allow access to the selected SDK root on older platforms. What doesn't help is that we'd essentially need a crystal ball to foresee whether a port will use an SDK or not - again, see qmake-based ones. ${configure.sdkroot} will be empty (because base assumes that / will be used and no SDK is necessary), but the port actually wants to use an SDK. That might be a good example for the qmake5-1.0 PG having to set use_xcode yes on older platforms, but that would likewise just work around a base shortcoming.

[satraul (Satryaji Aulia)]

My bad, I've confused the two. I submitted it because it built qt5-qtbase successfully. But let me get this straight first before submitting a fix.. so this line

                if { ![file exists ${sdks_dir}/MacOSX${configure.sdk_version}.sdk] } {
                    configure.sdk_version
                }

Clears configure.sdk_version? What is the purpose of this? In my understanding, it's checking to see if current version's SDK doesn't exist and if so "clear" it (which I assume is so qt can find the SDK by itself).

In Clemens' patch, to avoid running (clearing) configure.sdk_version:

1. Current version's SDK must be in Xcode's folder
2. If MacPorts is using CLT as Developer_Dir, current version's SDK must be in CLT folder

Why should we have both conditions met? Please correct my understanding.

[Ionic (Mihai Moldovan)]

This snippet clears ${configure.sdk_version}, yes. And yes, it does so if the selected SDK version doesn't exist. Qmake is special and will use the default "macosx" SDK value in that case, i.e., figure out which SDK to use.

Clemens's patch is fine. He's checking whether the developer dir has been determined as the CLT dir (i.e., is not the [default] Xcode developer dir, because use_xcode is false and Xcode should hence be avoided) and clears out ${configure.sdk_version} if the requested SDK (relative to determined developer_dir) does not exist.

My original attention call went out because of this line: if { ![file exists ${configure.sdkroot}/MacOSX${configure.sdk_version}.sdk] } { This will always be true, because ${configure.sdkroot} will either be empty (in which case a file like /MacOSX${configure.sdk_version}.sdk will not exist) or set to an SDK root directory, such as /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX${configure.sdk_version}.sdk or /Library/Developer/CommandLineTools/SDKs/MacOSX${configure.sdk_version}.sdk. Coupled with yet another set of MacOSX${configure.sdk_version}.sdk, such a directory will not exist. You're currently checking for, e.g., /Library/Developer/CommandLineTools/SDKs/MacOSX${configure.sdk_version}.sdk/MacOSX${configure.sdk_version}.sdk, which naturally won't exist.

[satraul (Satryaji Aulia)]

Replying to Ionic:

My original attention call went out because of this line: if { ![file exists ${configure.sdkroot}/MacOSX${configure.sdk_version}.sdk] } { This will always be true, because ${configure.sdkroot} will either be empty (in which case a file like /MacOSX${configure.sdk_version}.sdk will not exist) or set to an SDK root directory, such as /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX${configure.sdk_version}.sdk or /Library/Developer/CommandLineTools/SDKs/MacOSX${configure.sdk_version}.sdk. Coupled with yet another set of MacOSX${configure.sdk_version}.sdk, such a directory will not exist. You're currently checking for, e.g., /Library/Developer/CommandLineTools/SDKs/MacOSX${configure.sdk_version}.sdk/MacOSX${configure.sdk_version}.sdk, which naturally won't exist.

Okay, thanks for explaining. So it seems we can't find a clear solution to always use CLT for qt5 then. If I get it right: even if qmake could use CLT's SDK on <=10.9 there is difficulty in doing so, such as xcrun not behaving correctly as you pointed out. I initially thought that the solution was by hacking how to call qmake so it could resolve correctly to macosx, but that is now ruled out.

I'll wait for my mentors to chime in, but I guess the acceptable fix is to set use_xcode for qt5 temporarily for now, right? Thanks again.

Also: is there an easy way of testing across macOS versions? Would be nice to know

[Ionic (Mihai Moldovan)]

So it seems we can't find a clear solution to always use CLT for qt5 then.

If it only used the tools suite, that would have been possible. However, since Qt5 (and likely also Qt4) heavily use SDKs, it's impossible to find a generic way, since SDKs are only installed by Xcode on older platforms. I don't have a 10.10 machine that I could look into, but 10.9 only ships SDKs as part of Xcode, while 10.11 has an SDK embedded in the CLT. I suspect that CLTs started shipping SDKs with Xcode 7 (denoting 10.10), but I'm not sure. There is another slight problem in that the SDK shipped by CLT on the 10.11 box is one for 10.12, but that's a known problem (and SHOULD, theoretically, not cause problems if the deployment target is set correctly).

If I get it right: even if qmake could use CLT's SDK on <=10.9 there is difficulty in doing so, such as xcrun not behaving correctly as you pointed out. I initially thought that the solution was by hacking how to call qmake so it could resolve correctly to macosx, but that is now ruled out.

That's (part of) the problem: CLT doesn't ship an SDK on 10.9-. qmake expects one, though, and won't be happy if it can't resolve the SDK value it's looking for. I tried disabling its SDK usage, but that only led to some other error. The reason for that is that qmake tries to look for the xcodebuild binary, c.f., ${prefix}/libexec/qt5/mkspecs/features/mac/default_pre.prf. With DEVELOPER_DIR set to the CLT directory, that call fails. I could work around that failure by setting $QMAKE_XCODE_DEVELOPER_PATH (qmake/env variable) directly/explicitly to ${configure.developer_dir} in order to skip the xcodebuild check, but that would only lead to the build failing later AGAIN, because it would go on to auto-detect the Xcode version via /usr/bin/xcodebuild -version. That call fails if DEVELOPER_DIR is set to the CLT developer dir.

I wonder how that worked for neverpanic? Well, ​here's the explanation: they refactored the code to NOT check for xcodebuild when looking for the developer dir, split it out and made it non-fatal and they stopped checking for the Xcode version if the xcodebuild path is empty. This stands in stark contrast to ​code in older versions of qt5base.

I'll wait for my mentors to chime in, but I guess the acceptable fix is to set use_xcode for qt5 temporarily for now, right?

It would be a workaround. The better fix would be to backport the changes that make Xcode usage optional to older Qt5 versions (since, like previously said, Qt5 has a habit of deprecating macOS versions quite aggressively, so that older platforms are forced to use older, unmaintained Qt5 versions). I don't know how much effort that would be, though. There were definitely changes to that one file, which are crucial, but we might need even more changes to make it work correctly (see opportunistic compiler usage, which is not what we want).

Also: is there an easy way of testing across macOS versions? Would be nice to know

Not aware of any way. I have two VMs as build hosts for some other project, currently running 10.11 and 10.13, and my old MBP, running 10.9. Theoretically, it also includes a 10.6 VM, but that is hopelessly outdated by now. Haven't started it in a year or even longer.

CLTs should be freely downloadable through the developer portal, though. Even if one doesn't intend to install them, I think that getting the file list is still possible by executing the installer and using the "Show Package contents" option. It's not a comfortable way and actually extracting this stuff would be more difficult. The same applies more or less to Xcode. Note that such packages are HUGE.

[satraul (Satryaji Aulia)]

In 2b2840e6a55e8be3c6e97790d9b3eef0eaef1277/macports-ports (master):

qt5,qmake: fix account for CLT SDK and set use_xcode

References #58779

[Ionic (Mihai Moldovan)]

Fix grammar in original issue report.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

In 63e0613f9f371d8319aa50ef1b08c080a631bda2/macports-ports (master):

qmake5 PG: require Xcode if SDKs not provided by CLT

See #58779

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Between the ​two ​commits, I think this ticket can be marked fixed.

Please reopen if problems persist.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

In 64c4060aebef6ffcabfd8142b34a1cc88f283e35/macports-ports (master):

gmic: require Xcode if SDKs not provided by CLT

Fixes #59677
See #58779

[kencu (Ken)]

Are we meant to implement this fix on a port-by-port basis whenever we run across it, eg #59157, or is there a way to take care of all of these issues once and for all in base?

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

I am not sure about other instances of this error.
However, this ticket is specifically an error with qmake.
It is fixed in the qmake PG
It is also required in the Qt5 ports and gmic, which use qmake.

If we are to make changes in the base code, I am not 100% sure how to generalize to all ports.

[kencu (Ken)]

I would guess this test in portmain.tcl 6b27854906b3c19d209c50ff59a4fa78d42f3a03/macports-base might benefit from incorporating something further regarding the fix in this ticket.

Would that also fix #59157 (and probably others that have not been rebuilt since base 2.6 came out)?

default use_xcode {[expr {[option build.type] eq "xcode" || ![file exists /usr/lib/libxcselect.dylib] || ![file executable [file join $cltpath usr bin make]]}]}

should be?:

default use_xcode {[expr {[option build.type] eq "xcode" || ![file exists ${developer_dir}/SDKs] || ![file exists /usr/lib/libxcselect.dylib] || ![file executable [file join $cltpath usr bin make]]}]}

[kencu (Ken)]

In a754c7e1041bbebc7eacbe343db9a42a8178566d/macports-ports (master):

clang-3.7: needs use_xcode to build

see #59677
see #58779
see #60196