qt511-qtimageformats @5.11.3_2: Could not resolve SDK Path for 'macosx10.11' using --show-sdk-path

[thetrial (alabay)]

Again, this error occurs when trying to rebuild due to an update of ghostscript and ImageMagick. I’ll attach the logfile. Xcode is 8.2.1.

[thetrial (alabay)]

Addition: I installed the port MacOSX10.11.sdk, hoping that would solve this. It did not.

xcrun --show-sdk-path

still gives

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk

--sdk macosx10.12 --show-sdk-version

gives

10.12

--sdk macosx10.11 --show-sdk-version

gives

xcodebuild: error: SDK "macosx10.11" cannot be located.
xcodebuild: error: SDK "macosx10.11" cannot be located.
xcrun: error: unable to lookup item 'SDKVersion' in SDK 'macosx10.11'

What’s wrong?

[kencu (Ken)]

you’re on 10.11 but the sdk is 10.12

the portgroup logic is supposed to fall back to the generic sdk in this case, but it thinks it’s finding the 10.11 sdk for some reason when it runs it’s checks…

105	:debug:configure qt5 Portfile: the initial SDK value is: macosx10.11
106	:debug:configure qt5 Portfile: testing for system-specific SDK:
107	:debug:configure qt5 Portfile: system-specific SDK was found.
108	:debug:configure qt5 Portfile: the final SDK value is: macosx10.11

[kencu (Ken)]

have you made any sdk symlinks or otherwise modified the sdk folder in any way?

if not, the testing logic is somehow broken for your case…

[thetrial (alabay)]

No symlinks anymore. I just installed MacOSX10.11.sdk through MacPorts ad mentioned. But that does not help. So now is the question … can that be included or is it for nothing? And yes, it seems something is broken. These sdk things occur quite often, with several ports. I had it sometimes, others have it sometimes.

[kencu (Ken)]

What did you install exactly? MacPorts has no committed, debugged, ability to install an SDK. Where did you put it? If you remove it, does it work again?

Now that you have a machine that errors, you are in a unique position to debug a fix for it I would say.

All my systems work, and as far as I know, all the buildbots too.

[kencu (Ken)]

I'm sorry, I was wrong. I missed that a MacOSX10.11.sdk port was added. Sorry.

[kencu (Ken)]

could you check to see if you have the MacOSX10.11.sdk in both or either of:

/Library/Developer/CommandLineTools/SDKs

and

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/

I'm not really sure how things would work out if you had it in only one or the other place, but not both places.

[thetrial (alabay)]

MacPorts installs the sdk to /opt/local/Developer/SDKs/ – without comment. I tried to link MacOSX10.11.sdk to /Library/Developer/CommandLineTools/SDKs and /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ – but that does not help. xcrun --show-sdk-path points only to 10.12.

It's a bit annoying – I have xcode 8.2.1, the »regular« xcode für 10.11. That one has the sdk pointing to 1.12 instead 10.11. It seems, an older 7.3 points to 10.11. But I use 8.2.1 from the beginning on. And it is strange that a) the ports can’t handle that and b) the (via MacPorts) installed sdk does not have an effect.

[thetrial (alabay)]

Seems to be a known issue, but I don’t see a solution:

​https://developer.apple.com/forums/thread/62870

[kencu (Ken)]

this has to do with the way we wrote the tests to see which sdks you have installed. They should cover your case.

please show me the exact contents of the two sdk folders I referenced above.

[kencu (Ken)]

the testing code has been rewritten a few times.

​https://github.com/macports/macports-ports/blob/master/aqua/qt511/Portfile#L987

it seems the current version of it isn’t working for your setup. I can see a few potential possible issues with it, but to debug it we’ll need to know what sdks are in your sdks folders.

[thetrial (alabay)]

Both folders have MacOSX.sdk as folder and MacOSX10.12.sdk as link to that folder inside of them.

[kencu (Ken)]

OK, then that is supposed to work. The portfile logic should find that using macosx10.11 fails, and so falls back to use just macosx instead.

Josh last rewrote the code, so I’ll loop him in here to help sort it out.

[thetrial (alabay)]

So the 10.11 sdk that I installed via MacPorts is irrelevant? I thought, this could solve problems. What for was/is this port?

[kencu (Ken)]

Yeah, the new SDK you installed is only useful if software knows where to look for it.

If the MacOSX10.11.sdk was in both:

/Library/Developer/CommandLineTools/SDKs

and

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/

then you should be in business. You might try playing around with symlinking the MacOSX10.11.sdk you installed into those two locations, just for fun, I guess.

There was a lot of stuff added to the testing logic that I'm not up on (configure.developer_dir, etc) so I will leave the rest of this to Josh.

[thetrial (alabay)]

I tried to symlink this sdk into the two locations, namely to a homonymous folder – but that did not work. I’m not sure how the mechanism of --show-sdk-path works. Perhaps not with simple detection but with switches somewhere in a plist.

If MacPorts offers to install additional sdks it would be nie if there was a postflight script, adding the necessary elements to use these sdks when they are needed.

I’m really not sure if I should not give up the adequate macOS el capitan xcode 8.2.1, get rid of it and install xcode 7.3. This would be a strange and maybe big step, but these problems are annoying. Though it worked until this last stumble. But my revbump hangs.

[jmroot (Joshua Root)]

The silly thing about all this is that you don't even need to use an SDK on < 10.14; all the headers and libs are present under / (i.e. /usr/include for headers) if you have the CLTs installed. I think that's the logic behind Apple not including a 10.11 SDK with Xcode 8 or the CLTs.

Downloading Xcode 7.3.1 and copying the 10.11 SDK out of the app bundle and into /Library/Developer/CommandLineTools/SDKs does seem to work, at least for the purposes of env DEVELOPER_DIR=/Library/Developer/CommandLineTools xcrun --sdk macosx10.11 --show-sdk-path. However it seems like it would be preferable to patch whatever needs to be patched to convince the build system that the SDK flags can simply be omitted on these older OS versions.

[kencu (Ken)]

the real issue is the current portfile testing is not working right, and shows he has the 10.11 sdk when he doesn’t have it:

105	:debug:configure qt5 Portfile: the initial SDK value is: macosx10.11
106	:debug:configure qt5 Portfile: testing for system-specific SDK:
107	:debug:configure qt5 Portfile: system-specific SDK was found.
108	:debug:configure qt5 Portfile: the final SDK value is: macosx10.11

so he doesn’t properly fall back to “macosx” any longer.

[kencu (Ken)]

I could have another go at fixing that testing code, as I have the 10.11 system right in front of me, but I thought I would better leave it to you, as you just took it all apart and put it together again, and I don’t want to step on your toes about fixing it.

Or we might just always use “macosx” and forget about testing for the system-specific sdk if it’s so hard to get that right.

[jmroot (Joshua Root)]

Saying I rewrote this code is giving me a bit too much credit I think; all I did was a fix a couple of bugs, the basic logic is as you wrote it in [fbdae6e6c2cc9c2a8b8a625e322aeee6bd00e610/macports-ports]. I don't have any special insight into the behaviour of xcrun on 10.11. But I do strongly recommend using the equivalent of no SDK flags or -isysroot / if at all possible, because that's what we do everywhere else on the systems where it's possible.

[thetrial (alabay)]

So shall I wait and not delete Xcode 8.2? That would be the best solution indeed, as 8.2 is the typically given last Xcode under 10.11 (I guess).

But, what for are the MacPorts’ given SDKs then? I guess I can remove that afterwards, can I? It does not aid in problematic situations like this.

[kencu (Ken)]

I’ll fix this in time, but the SDK symlinks should have worked for you…

I also use Xcode 8.2 on 10.11, but I physically copied the 10.11 SDK into the two required locations, and that works 100% of the time for me, including with this port.

[thetrial (alabay)]

Hm … the symlinks did not work … though, did I symlink? Maybe I just linked via Finder. I now copied the sdk physically as you said – and the rev-upgrade ran through. Oh. Nice. Now I don’t know if that also would have worked with symlinks, I’ll try that on another machine next year with ln -s before.

[kencu (Ken)]

It looks like adding the DEVELOPER_DIR=${configure.developer_dir} part was the culprit.

On a system that does not have MacOSX10.11.sdk in either SDKs folder, we see this:

$ ls /Library/Developer/CommandLineTools/SDKs
MacOSX.sdk	MacOSX10.12.sdk

$ ls /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/
MacOSX.sdk	MacOSX10.12.sdk

$ /usr/bin/xcrun --sdk macosx10.11 --find ld
xcodebuild: error: SDK "macosx10.11" cannot be located.
xcrun: error: unable to find utility "ld", not a developer tool or in PATH

DEVELOPER_DIR=/Library/Developer/CommandLineTools /usr/bin/xcrun --sdk macosx10.11 --find ld
/Library/Developer/CommandLineTools/usr/bin/ld

We want to know if the MacOSX10.11.sdk exists, and adding the DEVELOPER_DIR part makes the test succeed in finding "ld" even if the SDK is not there.

So we have to remove that from everywhere it was added, and we should be good.

and that initially seems to work:

DEBUG: qt511 Portfile: the initial SDK value is: macosx10.11
DEBUG: qt511 Portfile: testing for system-specific SDK:
DEBUG: qt511 Portfile: system-specific SDK was not found, looking for generic SDK.
DEBUG: qt511-qtbase: using generic macosx SDK as macosx10.11 was not found
DEBUG: qt511 Portfile: the final SDK value is: macosx

however it comes around again for testing using the block of code in the qmake5 PortGroup, and that now fails to work properly, for no good reason I can see at present:

DEBUG: qmake5 Portfile: the initial SDK value is: macosx10.11
DEBUG: qmake5 Portfile: testing for system-specific SDK macosx10.11:
DEBUG: qmake5 Portfile: system-specific SDK was found.
DEBUG: qmake5 Portfile: the final SDK value is: macosx10.11

[kencu (Ken)]

Oh, the second check is possibly failing as MacPorts is setting DEVELOPER_DIR='/Library/Developer/CommandLineTools' in the environment, I bet...

[kencu (Ken)]

Well, now this is getting to be a major PITA, with the testing being influenced now by the setting of DEVELOPER_DIR.

I personally am just about ready to abandon this plan completely, and just clear configure.sdk_version in all the qt5 and qmake5 PortGroups and Portfiles, and let the port build with --sdk macosx all the time, and be done with it.

And as I know that will not be considered to be a very popular or elegant plan, I'll see what the consensus is here first.

It may be to leave things alone, and require that uses install the system-specific SDK in the SDKs folder to be supported by MacPorts, which is a fair requirement.

[jmroot (Joshua Root)]

So once again I have to ask why we're going through these contortions instead of patching the build system to use the SDK we tell it to use. I'm not eager to incur the build time that testing this would require, but this seems like a good start for the qt5 Portfile.

[jmroot (Joshua Root)]

mkspecs/features/mac/sdk.mk and/or mkspecs/features/mac/sdk.prf may also need to be patched; I don't know when and how they're used.

[kencu (Ken)]

Yeah, I got embroiled here out of kindness to try to fix this when the previous 27 attempts to get this fixed were all broken. It was briefly fixed, but that didn't make the linux system happy, and the refix to fix the linux system broke it again.

Of course the only time this fails now is when people don't actually have the system-matching SDK in place in the SDKs folder... which is actually, as I understand it, not really a supported MacPorts configuration anyway...

I'm afraid I'm not personally up for taking on such a project across all the system and qt versions required to test, so users should copy in a system-matching SDK into the two SDKs folders to avoid trouble until someone feels motivated to do that.

[jmroot (Joshua Root)]

Replying to kencu:

It was briefly fixed, but that didn't make the linux system happy, and the refix to fix the linux system broke it again.

If you mean the change to prevent the port from failing to parse on non-Darwin platforms, I doubt that caused this problem.

Of course the only time this fails now is when people don't actually have the system-matching SDK in place in the SDKs folder... which is actually, as I understand it, not really a supported MacPorts configuration anyway...

That is the normal, supported configuration on 10.13 and older.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

As far as I can tell, the main problem is QMAKE_MAC_SDK.
Qt uses to find tools (​https://code.qt.io/cgit/qt/qtbase.git/tree/mkspecs/features/mac/sdk.prf?h=5.15).
The best we may be able to do is set QMAKE_MAC_SDK to macosx${configure.sdk_version} if possible, otherwise default to macosx.

There is a ​pull request that attempts this.
I am sure the new procedure qt5pg::qmake_mac_sdk could be improved, but I just used a copied configure_get_sdkroot from base.

[kencu (Ken)]

Replying to jmroot:

If you mean the change to prevent the port from failing to parse on non-Darwin platforms, I doubt that caused this problem.

You do so much work to keep macports alive. Thank you for all that, if I have not said it often enough.

You are of course correct in pointing out that it would be best if the qt5* ports obeyed MacPorts' SDK settings instead of doing their own thing. I have tried to do this before, and it proved difficult to both implement and test. There are so many qt5 versions, all different, and so many different system configurations to test, and the qt5 configure script is all written in hand-tweaked shell, that it is overwhelming to do it robustly across all those versions and systems.

It is very easy to test if the last fix I pushed to the portfiles to test for the presence of the 10.11 SDK did or did not work. I will leave that as an exercise for the interested reader, if there is one. I am sorry I did not properly consider the non-darwin systems in so doing... my ubunutu macports build was broken just then, recently resuscitated.

Marcus has a whole new approach to testing for the SDK that avoids xcrun completely and works. Perhaps we just throw in the towel and push that now, and we can move on to more important things?

I have since fixed dozens and dozens of ports on M1 macs since I got my new MBPro screamer, and that is where my focus is going to be going forward, not back on this old headache.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

In 9a125f3f940fad9cbf6d729e98a852d90e7fb5df/macports-ports (master):

qt5 PG: be better at setting QMAKE_MAC_SDK

Ideally, QMAKE_MAC_SDK should be macosx${configure.sdk_version}.
However, this SDK is not always installed, so search for best fit.

Fixes #66549