gcc: cannot find systems headers with Xcode 11

[abyrvalk]

It seems to be a typical bug (internet search tells me), but I cannot deal with it. After the recent upgrade to Xcode version 11.0 (11A420a), the gcc installed from macports fails to compile:

A simple code (test.cc):

#include <iostream>

int main(void)
{
  std::cout << "Hello world\n";

  return 0;
}

$ g++-mp-8 test.cc
In file included from /opt/local/include/gcc8/c++/bits/postypes.h:40,
                 from /opt/local/include/gcc8/c++/iosfwd:40,
                 from /opt/local/include/gcc8/c++/ios:38,
                 from /opt/local/include/gcc8/c++/ostream:38,
                 from /opt/local/include/gcc8/c++/iostream:39,
                 from test.cc:1:
/opt/local/include/gcc8/c++/cwchar:44:10: fatal error: wchar.h: No such file or directory
 #include <wchar.h>
          ^~~~~~~~~
compilation terminated.

I have all the header files in /usr/include, together with commandline tools from Xcode. If I explicitly add include path the error changes but I have no success:

$ g++-mp-8 -I/usr/include test.cc
In file included from /opt/local/include/gcc8/c++/ext/string_conversions.h:41,
                 from /opt/local/include/gcc8/c++/bits/basic_string.h:6400,
                 from /opt/local/include/gcc8/c++/string:52,
                 from /opt/local/include/gcc8/c++/bits/locale_classes.h:40,
                 from /opt/local/include/gcc8/c++/bits/ios_base.h:41,
                 from /opt/local/include/gcc8/c++/ios:42,
                 from /opt/local/include/gcc8/c++/ostream:38,
                 from /opt/local/include/gcc8/c++/iostream:39,
                 from test.cc:1:
/opt/local/include/gcc8/c++/cstdlib:75:15: fatal error: stdlib.h: No such file or directory
 #include_next <stdlib.h>
               ^~~~~~~~~~
compilation terminated.

wchar.h and stdlib.h are both in /usr/include. The Apple Clang compiles the code successfully. Is it a problem of my environment variables or my system configuration by port?

iOS 10.14.6, Xcode 11.0

$ port installed | grep gcc
  gcc8 @8.3.0_4 (active)
  gcc_select @0.1_8 (active)
  libgcc @2.0_2 (active)
  libgcc8 @8.3.0_5 (active)
  libgcc9 @9.2.0_0 (active)

[kencu (Ken)]

you'll find lots of other open tickets about this right now. Our gcc versions were built against a versioned macOS SDK that no longer exists:

$ /opt/local/bin/gcc-mp-9 -v
Using built-in specs.
COLLECT_GCC=/opt/local/bin/gcc-mp-9
COLLECT_LTO_WRAPPER=/opt/local/libexec/gcc/x86_64-apple-darwin18/9.2.0/lto-wrapper
Target: x86_64-apple-darwin18
Configured with: /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_gcc9/gcc9/work/gcc-9.2.0/configure --prefix=/opt/local --build=x86_64-apple-darwin18 --enable-languages=c,c++,objc,obj-c++,lto,fortran --libdir=/opt/local/lib/gcc9 --includedir=/opt/local/include/gcc9 --infodir=/opt/local/share/info --mandir=/opt/local/share/man --datarootdir=/opt/local/share/gcc-9 --with-local-prefix=/opt/local --with-system-zlib --disable-nls --program-suffix=-mp-9 --with-gxx-include-dir=/opt/local/include/gcc9/c++/ --with-gmp=/opt/local --with-mpfr=/opt/local --with-mpc=/opt/local --with-isl=/opt/local --enable-stage1-checking --disable-multilib --enable-lto --enable-libstdcxx-time --with-build-config=bootstrap-debug --with-as=/opt/local/bin/as --with-ld=/opt/local/bin/ld --with-ar=/opt/local/bin/ar --with-bugurl=https://trac.macports.org/newticket --disable-tls --with-pkgversion='MacPorts gcc9 9.2.0_0' --with-sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
Thread model: posix
gcc version 9.2.0 (MacPorts gcc9 9.2.0_0) 

i.e.

--with-sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk

and -just for fun - you can't build gcc against the 10.15 Catalina headers right now.

SO -- my best bet would be to find the 10.14 SDK and put it where it is looking.

Others have downgraded their Xcode back down again (easy enough).

I rarely upgrade Xcode when it first comes out, BTW. Fine for apple stuff, but all the MacPorty stuff is going to be a trick.

[jmroot (Joshua Root)]

Restoring the 10.14 SDK (whether by downgrading Xcode or otherwise), or adding -isysroot`xcrun --show-sdk-path` to your compiler flags seem to be the currently known workarounds.

[mouse07410 (Mouse)]

Our gcc versions were built against a versioned macOS SDK that no longer exists...

I hope there's work in progress to change this to build against /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/? Which, coincidentally, is exactly what xcrun --show-sdk-path returns?

[jmroot (Joshua Root)]

That is now the 10.15 SDK, which as Ken mentioned, gcc doesn't currently build with.

[abyrvalk]

Thanks a lot:

$ g++-mp-8 -isysroot`xcrun --show-sdk-path` test.cc

cures the issue, but I'll try to downgrade Xcode

[kencu (Ken)]

see <​https://gcc.gnu.org/bugzilla//show_bug.cgi?id=79885> for some further context.

and <​https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87243>

[mouse07410 (Mouse)]

-isysroot `xcrun --show-sdk-path` workaround works, and is simple. The problem with it is that it breaks platform independence - now the source code must figure whether it's on Mac or not, and if on Mac - add this workaround.

@Joshua, what do you mean by "doesn't ... build"? gcc itself cannot be compiled from its source under Xcode-11 (10.15 SDK)?

[kencu (Ken)]

That is exactly what I mean. You can't compile gcc against the 10.15 SDK right now. There is a patch in the gcc tree to fix this, so it will get fixed ...

[mouse07410 (Mouse)]

Ken, thanks. Understood. Thankfully, my Macports is still able to use gcc8 binaries (did not have to rebuild it locally after the Xcode upgrade). I hope the fix gets percolated downstream fast though.

[kencu (Ken)]

Mouse, it shouldn't be so hard for you to find a macOS 10.14 SDK, put it here:

/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk

and (probably also have to) symlink it to

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk

AFAICT, that should solve the issue.

I'm going to copy my current MacOSX10.14.sdk into that location before I upgrade Xcode and then do exactly that.

[kencu (Ken)]

BTW, here is where gcc devs are working on this issue with the 10.15 headers <​https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90835>

[kencu (Ken)]

How to use Xcode 11 with MacPorts for now:

1. Find an 10.14 SDK (like in your Xcode 10, for example).
2. Put it somewhere, like /Library/Developer/SDKs/:
3. Upgrade to Xcode 11, and symlink the MacOSX10.14.sdk back:

   sudo ln -s /Library/Developer/SDKs/MacOSX10.14.sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
   

You're done.

[mouse07410 (Mouse)]

Mouse, it shouldn't be so hard for you to find a macOS 10.14 SDK

It is possible to find a 10.14 SDK, but not desirable for several reasons (besides, the update to Xcode-11 was pushed by the IT).

[czender (Charlie Zender)]

Thank you all for hashing through this. Good to know people are working on it and there is a workaround. In the meantime, I do not like custom sudos so I have switched to clang until gcc9 works again with the latest macports.

[ogourgue (Olivier Gourgue)]

Is there something specific to do when downgrading from Xcode 11 to 10.3? Or is it simply:

1) deleting Xcode.app 2) downloading and installing Xcode 10.3.xip and Command_Line_Tools_macOS_10.14_for_Xcode_10.3.dmg from Apple Developper website?

In particular, is there something to do about the ports successfully upgraded with Xcode 11 before one port failed to upgrade (openblas in my case)?

Thanks for your help!

[jeremyhu (Jeremy Huddleston Sequoia)]

1) You don't need to delete Xcode.app. You can just install both Xcode 11 and Xcode 10 side by side

2) You don't need to install Command_Line_Tools_macOS_10.14_for_Xcode_10.3.dmg. I suggest installing the latest CLTools (from Xcode 11) as they contain the 10.14 SDK as well.

--

Regarding GCC not locating the SDK automatically, you can pass -isysroot xcrun --show-sdk-path on the command line.

[mouse07410 (Mouse)]

I suggest installing the latest CLTools (from Xcode 11) as they contain the 10.14 SDK as well.

I confirm that it does. But there seem to be some problems. After installing CLTools for Xcode-11, I get

$ xcrun --show-sdk-path
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk
$ xcrun --show-sdk-platform-path
xcodebuild: error: SDK "/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk" cannot be located.
xcrun: error: unable to lookup item 'PlatformPath' in SDK '/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk'
$ clang -v
Apple clang version 11.0.0 (clang-1100.0.33.8)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$ xcodebuild -showsdks
iOS SDKs:
	iOS 13.0                      	-sdk iphoneos13.0

iOS Simulator SDKs:
	Simulator - iOS 13.0          	-sdk iphonesimulator13.0

macOS SDKs:
	DriverKit 19.0                	-sdk driverkit.macosx19.0
	macOS 10.15                   	-sdk macosx10.15

tvOS SDKs:
	tvOS 13.0                     	-sdk appletvos13.0

tvOS Simulator SDKs:
	Simulator - tvOS 13.0         	-sdk appletvsimulator13.0

watchOS SDKs:
	watchOS 6.0                   	-sdk watchos6.0

watchOS Simulator SDKs:
	Simulator - watchOS 6.0       	-sdk watchsimulator6.0

$ xcrun --sdk macosx clang -v
Apple clang version 11.0.0 (clang-1100.0.33.8)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$ xcrun --sdk macosx10.15 clang -v
Apple clang version 11.0.0 (clang-1100.0.33.8)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$ xcrun --sdk macosx10.14 clang -v
xcodebuild: error: SDK "macosx10.14" cannot be located.
xcrun: error: sh -c '/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk macosx10.14 -find clang 2> /dev/null' failed with exit code 16384: (null) (errno=No such file or directory)
xcrun: error: unable to find utility "clang", not a developer tool or in PATH
$ 

Could you maybe comment on the above?

[mouse07410 (Mouse)]

Found one problem: in addition to installing CLTools for Xcode-11, one must manually symlink:

$ sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk

Then everything works fine.

[kencu (Ken)]

that would be in here comment:14, point #3. :>

[mouse07410 (Mouse)]

I found comment:18 (particularly item 2) much easier to follow, with one exception of the change I mentioned. My confusion was - that item was written in such a way that it made me think nothing else was needed.

AFAICT, comment:14 only makes sense if you still have Xcode-10 and are only considering upgrading to Xcode-11. In my case Xcode-11 was dropped onto my computers by IT replacing Xcode-10, so I had to figure how to retrieve the situation when Xcode-10 is gone and Xcode-11 is the only one installed.

comment:18 item 2 offers the lowest-expenditure workaround, at the cost of one extra manual operation (symlinking).

[kencu (Ken)]

Yes, jeremy sure knows his tricks -- isn't it interesting that Apple included the 10.14 SDK in the XCode 11 command line tools, but not in Xcode 11?

[mouse07410 (Mouse)]

I won't speak my mind about what Apple did here, but nothing short of expletives could adequately convey the feelings. Note, that it's just one item in the list of things that Apple screwed up in Xcode-10 and made even worse in Xcode-11.

[kencu (Ken)]

TBH, they couldn't know that MacPorts would hard code the path to "MacOSX10.14.sdk" in 10,000 places...

[mouse07410 (Mouse)]

Hardcoding the path was not the smartest decision, but removing the SDK isn't the best one either.

And I haven't started yet on the sudden inability of Xcode to login to Enterprise GitHub (and often - to GitHub.com itself). Somehow, what worked up until Xcode-10.1 (SSH and HTTPS access) doesn't work any more (broke in Xcode-10.2.1, and became hopeless in Xcode-11). Regardless of what password of access token you pass to Xcode, or what SSH key you point it at - it fails to access the repos. Thankfully, there's IntelliJ IDEA and CLion, who have absolutely no problem connecting to all those sites with exactly the credentials that Xcode now fails. Go figure.

[seanngpack (sean ng pack)]

Replying to mouse07410:

Found one problem: in addition to installing CLTools for Xcode-11, one must manually symlink:

$ sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk

Then everything works fine.

I'm having issues with sudo port install gcc9 on Catalina & Xcode 11 even after symbolic linking the 10.14 SDK. Are there any other steps after that?

It may also be because I do not see a symbolic link generated?

Seans-MacBook:MacOSX10.14.sdk seanngpack$ ls -l
total 24
-rw-r--r--  1 root  wheel   127 Feb 22  2019 Entitlements.plist
lrwxr-xr-x  1 root  wheel    39 Oct 11 20:06 MacOSX10.14.sdk -> /Library/Developer/SDKs/MacOSX10.14.sdk
-rw-r--r--  1 root  wheel  1150 Feb 22  2019 SDKSettings.json
-rw-r--r--  1 root  wheel  1326 Feb 22  2019 SDKSettings.plist
drwxr-xr-x  3 root  wheel    96 Jul 15 17:44 System
drwxr-xr-x  7 root  wheel   224 Jul 15 17:44 usr

• also tried downgrading to Xcode 10.3 with using SDK 10.14 and setting clang 8.0 (mp-clang 8.0) as active.

finally got it installed

• I had to manually set the current CLtools in Xcode -> preferences -> locations
• delete the reference of /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk pointing to MacOSX10.15.sdk

[kencu (Ken)]

I believe this has been fixed now by changes in base and the noted ports to spec an existing SDK.

please reopen with new information if you find this is still and issue.