libtool does not respect -syslibroot when linking

[kencu (Ken)]

This ticket is to show some of the behaviour I've been noticing when trying to build on Mojave +universal against the MacOSX10.13.sdk, in particular to try to understand why the linking behaviour fails by default.

I picked a clean example, building libedit.

macports has been installed in /opt/universal, and macports.conf has two entries added:

macosx_deployment_target     10.13
macosx_sdk_version           10.13

variants.conf has one entry:

+universal

A copy of the MacOSX10.13.sdk is installed in the proper place in Xcode:

$ ls -la /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/
total 0
drwxr-xr-x  5 root  wheel  160  8 Nov 21:38 .
drwxr-xr-x  5 root  wheel  160 19 Oct 18:18 ..
drwxr-xr-x  7 root  wheel  224 30 Oct 19:25 MacOSX.sdk
drwxr-xr-x  5 root  wheel  160  8 Nov 21:37 MacOSX10.13.sdk
lrwxr-xr-x  1 root  wheel   10 25 Sep 22:31 MacOSX10.14.sdk -> MacOSX.sdk

libtool compiles correctly with the proper -isysroot

libtool: compile:  /usr/bin/clang -DHAVE_CONFIG_H -I. -I.. -I/opt/universal/include -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -pipe -Os -arch i386 -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -c readline.c -o readline.o >/dev/null 2>&1

The build tries to set up the link correctly. -isysroot and syslibroot are sent to libtool, and libtool passes only syslibroot along to clang to link, which should work:

/bin/sh ../libtool  --tag=CC   --mode=link /usr/bin/clang  -pipe -Os -arch i386 -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -no-undefined -version-info 0:58:0 -L/opt/universal/lib -Wl,-headerpad_max_install_names -arch i386 -Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -o libedit.la -rpath /opt/universal/lib chared.lo common.lo el.lo eln.lo emacs.lo hist.lo keymacro.lo map.lo chartype.lo parse.lo prompt.lo read.lo refresh.lo search.lo sig.lo terminal.lo tty.lo vi.lo wcsdup.lo tokenizer.lo tokenizern.lo history.lo historyn.lo filecomplete.lo readline.lo literal.lo        -lncurses 
libtool: link: /usr/bin/clang -dynamiclib  -o .libs/libedit.0.dylib  .libs/chared.o .libs/common.o .libs/el.o .libs/eln.o .libs/emacs.o .libs/hist.o .libs/keymacro.o .libs/map.o .libs/chartype.o .libs/parse.o .libs/prompt.o .libs/read.o .libs/refresh.o .libs/search.o .libs/sig.o .libs/terminal.o .libs/tty.o .libs/vi.o .libs/wcsdup.o .libs/tokenizer.o .libs/tokenizern.o .libs/history.o .libs/historyn.o .libs/filecomplete.o .libs/readline.o .libs/literal.o   -L/opt/universal/lib -lncurses  -Os -arch i386 -Wl,-headerpad_max_install_names -arch i386 -Wl,-syslibroot -Wl,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk   -install_name  /opt/universal/lib/libedit.0.dylib -compatibility_version 1 -current_version 1.58 -Wl,-single_module

But does not, in the end, work. Instead, the MacOSX10.14.sdk is somehow called instead as the syslibroot to do the link, and this is the unexpected behaviour, which fails:

ld: warning: The i386 architecture is deprecated for macOS (remove from the Xcode build setting: ARCHS)
ld: warning: ignoring file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/lib/libSystem.tbd, missing required architecture i386 in file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/lib/libSystem.tbd
Undefined symbols for architecture i386:
Undefined symbols for architecture i386:
  "__DefaultRuneLocale", referenced from:
      _ce__isword in chared.o
      _cv__isword in chared.o
      _cv__isWord in chared.o
      _cv_next_word in chared.o
      _cv_prev_word in chared.o
      _cv__endword in chared.o
      _ed_move_to_beg in common.o
      ...

[kencu (Ken)]

If, however, we add the -isysroot into the link mix (which we know should not be needed), by adding this to the Portfile:

# accept an sdkroot if one is set
if {${configure.sdkroot} ne ""} {
    configure.cxx        "${configure.cxx} -isysroot${configure.sdkroot}"
    configure.cc         "${configure.cc}  -isysroot${configure.sdkroot}"
}

presto -- we have success. The only difference I can see here is that now -isysroot had been added to the link command, and then it works:

/bin/sh ../libtool  --tag=CC   --mode=link /usr/bin/clang  -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk  -pipe -Os -arch i386 -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -no-undefined -version-info 0:58:0 -L/opt/universal/lib -Wl,-headerpad_max_install_names -arch i386 -Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -o libedit.la -rpath /opt/universal/lib chared.lo common.lo el.lo eln.lo emacs.lo hist.lo keymacro.lo map.lo chartype.lo parse.lo prompt.lo read.lo refresh.lo search.lo sig.lo terminal.lo tty.lo vi.lo wcsdup.lo tokenizer.lo tokenizern.lo history.lo historyn.lo filecomplete.lo readline.lo literal.lo        -lncurses 
libtool: link: /usr/bin/clang  -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -dynamiclib  -o .libs/libedit.0.dylib  .libs/chared.o .libs/common.o .libs/el.o .libs/eln.o .libs/emacs.o .libs/hist.o .libs/keymacro.o .libs/map.o .libs/chartype.o .libs/parse.o .libs/prompt.o .libs/read.o .libs/refresh.o .libs/search.o .libs/sig.o .libs/terminal.o .libs/tty.o .libs/vi.o .libs/wcsdup.o .libs/tokenizer.o .libs/tokenizern.o .libs/history.o .libs/historyn.o .libs/filecomplete.o .libs/readline.o .libs/literal.o   -L/opt/universal/lib -lncurses  -Os -arch i386 -Wl,-headerpad_max_install_names -arch i386 -Wl,-syslibroot -Wl,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk   -install_name  /opt/universal/lib/libedit.0.dylib -compatibility_version 1 -current_version 1.58 -Wl,-single_module
ld: warning: The i386 architecture is deprecated for macOS (remove from the Xcode build setting: ARCHS)
libtool: link: (cd ".libs" && rm -f "libedit.dylib" && ln -s "libedit.0.dylib" "libedit.dylib")
libtool: link: ar cru .libs/libedit.a  chared.o common.o el.o eln.o emacs.o hist.o keymacro.o map.o chartype.o parse.o prompt.o read.o refresh.o search.o sig.o terminal.o tty.o vi.o wcsdup.o tokenizer.o tokenizern.o history.o historyn.o filecomplete.o readline.o literal.o
libtool: link: ranlib .libs/libedit.a

and

$ port -v installed libedit
The following ports are currently installed:
  libedit @20170329-3.1_2+universal platform='darwin 18' archs='i386 x86_64' date='2018-11-05T20:57:12-0800'
  libedit @20180525-3.1_0+universal (active) platform='darwin 18' archs='i386 x86_64' date='2018-11-15T15:46:50-0800'

[kencu (Ken)]

same build, with -isysroot added on to the compiler spec

[kencu (Ken)]

You might think this should work as well:

if {${configure.sdkroot} ne ""} {
     configure.ldflags-append -isysroot${configure.sdkroot}
}

But it doesn't. Although that is passed to libtool, libtool does not pass it on to the link line, and the link fails:

libtool: compile:  /usr/bin/clang -DHAVE_CONFIG_H -I. -I.. -I/opt/universal/include -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -pipe -Os -arch i386 -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -c readline.c -o readline.o >/dev/null 2>&1
/bin/sh ../libtool  --tag=CC   --mode=link /usr/bin/clang  -pipe -Os -arch i386 -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -no-undefined -version-info 0:58:0 -L/opt/universal/lib -Wl,-headerpad_max_install_names -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -arch i386 -Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -o libedit.la -rpath /opt/universal/lib chared.lo common.lo el.lo eln.lo emacs.lo hist.lo keymacro.lo map.lo chartype.lo parse.lo prompt.lo read.lo refresh.lo search.lo sig.lo terminal.lo tty.lo vi.lo wcsdup.lo tokenizer.lo tokenizern.lo history.lo historyn.lo filecomplete.lo readline.lo literal.lo        -lncurses 
libtool: link: /usr/bin/clang -dynamiclib  -o .libs/libedit.0.dylib  .libs/chared.o .libs/common.o .libs/el.o .libs/eln.o .libs/emacs.o .libs/hist.o .libs/keymacro.o .libs/map.o .libs/chartype.o .libs/parse.o .libs/prompt.o .libs/read.o .libs/refresh.o .libs/search.o .libs/sig.o .libs/terminal.o .libs/tty.o .libs/vi.o .libs/wcsdup.o .libs/tokenizer.o .libs/tokenizern.o .libs/history.o .libs/historyn.o .libs/filecomplete.o .libs/readline.o .libs/literal.o   -L/opt/universal/lib -lncurses  -Os -arch i386 -Wl,-headerpad_max_install_names -arch i386 -Wl,-syslibroot -Wl,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk   -install_name  /opt/universal/lib/libedit.0.dylib -compatibility_version 1 -current_version 1.58 -Wl,-single_module
ld: warning: The i386 architecture is deprecated for macOS (remove from the Xcode build setting: ARCHS)
ld: warning: ignoring file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/lib/libSystem.tbd, missing required architecture i386 in file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/lib/libSystem.tbd
Undefined symbols for architecture i386:
  "__DefaultRuneLocale", referenced from:
      _ce__isword in chared.o
      _cv__isword in chared.o
      _cv__isWord in chared.o
      _cv_next_word in chared.o
      _cv_prev_word in chared.o
      _cv__endword in chared.o
      _ed_move_to_beg in common.o
      ...

Likewise, libtool strips it out if the -isysroot is added to the cxxflags, or to the cflags. All those fail, as we saw previously in the original build failure.