cctools @927.0.2: not fully compatible with llvm11 Unknown command line argument '-macho'.

[blair (Blair Zajac)]

When cctools +llvm11 is installed than gobject-introspection fails to build:

CPPFLAGS="-I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk" CFLAGS="-pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -Wno-duplicated-branches -Wno-switch-enum" LDFLAGS="-L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64" CC="/usr/bin/clang -arch x86_64" PKG_CONFIG="/opt/local/bin/pkg-config" GI_HOST_OS="" DLLTOOL="false"  env PATH=".libs:/opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin" LPATH=.libs CC="/usr/bin/clang -arch x86_64" PYTHONPATH=".:.:" UNINSTALLED_INTROSPECTION_SRCDIR=. UNINSTALLED_INTROSPECTION_BUILDDIR=. ./g-ir-scanner  --verbose -I. --add-include-path=. --add-include-path=./gir --add-include-path=. --add-include-path=. --add-include-path=./gir --namespace=GLib --nsversion=2.0 --libtool="/bin/sh ./libtool" --pkg=glib-2.0     --library=glib-2.0 --library=gobject-2.0 --external-library --reparse-validate --identifier-prefix=G --symbol-prefix=g --symbol-prefix=glib --c-include="glib.h"  --cflags-begin -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -DGETTEXT_PACKAGE=Dummy -DGLIB_COMPILATION -D__G_I18N_LIB_H__ --cflags-end  /opt/local/lib/glib-2.0/include/glibconfig.h /opt/local/include/glib-2.0/glib/galloca.h /opt/local/include/glib-2.0/glib/garray.h /opt/local/include/glib-2.0/glib/gasyncqueue.h /opt/local/include/glib-2.0/glib/gatomic.h /opt/local/include/glib-2.0/glib/gbacktrace.h /opt/local/include/glib-2.0/glib/gbase64.h /opt/local/include/glib-2.0/glib/gbitlock.h /opt/local/include/glib-2.0/glib/gbookmarkfile.h /opt/local/include/glib-2.0/glib/gbytes.h /opt/local/include/glib-2.0/glib/gcharset.h /opt/local/include/glib-2.0/glib/gchecksum.h /opt/local/include/glib-2.0/glib/gconvert.h /opt/local/include/glib-2.0/glib/gdataset.h /opt/local/include/glib-2.0/glib/gdate.h /opt/local/include/glib-2.0/glib/gdatetime.h /opt/local/include/glib-2.0/glib/gdir.h /opt/local/include/glib-2.0/glib/genviron.h /opt/local/include/glib-2.0/glib/gerror.h /opt/local/include/glib-2.0/glib/gfileutils.h /opt/local/include/glib-2.0/glib/ggettext.h /opt/local/include/glib-2.0/glib/ghash.h /opt/local/include/glib-2.0/glib/ghmac.h /opt/local/include/glib-2.0/glib/ghook.h /opt/local/include/glib-2.0/glib/ghostutils.h /opt/local/include/glib-2.0/glib/gi18n-lib.h /opt/local/include/glib-2.0/glib/gi18n.h /opt/local/include/glib-2.0/glib/giochannel.h /opt/local/include/glib-2.0/glib/gkeyfile.h /opt/local/include/glib-2.0/glib/glist.h /opt/local/include/glib-2.0/glib/gmacros.h /opt/local/include/glib-2.0/glib/gmain.h /opt/local/include/glib-2.0/glib/gmappedfile.h /opt/local/include/glib-2.0/glib/gmarkup.h /opt/local/include/glib-2.0/glib/gmem.h /opt/local/include/glib-2.0/glib/gmessages.h /opt/local/include/glib-2.0/glib/gnode.h /opt/local/include/glib-2.0/glib/goption.h /opt/local/include/glib-2.0/glib/gpattern.h /opt/local/include/glib-2.0/glib/gpoll.h /opt/local/include/glib-2.0/glib/gprimes.h /opt/local/include/glib-2.0/glib/gprintf.h /opt/local/include/glib-2.0/glib/gqsort.h /opt/local/include/glib-2.0/glib/gquark.h /opt/local/include/glib-2.0/glib/gqueue.h /opt/local/include/glib-2.0/glib/grand.h /opt/local/include/glib-2.0/glib/grcbox.h /opt/local/include/glib-2.0/glib/grefcount.h /opt/local/include/glib-2.0/glib/grefstring.h /opt/local/include/glib-2.0/glib/gregex.h /opt/local/include/glib-2.0/glib/gscanner.h /opt/local/include/glib-2.0/glib/gsequence.h /opt/local/include/glib-2.0/glib/gshell.h /opt/local/include/glib-2.0/glib/gslice.h /opt/local/include/glib-2.0/glib/gslist.h /opt/local/include/glib-2.0/glib/gspawn.h /opt/local/include/glib-2.0/glib/gstdio.h /opt/local/include/glib-2.0/glib/gstrfuncs.h /opt/local/include/glib-2.0/glib/gstring.h /opt/local/include/glib-2.0/glib/gstringchunk.h /opt/local/include/glib-2.0/glib/gtestutils.h /opt/local/include/glib-2.0/glib/gthread.h /opt/local/include/glib-2.0/glib/gthreadpool.h /opt/local/include/glib-2.0/glib/gtimer.h /opt/local/include/glib-2.0/glib/gtimezone.h /opt/local/include/glib-2.0/glib/gtrashstack.h /opt/local/include/glib-2.0/glib/gtree.h /opt/local/include/glib-2.0/glib/gtypes.h /opt/local/include/glib-2.0/glib/gunicode.h /opt/local/include/glib-2.0/glib/gurifuncs.h /opt/local/include/glib-2.0/glib/gutils.h /opt/local/include/glib-2.0/glib/guuid.h /opt/local/include/glib-2.0/glib/gvariant.h /opt/local/include/glib-2.0/glib/gvarianttype.h /opt/local/include/glib-2.0/glib/gversion.h /opt/local/include/glib-2.0/glib/gversionmacros.h /opt/local/include/glib-2.0/glib/gwin32.h /opt/local/include/glib-2.0/gobject/glib-types.h gir/glib-2.0.c /opt/local/include/glib-2.0/glib-unix.h g-ir-scanner g-ir-compiler --output GLib-2.0.gir
/usr/bin/clang -arch x86_64 -E -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DGETTEXT_PACKAGE=Dummy -DGLIB_COMPILATION -D__G_I18N_LIB_H__ -U__BLOCKS__ -I. -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2 -o g-ir-cpp-ygnxvj6t.i -C /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/g-ir-cpp-ygnxvj6t.c
/usr/bin/clang -arch x86_64 -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -Wno-duplicated-branches -Wno-switch-enum -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -D_REENTRANT -DGETTEXT_PACKAGE=Dummy -DGLIB_COMPILATION -D__G_I18N_LIB_H__ -I/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2 -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -c /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0.c -o /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0.o -Wno-deprecated-declarations
warning: unknown warning option '-Wno-duplicated-branches' [-Wunknown-warning-option]
1 warning generated.
g-ir-scanner: link: /bin/sh ./libtool --mode=link --tag=CC /usr/bin/clang -arch x86_64 -o /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0 -export-dynamic -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -Wno-duplicated-branches -Wno-switch-enum /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0.o -lgio-2.0 -lgobject-2.0 -lgmodule-2.0 -lglib-2.0 -lintl -Wl,-framework -Wl,CoreFoundation -lglib-2.0 -lgobject-2.0 -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64
libtool: link: /usr/bin/clang -arch x86_64 -o /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0 -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -Wno-duplicated-branches -Wno-switch-enum /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0.o -Wl,-framework -Wl,CoreFoundation -Wl,-headerpad_max_install_names -Wl,-syslibroot -Wl,/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64  -lgio-2.0 -lgmodule-2.0 -lintl -lglib-2.0 -lgobject-2.0 -L/opt/local/lib
llvm-objdump: Unknown command line argument '-macho'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '-h'?
llvm-objdump: Unknown command line argument '-dylibs-used'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '--dylibs-used'?
llvm-objdump: Unknown command line argument '-non-verbose'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '--non-verbose'?
Traceback (most recent call last):
  File "./g-ir-scanner", line 99, in <module>
    sys.exit(scanner_main(sys.argv))
  File "./giscanner/scannermain.py", line 579, in scanner_main
    shlibs = create_binary(transformer, options, args)
  File "./giscanner/scannermain.py", line 419, in create_binary
    shlibs = resolve_shlibs(options, binary, options.libraries)
  File "./giscanner/shlibs.py", line 183, in resolve_shlibs
    _resolve_non_libtool(options, binary, non_libtool))
  File "./giscanner/shlibs.py", line 103, in _resolve_non_libtool
    output = subprocess.check_output(args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/bin/sh', './libtool', '--mode=execute', 'otool', '-L', '/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0']' returned non-zero exit status 1.
make[2]: *** [GLib-2.0.gir] Error 1

When cctools +llvm90 is installed than gobject-introspection builds fine.

Do all the calls to add_execute_list() in otool/main.c need to be updated to add another - to the argument?

[kencu (Ken)]

Hi Blair, this is a known issue, and is not a bug -- they did this on purpose. There are a number of other tickets against various software builds that ran into the same thing.

Newer versions of llvm come with a new version of llvm-objdump, and they decided upstream to change the arguments that llvm-objdump accepts. llvm-objdump will no longer accept -macho. You can use -m and you can use --macho now. All the old scripts that use -macho are broken, and will need to be gradually updated to no longer do that.

If you want to get on with your work, please do this:

sudo port -f uninstall cctools
sudo port -v install cctools

and then you should get a proper working version of cctools for your system, which is cctools +xcode.

If you want to help fix all the scripts that are broken, you can stay on cctools +llvm11 and start submitting PRs (upstream, preferably) to get them all updated.

[kencu (Ken)]

have to do a bit of sleuthing to sort out if it's gobject-introspection or python that needs to be updated here to the new call.

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #61901.

[michaelld (Michael Dickens)]

has duplicate #62095

[michaelld (Michael Dickens)]

Copied from #62095 ... where we're telling poppler to save-temps and thus we can test this issue outside the MacPorts' environment.

Looking at the cctools code in cctools-927.0.2/otool/main.c, line 80 reads:

enum bool show_objdump_command = FALSE; /* print the objdump command */

change this to TRUE; build and install. then run the otool command again:

$ otool -L /opt/local/var/macports/build/_opt_sources_MacPorts_ports_github_macports_graphics_poppler/poppler/work/build/glib/tmp-introspectdnhi97mq/Poppler-0.18
+ /opt/local/bin/llvm-objdump-mp-11 -macho -dylibs-used -non-verbose /opt/local/var/macports/build/_opt_sources_MacPorts_ports_github_macports_graphics_poppler/poppler/work/build/glib/tmp-introspectdnhi97mq/Poppler-0.18
llvm-objdump: Unknown command line argument '-macho'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '-h'?
llvm-objdump: Unknown command line argument '-dylibs-used'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '--dylibs-used'?
llvm-objdump: Unknown command line argument '-non-verbose'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '--non-verbose'?

... so ... this is a cctools issue, not GO-I or poppler.

[kencu (Ken)]

Uhm -- this is expected behaviour from cctools/llvm-objdump (this command comes from the llvm tree), it does not accept "-macho" as an argument any more, it is not a bug, they did this on purpose, they won't be fixing it or reverting it.

The error is this:

/opt/local/bin/llvm-objdump-mp-11 -macho

as -macho is not an accepted flag any more.

Am I backwards on this somehow? I'm multitasking here...

[kencu (Ken)]

Oh -- do you mean that somehow the cctools otools driver is calling llvm-objdump and that is what is adding the -macho flag?

If that is what is happening, then yes, 100%, we should patch that out of cctools (or update cctools). Sorry if I missed this first time round.

[michaelld (Michael Dickens)]

Yes it's this latter: cctools otools driver is calling llvm-objdump and that is what is adding the -macho flag! It's easy to fix in the cctools code luckily!

[michaelld (Michael Dickens)]

Given that the current cctools variants are just "llvm10, llvm11, llvmdev, [+]xcode", we can patch the cctools otool code to use -- instead of - for those arguments. I think that's the only change required.

[kencu (Ken)]

those are the only llvm variants on 10.19+, anyway

on 10.4 and up, I need to see if the old llvms accept "--macho"; otherwise we can just use "-m" which is accepted still, for no good reason :>

[kencu (Ken)]

looks like llvm-objdump-3.4 accepts "--macho" as well as "-m" and "-macho".

So I can patch it to "--macho".

[michaelld (Michael Dickens)]

makes sense! make it so!

[kencu (Ken)]

oh oh. Looks like there is a whole mess of single-hyphen arguments that are no longer accepted:

% otool -l /opt/local/bin/mtoc
llvm-objdump: Unknown command line argument '-private-headers'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '--private-headers'?
llvm-objdump: Unknown command line argument '-non-verbose'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '--non-verbose'?

[kencu (Ken)]

I smell my cctools update coming ...