gtk3 @3.24.23_0: Can't update because of broken llvm

[rubendibattista (Ruben Di Battista)]

It has been a bit since I did a selfupdate. But now I cannot build gtk3 because of broken llvm (that is not listed as dependency in the Portfile).

[rubendibattista (Ruben Di Battista)]

build log

[kencu (Ken)]

I saw something like this too, when I was updating one of my systems ... there is something to be tweaked in the way MacPorts' calculates dependencies, you're right. I found it easy enough to fix -- just update llvm-8.0 (in your case) first, and then you should be good to go again.

:info:build libtool: link: /usr/bin/clang -arch x86_64 -o /opt/local/var/macports/build/_Users_rubendibattista_git_macports-ports_gnome_gtk3/gtk3/work/gtk+-3.24.23/gdk/tmp-introspectr_sd7yds/.libs/Gdk-3.0 -I/opt/local/include -DX_LOCALE -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -pipe -Os -fstrict-aliasing -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -Wall /opt/local/var/macports/build/_Users_rubendibattista_git_macports-ports_gnome_gtk3/gtk3/work/gtk+-3.24.23/gdk/tmp-introspectr_sd7yds/Gdk-3.0.o -Wl,-framework -Wl,CoreFoundation -Wl,-headerpad_max_install_names -Wl,-syslibroot -Wl,/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64  -L. ./.libs/libgdk-3.dylib -L/opt/local/lib -lpangocairo-1.0 -lpango-1.0 -lgdk_pixbuf-2.0 -lcairo-gobject -lcairo -lepoxy -lfribidi -lm -lgio-2.0 -lgobject-2.0 -lgmodule-2.0 -lglib-2.0 -lintl
:info:build dyld: Library not loaded: /opt/local/lib/libffi.6.dylib
:info:build   Referenced from: /opt/local/libexec/llvm-8.0/lib/libLLVM.dylib
:info:build   Reason: image not found
:info:build fatal error: otool: fatal error in /opt/local/bin/llvm-objdump-mp-8.0

so :

sudo port -v upgrade llvm-8.0

and when that is done, DON'T allow rev-upgrade to rebuild anything broken.

Then go ahead with

sudo port -v upgrade outdated

[rubendibattista (Ruben Di Battista)]

Thanks @kencu. That's exactly what I did, but I think this is a bug nonetheless, Macports should build llvm before gtk3, shouldn't it?

[kencu (Ken)]

Something needs to be tweaked, yes. You're on Catalina -- why is llvm-objdump-mp-8.0 being called, exactly, instead of your xcode one?

Is that coming from ld64? On your system, it should be installed with the +xcode variant...so not sure why it would be called.

[rubendibattista (Ruben Di Battista)]

I have no idea. Dunno if related, but every now and then I get the warning about the Xcode versions not recognized by Macports and I need to reinstall the Command Line Tools. Could it be related?

ld64 @3_3+ld64_xcode (active)

is indeed installed with the ld64_xcode variant.

[kencu (Ken)]

by any chance, could you have done this at some point: sudo port select llvm mp-llvm-8.0?

[rubendibattista (Ruben Di Battista)]

$> sudo port select --list llvm
Available versions for llvm:
	mp-llvm-8.0
	mp-llvm-9.0
	none (active)

Not that I remember...

[kencu (Ken)]

OK. I see this at one point in your log:

:info:configure checking for objdump... objdump

and then later, when libtool goes to use it, it is this:

:info:build fatal error: otool: fatal error in /opt/local/bin/llvm-objdump-mp-8.0

which it probably shouldn't be, but that is what brings in the error somehow...

gtk3 should not be listing a dep on llvm-8.0, that much is for sure... but what should be listing a dep for what is less clear to me...

[kencu (Ken)]

I bet it is gir-scanner -- it does lots of funky business:

:info:build g-ir-scanner: link: /bin/sh ../libtool --mode=link --tag=CC /usr/bin/clang -arch x86_64 -o /opt/local/var/macports/build/_Users_rubendibattista_git_macports-ports_gnome_gtk3/gtk3/work/gtk+-3.24.23/gdk/tmp-introspectbhhu8fe9/Gdk-3.0 -export-dynamic -I/opt/local/include -DX_LOCALE -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -pipe -Os -fstrict-aliasing -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -Wall /opt/local/var/macports/build/_Users_rubendibattista_git_macports-ports_gnome_gtk3/gtk3/work/gtk+-3.24.23/gdk/tmp-introspectbhhu8fe9/Gdk-3.0.o -L. libgdk-3.la -L/opt/local/lib -lgio-2.0 -lgobject-2.0 -lgmodule-2.0 -lglib-2.0 -lintl -Wl,-framework -Wl,CoreFoundation -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64

[kencu (Ken)]

it is gir-scanner, as the tail of your log shows.

Do you have your cctools installed with +xcode variant as well?

If your cctools is installed with llvm80 variant, at least we have a clue there.

[kencu (Ken)]

I think it must be cctools. We install it with an llvmXY variant, and it gets used (opportunistically I think) by gir-scanner, but it is broken in those rare situations where we do something like upgrading libffi, which breaks all the llvms.

nothing in gtk3 declares a dep on cctools, so then nothing declares a dep on llvmXY, so therefore the build breaks.

[ryandesign (Ryan Carsten Schmidt)]

Such problems exist from time to time, but I can't suggest a good way that we could prevent it.

We offer newer versions of Apple's toolchain in the cctools port. If you have it installed, many ports will opportunistically use the programs it provides. If they're broken in some way, for example because one of their dependencies like libffi was updated and you haven't updated cctools yet, then that will cause those many other ports to fail to build. We do not want to "fix" it by adding a cctools dependency to those many ports, because that would normally be unnecessary; those ports would build fine with Xcode's copy of those tools.

So when you encounter these kinds of problems you have to do a little investigation and fix it yourself. For example, when you see:

dyld: Library not loaded: /opt/local/lib/libffi.6.dylib
  Referenced from: /opt/local/libexec/llvm-8.0/lib/libLLVM.dylib
  Reason: image not found

use port provides /opt/local/libexec/llvm-8.0/lib/libLLVM.dylib to figure out what port provides it, check if it's outdated, and if so, upgrade it. Then try again.

[rubendibattista (Ruben Di Battista)]

Can't the dependency be conditionally imposed on those ports being already installed?

[kencu (Ken)]

So to do that, any port that uses gobject-introspection would need to see if cctools is installed, and if it is installed, add a dependency on it. That will not be a very popular thing to do, as it makes the dependency chain random. Technically, though, that's trivially easy to do.

But if that was accepted, if I understand how that works, that would force cctools (and it's dependency on some LLVM version) to update itself before the port using gir-scanner.

I think pretty much all these ports are using the gobject-introspection portgroup, so the place to put the code is already there.

[rubendibattista (Ruben Di Battista)]

Why do you say "random dependency chain"? Wouldn't it be double branched, but still deterministic?

[kencu (Ken)]

That's the argument that will come up.

On any given system, there will, or won't, be a dep added on cctools that is non-deterministic from the point of view of the Portfile.

[kencu (Ken)]

that would force cctools (and it's dependency on some LLVM version) to update itself

well -- actually -- would it?

Even if you added a dep on cctools to the port using gir-scanner, the test would show that cctools is installed and up-to-date.

Would that somehow trigger to the port dep mechanism that the llvm cctools is using needs to have it's upgrade done first? Maybe it would. I'm not 100% sure.

[ryandesign (Ryan Carsten Schmidt)]

Replying to kencu:

I think pretty much all these ports are using the gobject-introspection portgroup, so the place to put the code is already there.

Dave has been undoing this lately because #60644 has not been implemented, so it will be some work to re-do this to all the ports where it has been undone.

[ryandesign (Ryan Carsten Schmidt)]

The challenge of ports opportunistically using components from cctools is not limited to those that use gobject-introspection, and for that matter also includes ports that opportunistically use gawk, gsed, grep, and gmake.

[rubendibattista (Ruben Di Battista)]

Ryan, what should be done to account for this in the most complete way then? At least with your vision? Such that I could try to do it...