libgcc @2.0_1: should not be noarch for dependents architecture compatibility checking

[Ionic (Mihai Moldovan)]

OpenBLAS @0.6.3_1 +universal fails to build due to this:

[...]
/opt/local/bin/gfortran-mp-8 -O3 -m128bit-long-double -Wall -frecursive -m32   -o sblat1 sblat1.o ../libopenblas-r1.a -lpthread -lgfortran -lpthread -lgfortran
/opt/local/bin/gfortran-mp-8 -O3 -m128bit-long-double -Wall -frecursive -m32   -o dblat1 dblat1.o ../libopenblas-r1.a -lpthread -lgfortran -lpthread -lgfortran
ld: warning: ignoring fileld : warning: ignoring file /opt/local/lib/gcc8//opt/local/lib/gcc8/i386i386/libgfortran.dylib, file was /builtlibgfortran.dylib,  filefor x86_64 which is  notwas  the built for x86_64 which architecture being is not linked (i386): /opt/local/lib/gcc8/i386/libgfortran.dylib
ld: warning: ignoring file /opt/local/lib/gcc8/libgcc_ext.10.5.dylib, missing required architecture i386 in file /opt/local/lib/gcc8/libgcc_ext.10.5.dylib (1 slices)
ld: warning: ignoring file /opt/local/lib/gcc8/i386/libquadmath.dylib, file was built for x86_64 which is not thethe architecture being linked (i386): /opt/local/lib/gcc8/i386/libgfortran.dylib
ld: warning: ignoring file /opt/local/lib/gcc8/libgcc_ext.10.5.dylib, missing required architecture i386 in file /opt/local/lib/gcc8/libgcc_ext.10.5.dylib (1 slices)
ld: warning: ignoring file /opt/local/lib/gcc8/i386/libquadmath.dylib, file was built for x86_64 which is not the architecture being linked (i386): /opt/local/lib/gcc8/i386 /libquadmath.dylib
Undefined symbols for architecture i386:
  "__gfortran_internal_pack", referenced from:
      architecture being linked (i386): /opt/local/lib/gcc8/i386/libquadmath.dylib
Undefined symbols for architecture i386:
  "__gfortran_set_args", referenced from:
[...]

The output is mangled which is probably a side-effect of -j2. That's not something that should concern us a lot.

What does concern me is that linking against libgfortran.dylib, libgcc_ext.dylib et al fails in i386 mode, because the libraries do not contain the needed architecture.

$ lipo -info /opt/local/lib/gcc8/i386/libgfortran.dylib /opt/local/lib/gcc8/libgcc_ext.10.5.dylib /opt/local/lib/gcc8/i386/libquadmath.dylib
Architectures in the fat file: /opt/local/lib/gcc8/libgcc_ext.10.5.dylib are: x86_64 
Non-fat file: /opt/local/lib/gcc8/i386/libgfortran.dylib is architecture: x86_64
Non-fat file: /opt/local/lib/gcc8/i386/libquadmath.dylib is architecture: x86_64

I'm CCing the last few people to update the port because it's nomaintainer. Not creating a ticket against OpenBLAS, because OpenBLAS doesn't seem to at fault here.

As pointed out in the comments, these files are just symlinks to libraries installed by the libgccX ports. Since the libgcc port is noarch, architecture checking is disabled for it and its dependencies, which means that the required libgccX port will not be rebuilt for other arches, even when OpenBLAS would require that.

[Ionic (Mihai Moldovan)]

OpenBLAS.main.log

[kencu (Ken)]

hmmm....on 10.6.8:

$ port -v installed libgcc8
The following ports are currently installed:
  libgcc8 @8.3.0_5+universal (active) platform='darwin 10' archs='i386 x86_64' date='2019-05-06T23:35:19-0700'

$ port provides /opt/local/lib/libgcc/libquadmath.0.dylib
/opt/local/lib/libgcc/libquadmath.0.dylib is provided by: libgcc8

$ lipo -info /opt/local/lib/libgcc/libquadmath.0.dylib
Architectures in the fat file: /opt/local/lib/libgcc/libquadmath.0.dylib are: x86_64 i386 

$ port provides /opt/local/lib/libgcc/libgfortran.5.dylib
/opt/local/lib/libgcc/libgfortran.5.dylib is provided by: libgcc8

$ lipo -info /opt/local/lib/libgcc/libgfortran.5.dylib
Architectures in the fat file: /opt/local/lib/libgcc/libgfortran.5.dylib are: x86_64 i386 

[Ionic (Mihai Moldovan)]

Yeees, but OpenBLAS tries to link files from the gccX port, not libgcc. Not sure if that is a bug with OpenBLAS or intended that way.

On 10.9, libgcc is being provided by libgcc9 and sadly I don't have that installed +universal. I do have libgcc8 installed +universal, but that isn't helping, since libgcc8 is a stub port that only installs a README file.

[kencu (Ken)]

but all the dylibs in gcc8 are just symlinks to the active libgccN port, right?

$ ls -la /opt/local/lib/gcc8/libgfortran.5.dylib
lrwxr-xr-x  1 root  admin  41  6 May 06:44 /opt/local/lib/gcc8/libgfortran.5.dylib -> /opt/local/lib/libgcc/libgfortran.5.dylib

[Ionic (Mihai Moldovan)]

... yes, right. I didn't check that.

So I guess the gcc ports aren't really broken, but with all that stuff, how do we make sure that a compatible libgcc version is installed?

Look at OpenBLAS: I previously installed it with the (then default) +gcc8 variant and it did work back then, because libgcc8 was a "full" port and installed with +universal due to some dependency. That worked fine. Recently, libgcc8 was replaced with a stub port and libgcc9 is now providing the libraries needed. That's fine, because GCC and its libraries are generally forward compatible (and if they aren't, the soname changes). However, nothing pulled in the +universal variant when libgcc9 was installed. Boom, no more +universal libgfortran.5.dylib, OpenBLAS +universal fails.

I *can* fix that situation manually by rebuilding libgcc9 +universal, but that isn't exactly a great solution for other users. Fixing the port is difficult because the actual libgcc port that provides a specific library is not easy to determine. For instance, just because libgcc8 is a stub port on my system that has been "replaced" by the contents of libgcc9, this needs not be true for other systems - i.e., old systems that can't use libgcc9 retain libgcc8 as a "full" port with the original contents.

[kencu (Ken)]

You're meant to require the libgcc port, and that port then figures out the various libgcc6, 7, 8, 9 parts as needed depending on your system.

Not sure why OpenBLAS is specifically forcing a gcc variant. Perhaps that is not needed.

[cjones051073 (Chris Jones)]

If my memory serves me correctly, OpenBLAS itself does not specify anything w.r.t. gcc. It uses the compilers portgroup which does this for it.

I guess what happened is when gcc9 was added and made the default gcc version, gcc9 became the default variant for ports like this, but due to the way macports works anyone who previously had it installed with gcc8 as the vairant, would not be automatically switched up. Macports does not really have a neat way to handle this.

Its not clear to me why OpenBLAS really needs to expose the gcc version used as a variant ? If it is just to get a compiler version, I think it would be better served just making the decision on which to use internally, and to not expose that decision to the user via a variant. I am not sufe I see the advantage in do so, and it has disadvantages like those here.

[Ionic (Mihai Moldovan)]

OpenBLAS uses Fortran and... I guess that for some reason people want to select what compiler to use. For me it's just another dependency, but we also have similar setups for, e.g., the root ports. Not exposing the selection to the user would not help in this case, though, since even when only used internally it wouldn't depend on any libgcc version directly or indirectly.

Yes, the variant was not automatically switched. But additionally, libgcc9 replaced content installed by libgcc8, but nothing pulled in the +universal variant.

Installing OpenBLAS +universal+gcc9 fails in exactly the same way. Even though the compilers PG is used, no dependency upon the correct libgcc version is added, so no architecture check takes place which would rebuild libgcc stuff with +universal.

[kencu (Ken)]

I don't know exactly how the +universal variant propagates through dependencies, but I would have thought that it would have worked like this:

openblas +universal --> gcc9 +universal --> libgcc +universal --> libccN +universal

Is there a breakdown somewhere in this chain of dependencies?

[cjones051073 (Chris Jones)]

Replying to kencu:

I don't know exactly how the +universal variant propagates through dependencies, but I would have thought that it would have worked like this:

openblas +universal --> gcc9 +universal --> libgcc +universal --> libccN +universal

Is there a breakdown somewhere in this chain of dependencies?

Personally, i have no idea. I have never had any need for the universal variants for any ports, so have no experience with how it works.

[cjones051073 (Chris Jones)]

Replying to Ionic:

OpenBLAS uses Fortran and... I guess that for some reason people want to select what compiler to use. For me it's just another dependency, but we also have similar setups for, e.g., the root ports. Not exposing the selection to the user would not help in this case, though, since even when only used internally it wouldn't depend on any libgcc version directly or indirectly.

Well it would, if it manually set up that dependency on libgcc itself....

If you check the latest root6 port version you will see I actually have removed the gcc variants, for very similar reasons to those here, to not expose the gcc version to the user as a variant. I think it does make sense to also do this for openblas.

Yes, the variant was not automatically switched. But additionally, libgcc9 replaced content installed by libgcc8, but nothing pulled in the +universal variant.

Installing OpenBLAS +universal+gcc9 fails in exactly the same way. Even though the compilers PG is used, no dependency upon the correct libgcc version is added, so no architecture check takes place which would rebuild libgcc stuff with +universal.

[cjones051073 (Chris Jones)]

​https://github.com/macports/macports-ports/blob/b17a2b3327d48092ae95d060af2063e1d6d5a194/science/root6/Portfile#L54

[Ionic (Mihai Moldovan)]

Debug log, trying to install OpenBLAS +universal+gcc8+lapack

[Ionic (Mihai Moldovan)]

Hmm, yeah, that should be how it works. I did take a look at the logfile again and can see gcc8, libgcc, libgcc8 and libgcc9 being added as dependencies (like you said transitively). I can't see any arch check being carried out in the log file though.

I ran port -Nsvtd install OpenBLAS +universal+gcc9+lapack to get a debug log. Base is doing... let's say... interesting things there.

For instance, this:

DEBUG: epoch: in tree: 3 installed: 3
DEBUG: libgcc 2.0_1 exists in the ports tree
DEBUG: libgcc 2.0_1  is the latest installed
DEBUG: libgcc 2.0_1  is active
DEBUG: Merging existing variants '' into variants
DEBUG: new fully merged portvariants: a52 + universal - mng + x11 + pulseaudio - faac + cups + mencoder_extras + sdl + xvid + esd + x264 + caca + fribidi + twolame + lirc + dv + aa + smb + dts + ipv6 + speex + osd +
DEBUG: Changing to port directory: /opt/local/var/macports/sources/macports.rsync.ionic.de/release/ports/lang/libgcc
DEBUG: OS darwin/13.4.0 (macOS 10.9) arch i386
DEBUG: Sourcing PortGroup select 1.0 from /opt/local/var/macports/sources/macports.rsync.ionic.de/release/ports/_resources/port1.0/group/select-1.0.tcl
DEBUG: universal_variant is false, so not adding the default universal variant
DEBUG: Requested variant +a52 is not provided by port libgcc.
DEBUG: Requested variant -universal is not provided by port libgcc.
DEBUG: Requested variant +mng is not provided by port libgcc.
DEBUG: Requested variant +x11 is not provided by port libgcc.
DEBUG: Requested variant -pulseaudio is not provided by port libgcc.
DEBUG: Requested variant +faac is not provided by port libgcc.
DEBUG: Requested variant +cups is not provided by port libgcc.
DEBUG: Requested variant +mencoder_extras is not provided by port libgcc.
DEBUG: Requested variant +sdl is not provided by port libgcc.
DEBUG: Requested variant +xvid is not provided by port libgcc.
DEBUG: Requested variant +esd is not provided by port libgcc.
DEBUG: Requested variant +x264 is not provided by port libgcc.
DEBUG: Requested variant +caca is not provided by port libgcc.
DEBUG: Requested variant +twolame is not provided by port libgcc.
DEBUG: Requested variant +fribidi is not provided by port libgcc.
DEBUG: Requested variant +smb is not provided by port libgcc.
DEBUG: Requested variant +aa is not provided by port libgcc.
DEBUG: Requested variant +dv is not provided by port libgcc.
DEBUG: Requested variant +lirc is not provided by port libgcc.
DEBUG: Requested variant +dts is not provided by port libgcc.
DEBUG: Requested variant +speex is not provided by port libgcc.
DEBUG: Requested variant +ipv6 is not provided by port libgcc.
DEBUG: Requested variant +osd is not provided by port libgcc.
DEBUG: Running callback portconfigure::add_automatic_compiler_dependencies
DEBUG: Finished running callback portconfigure::add_automatic_compiler_dependencies
DEBUG: Running callback portbuild::add_automatic_buildsystem_dependencies
DEBUG: Finished running callback portbuild::add_automatic_buildsystem_dependencies
DEBUG: Running callback portstartupitem::add_notes
DEBUG: Finished running callback portstartupitem::add_notes
DEBUG: No need to upgrade! libgcc 2.0_1 >= libgcc 2.0_1

or this:

DEBUG: epoch: in tree: 2 installed: 2
DEBUG: libgcc9 9.1.0_2 exists in the ports tree
DEBUG: libgcc9 9.1.0_2  is the latest installed
DEBUG: libgcc9 9.1.0_2  is active
DEBUG: Merging existing variants '-universal' into variants
DEBUG: new fully merged portvariants: a52 + universal - mng + x11 + pulseaudio - faac + cups + mencoder_extras + sdl + xvid + esd + x264 + caca + fribidi + twolame + lirc + dv + aa + smb + dts + ipv6 + speex + osd +

-universal seems to come directly from cctools:

DEBUG: epoch: in tree: 0 installed: 0
DEBUG: cctools 921_2 exists in the ports tree
DEBUG: cctools 921_2 +llvm37 is the latest installed
DEBUG: cctools 921_2 +llvm37 is active
DEBUG: Merging existing variants '+llvm37-universal' into variants
DEBUG: new fully merged portvariants: a52 + universal - mng + x11 + pulseaudio - faac + cups + mencoder_extras + sdl + xvid + x264 + esd + caca + fribidi + twolame + lirc + dv + aa + smb + dts + ipv6 + speex + llvm37 + osd +

But that's not how variant mangling is supposed to work... +universal should be getting passed down to all port dependencies (and ports be rebuilt). Maybe base master is broken?

[ryandesign (Ryan Carsten Schmidt)]

Replying to kencu:

I don't know exactly how the +universal variant propagates through dependencies, but I would have thought that it would have worked like this:

openblas +universal --> gcc9 +universal --> libgcc +universal --> libccN +universal

Is there a breakdown somewhere in this chain of dependencies?

It behaves the same way as any other variant. The variants specified are passed down to any dependencies that have not yet been installed. Any dependencies that have already been installed are kept at whatever variants they already had.

[Ionic (Mihai Moldovan)]

That doesn't make sense, though; especially not for the universal variant.

I definitely remember a different (more correct) base behavior in the past: some (upgraded) port (I don't remember the details) depended on a not-yet-installed port like a gcc, libgcc or clang variant. Both ports were hence installed -universal. Later on, a port using +universal was upgraded and that lead to the gcc/libgcc/clang port being rebuilt with +universal as a dependency, too.

Especially for the universal variant, base must rebuild dependencies with +universal as well, unless the dependencies don't feature a universal variant. Not rebuilding installed dependencies will lead to the mess described in this ticket.

[Ionic (Mihai Moldovan)]

I've now downgraded base to 2.5.0, 2.4.4, 2.3.5 and they all behaved the same (i.e., tried to upgrade OpenBLAS directly instead of working on any dependencies).

However, 2.2.1 behaves differently. Not exactly like I would like to have it, but still better. A dry run would have rebuilt at least cctools and llvm-3.7 due to changes in the universal variant (both ports are currently installed -universal), so that's a start. The dry run did not reveal a libgcc9 rebuild, though.

So there was definitely a behavioral change from 2.2.1 to 2.3.5 at least, but it's still not quite the one I was looking for. I'll try to go back even further (if possible).

[Ionic (Mihai Moldovan)]

port -ysvtd install OpenBLAS +gcc9+lapack+universal with base 2.2.1

[Ionic (Mihai Moldovan)]

Okay, the cctools and llvm-3.7 rebuilds were due to older versions of base not honoring installs_libs no. Shortly before the release of 2.3.0, jmr pushed a change that ignores such type of dependencies. So that's sane, because the only way cctools will be used is by spawning binaries so any executable architecture will suffice.

I was able to test down to 2.1.3, but 2.0.x is too old to reasonably test since compiler.blacklist and co. aren't supported by that version.

All of this leaves me a bit puzzled. For instance, I don't understand why older base versions did try to rebuild cctools (and it dependency clang-3.7) with +universal, but not libgcc9 for instance. I do understand that newer port versions do not try to rebuild cctools any longer, because it uses installs_libs no, but that's really just a side-effect.

I noticed that the compilers-1.0 PG adds a depends_skip_archcheck clause on the compiler ports themselves, but not on any other dependencies, like the path-based libgcc dependency. I removed that clause just to see what happens, but it changed nothing. I also tried to hard-depend upon libgcc9 without using a path-based dependency style, but this also changed nothing, even with the older base version.

I still think that this is a bug because variants should be getting synced up with dependencies.

Joshua, might I ask for your opinion on this?

[jmroot (Joshua Root)]

It's hard to tell what you're saying the actual problem is with all the back and forth in the comments. A few things that seem relevant though:

• libgcc is noarch. That's technically correct, but means no architecture requirements are enforced on its dependencies.
• You can't depend on a variant. (The famous #126)
• If a port is installed with an explicitly negated variant, that variant will stay negated across upgrades unless explicitly enabled.

[Ionic (Mihai Moldovan)]

Replying to jmroot:

It's hard to tell what you're saying the actual problem is with all the back and forth in the comments.

Okay, let me summarize what happened.

1. OpenBLAS was installed +universal (and with other variants) at some point in the past
2. libgcc9 was automatically installed as some dependency with the default variants, i.e., -universal
3. OpenBLAS was updated, uses compilers-1.0 PG which adds compiler variants and explicitly depends on the default libgcc port (which is libgcc9 currently for most OS versions)
4. upgrading OpenBLAS fails because it can't link libraries from libgcc9 for the universal architecture(s) and base/port does not try to rebuild the dependency libgcc9 with +universal

A few things that seem relevant though:

• libgcc is noarch. That's technically correct, but means no architecture requirements are enforced on its dependencies.

Could be a problem if the path-based dependency in compilers-1.0 picked the libgcc port, but that really can't happen. Either libgcc is a proper, non-noarch port or a different non-noarch port will satisfy the dependency.

• You can't depend on a variant. (The famous #126)

Not relevant in this case, I think.

• If a port is installed with an explicitly negated variant, that variant will stay negated across upgrades unless explicitly enabled.

That's probably causing the trouble described here. Isn't it better to rebuild dependencies with the same set of variants as enabled on the port-to-be-upgraded? Otherwise users will always run into this sort of problems:

1. portA installed -universal; portB installed +universal
2. portA and portB both depend (directly or indirectly) on portC
3. portA and portB are updated and now depend on portD instead of portC
4. portD is installed -universal on portA's upgrade
5. portB is upgraded later, but portD not rebuilt with +universal
6. fail, user needs to fix situation manually (reinstalling portD +universal)

[jmroot (Joshua Root)]

Replying to Ionic:

5. portB is upgraded later, but portD not rebuilt with +universal

Shouldn't happen; arch checking should apply to portB's direct dependencies.

Can you make a reduced test case with a few stub ports?

[jmroot (Joshua Root)]

Actually, I think this is the problem:

% port -v deps openblas
Full Name: OpenBLAS @0.3.6_1+gcc9+lapack
Build Dependencies:   port:cctools, port:gcc9
Library Dependencies: path:lib/libgcc/libgcc_s.1.dylib:libgcc

That can't work properly because libgcc doesn't actually provide ${prefix}/lib/libgcc/libgcc_s.1.dylib. If the named port is installed, the depspec is considered satisfied by it, not the owner of the file (if those are different, which they shouldn't be).

[Ionic (Mihai Moldovan)]

I thought I did change that to a port:libgcc9 depspec temporarily to see if the path-based depspec might cause problems, but I did so wrongly. Yes, if the path-based depspec is replaced with a port-based one, libgcc9 is being arch-checked and rebuilt correctly.

[Ionic (Mihai Moldovan)]

path-based dependencies are used all over the place to mean "be satisfied by that file (and hence the port that installed it), but install the named port if it isn't". The current behavior essentially disables arch-checking if the named port does not match the file's provider, which can be troublesome, I guess.

While that sort of dependency is typically used for -devel ports and their counterparts, libgcc is special in that the libgcc port is nowadays a stub depending on a proper libgccX port. This selection is based upon the OS version.

Some time ago, libgcc was a proper port providing a default GCC runtime with libgccX ports being older or newer alternatives installing into unique directories, so using such a path-based depspec incidentally used to work.

[jmroot (Joshua Root)]

Replying to Ionic:

path-based dependencies are used all over the place to mean "be satisfied by that file (and hence the port that installed it), but install the named port if it isn't".

The actual semantics are "here's a file that is needed, and this port can provide it".

Anyway, the easiest fix is probably just to not call libgcc noarch.

[Ionic (Mihai Moldovan)]

That doesn't seem to work. I removed supported_archs, added an empty-block universal variant to libgcc, touched the OpenBLAS portfile for good measure and regenerated the PortIndex, but base doesn't want to upgrade either libgcc or libgcc9.

[jmroot (Joshua Root)]

Did you reinstall libgcc?

[Ionic (Mihai Moldovan)]

... of course not. Yeah, after forcing an upgrade, it works like expected. Didn't think about the arch check getting information from the database and seeing "noarch"...

[Ionic (Mihai Moldovan)]

Updating original description, removing CC'd devs that haven't touched libgcc.

I guess I'll push the change to libgcc soon since it's nomaintainer... unless Chris doesn't want to, since he essentially created the port?

[Mihai Moldovan <ionic@…>]

In 20218df10e6b4be60eedafe5857285545129fd0a/macports-ports (master):

lang/libgcc: make non-noarch, add stub universal variant, revbump.

While technically noarch is correct for this port, architecture checking
will be disabled for any noarch ports and their dependencies.

Since a lot of ports depend upon libgcc (either directly or indirectly),
and those ports may be built universally and link against libraries from
libgcc (or rather the default libgccX for the system in question), we'll
have to make sure that architectures match.

Not marking the libgcc stub port as noarch is the easiest workaround.

Fixes: #58646