gmp @6.0.0_0: Autotools incorrectly selects "-undefined suppress" symbol lookup on Yosemite

[wichert@…]

The libgcc build fails when trying to compile unwind-dw2-fde-darwin.c with an internal compile error. main.log is attached.

This happens on a current yosemite / xcode 6b4 box.

[borodiychuk (Andriy Borodiychuk)]

Cc Me!

[skymoo (Adam Mercer)]

Cc Me!

[davidchambers (David Chambers)]

Cc Me!

[mtwest729@…]

Cc Me!

[mtwest729@…]

log from libgcc install attempt on machine running OSX 10.5.8

[mtwest729@…]

Replying to wichert@…:

From what I am able to tell from the log file, I am getting the same build error as wichert, but on a significantly older system. The main.log from sudo port -d install libgcc is attached and labeled accordingly.

• OSX 10.5.8 on 2007 Intel Mac
• Xcode 3.1.4

[ryandesign (Ryan Carsten Schmidt)]

The log from Yosemite ends after 33000 lines with "internal compiler error: Segmentation fault: 11"; the log from Leopard ends after just 9000 lines with "collect2: error: ld returned 1 exit status". So they are different problems. You should probably file a new ticket for the problem you're seeing on Leopard since they'll probably have different solutions. In either case, they're issues the developers of gcc will probably have to solve; we just package and deliver the software they developer.

[AP1010]

Cc Me!

[eirnym (Eir Nym)]

Cc Me!

[corsaroquad (Giovanni Grieco)]

Cc Me!

[larryv (Lawrence Velázquez)]

Cc Me!

[larryv (Lawrence Velázquez)]

Still seeing this with beta 6 (6A280e).

[mf2k (Frank Schima)]

And I still see this with Xcode beta 7 (6A280n).

[platipodium (Carsten Lemmen)]

Cc Me!

[sultanqasim@…]

This issue breaks almost every port on Yosemite because libgcc is so critical.

[sultanqasim@…]

Cc Me!

[garethsimons@…]

Cc Me!

[garethsimons@…]

Cc Me!

[larryv (Lawrence Velázquez)]

Replying to sultanqasim@…:

This issue breaks almost every port on Yosemite because libgcc is so critical.

What? This is demonstrably hyperbolic. The vast majority of ports don’t use libgcc in any way. This only affects the gcc* ports and those that depend on them; there are certainly many, but nowhere near “almost every” one.

[sultanqasim@…]

Replying to larryv@…:

Replying to sultanqasim@…:

This issue breaks almost every port on Yosemite because libgcc is so critical.

What? This is demonstrably hyperbolic. The vast majority of ports don’t use libgcc in any way. This only affects the gcc* ports and those that depend on them; there are certainly many, but nowhere near “almost every” one.

While I admit a bit of an exaggeration, a huge number of ports have dependency chains that lead to libgcc. This has broken all kinds of ports for me ranging from py##-lxml to texlive-latex-recommended.

EDIT: I'm not sure why so many ports are trying to build libgcc. I traced the dependencies of the the ports I gave as examples and couldn't find why they wanted libgcc. For years I've noticed gcc-related dependencies on ports that seem unrelated to gcc. Sorry about derailing this discussion.

[ryandesign (Ryan Carsten Schmidt)]

Replying to sultanqasim@…:

While I admit a bit of an exaggeration, a huge number of ports have dependency chains that lead to libgcc. This has broken all kinds of ports for me ranging from py##-lxml to texlive-latex-recommended.

The dependency chains of py*-lxml and texlive-latex-recommended do not seem to include libgcc, from what I can tell.

[sultanqasim@…]

Sorry for this distraction again, but as an update, removing the gcc ports seemed to solve my issue of libgcc breaking many other ports. I found that py*-scipy was the port that led me to install gcc in the first place. It appears that macports was unhappy about broken files from when it tried to upgrade libgcc, and it often tried to rebuild libgcc when installing other ports.

[larryv (Lawrence Velázquez)]

Also applies to 4.10-20140810.

[larryv (Lawrence Velázquez)]

Anyone seeing this with Yosemite + Xcode 6 on actual hardware (i.e., not in a VM)?

[sultanqasim@…]

Replying to larryv@…:

Anyone seeing this with Yosemite + Xcode 6 on actual hardware (i.e., not in a VM)?

I am seeing this on an actual mid-2012 MacBook Pro running Yosemite natively.

[mf2k (Frank Schima)]

Me too. Although on a 2013 MacBook Pro.

[larryv (Lawrence Velázquez)]

Okay, thanks. I wanted to rule out Fusion shenanigans.

[platipodium (Carsten Lemmen)]

Replying to larryv@…:

Anyone seeing this with Yosemite + Xcode 6 on actual hardware (i.e., not in a VM)?

Yep, a 2014 MacBook Pro

[larryv (Lawrence Velázquez)]

Still a problem on 10.10 14A343f with Xcode 6.1 6A1027 and clang-600.0.53 but not on 10.9.5 13F31 with the same dev tools.

[larryv (Lawrence Velázquez)]

Builds fine with Homebrew. Might be a problem with our packaging or with a dependency.

[hiro.masa.yoshi.moto@…]

I found a workaround to avoid this internal compile error. The steps are as follows;

• step-1: downgrade gmp to 4.3.2
• step-2: downgrade libmpc to 0.8.1
• step-3: port install libgcc

For step 1 and 2, I used "port edit" to modify the version numbers and checksums. For step 2, I also wrote small patch;

--- src/acos.c.old~     2014-09-13 01:26:33.000000000 +0900
+++ src/acos.c  2014-09-13 01:26:53.000000000 +0900
@@ -189,7 +189,7 @@ mpc_acos (mpc_ptr rop, mpc_srcptr op, mp
     rnd_im = rnd_im == GMP_RNDU ? GMP_RNDD
       : rnd_im == GMP_RNDD ? GMP_RNDU
 #if MPFR_VERSION_MAJOR >= 3
-      : rnd_im == GMP_RNDA ? GMP_RNDZ
+      : rnd_im == MPFR_RNDA ? GMP_RNDZ
 #endif
       : rnd_im;
   rnd1 = RNDC(GMP_RNDN, rnd_im);

See ​http://lists.gforge.inria.fr/pipermail/mpc-discuss/2010-April/000679.html for details.

I hope this will help you :-)

[larryv (Lawrence Velázquez)]

Thank you, but I’ve discovered the problem.

[Schamschula (Marius Schamschula)]

Cc Me!

[larryv (Lawrence Velázquez)]

The GCC ICE was a red herring. You’ll be unsurprised to hear that this is Autotools’ fault.

1. We set the MACOSX_DEPLOYMENT_TARGET environment variable to “10.10”. (Homebrew doesn’t set it at all and thus doesn’t start down this road to perdition.)
2. Libtool (and, by extension, configure scripts built with it) interprets “10.10” as “10.1*”.
3. OS X did not support dynamic lookup of undefined symbols until 10.3. Since Libtool thinks 10.10 Yosemite is 10.1 Puma, it tells the linker to ignore undefined symbols.

   case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in
       10.0,*86*-darwin8*|10.0,*-darwin[[91]]*)
           _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
       10.[[012]]*)
           _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
       10.*)
           _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
   esac
   

4. GMP binaries contain several unresolved symbols that they no doubt expect dyld(1) to look up later.
5. Thanks to Libtool, dyld(1) has no such plans. It makes this very clear during GMP’s test suite.

   Crashed Thread:        0  Dispatch queue: com.apple.main-thread
   
   Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
   Exception Codes:       EXC_I386_GPFLT
   
   Error Formulating Crash Report:
   Failed to read dyld_all_image_infos: Coudln't get the data
   
   Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
   0   libdyld.dylib                 	0x7fff88c64432 stack_not_16_byte_aligned_error + 0
   

6. This filters up to anything trying to use GMP. Like, say, GCC trying to optimize some code.

Marcus, I’m attaching my fix; may I commit it? (I bumped the revision because this problem only manifests at runtime.)

[larryv (Lawrence Velázquez)]

Fix for GMP build on Yosemite

[wichert@…]

That's some amazing detective work from larryv!

[cjones051073 (Chris Jones)]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Replying to larryv@…:

1. We set the MACOSX_DEPLOYMENT_TARGET environment variable to “10.10”. (Homebrew doesn’t set it at all and thus doesn’t start down this road to perdition.)

Ever since Mac OS X 10.5, the default value for MACOSX_DEPLOYMENT_TARGET is the currently running OS X version, so I would expect everyone else to be affected too.

[larryv (Lawrence Velázquez)]

Perhaps Xcode or Clang/GCC have that default behavior, but there’s no magic default value for the environment variable. Unless it’s present in the environment, a script that tries to substitute $MACOSX_DEPLOYMENT_TARGET will get a null value.

So Libtool does the right thing if MACOSX_DEPLOYMENT_TARGET is absent from its environment (the first case alternative in the code snippet above). It only chokes when that variable is explicitly set to “10.10” (or “10.11” or “10.12” or “10.27”).

The variable is not present in Homebrew’s build environment, so their build works fine. (I personally tested both GCC and GMP.)

[ryandesign (Ryan Carsten Schmidt)]

Oh, you're right. I was thinking of the default value when the environment variable is not specified.

In any case, thanks for the patch! After rebuilding gmp with it I was able to build libgcc on Yosemite beta 2. (gcc itself is still building.)

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Should be fixed in r125350.
Thanks for the patch.

I submitted an upstream bug report at:
​https://gmplib.org/list-archives/gmp-bugs/2014-September/thread.html

[larryv (Lawrence Velázquez)]

Great, thanks!

(As for aclocal.m4, I just fixed that for completeness. Upstream should ultimately patch their Autotools, or this will crop up again.)

[cooljeanius (Eric Gallager)]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

As the gmp developers pointed out, this is a libtool problem. Larry fixed the libtool port in r125325. Did the problem also get reported to the libtool developers?

[larryv (Lawrence Velázquez)]

I posted to libtool-patches ​in September, but the latest commit to their repository ​is from May.

[larryv (Lawrence Velázquez)]

Providentially, Libtool upstream ​will be cutting a release this weekend to take care of this problem.

[jeremyhu (Jeremy Huddleston Sequoia)]

Cc Me!

[jeremyhu (Jeremy Huddleston Sequoia)]

-undefined dynamic_lookup ? Yuck. That usually indicates a bad design decision if it is used in shipping code. Why do we need to lookup undefined symbols at runtime? Why are they not available at build time?

[mojca (Mojca Miklavec)]

I see Ryan committing changes to various ports referencing this ticket. What's the proper solution? Ask upstream to regenerate the configure files with a newer version of libtool? (Judging from the logs I assume that at least 2.4.3 or 2.4.4 is needed.)

[jeremyhu (Jeremy Huddleston Sequoia)]

The ultimate proper solution is to make sure that binaries link correctly. Allow the linker to emit errors when it detects them and fix those errors instead of suppressing those errors and letting the user hit them at runtime.

Politely asking upstream to pickup the latest glibtool, automake, and autoconf is generally a good idea. There are many other bugs in older versions that bite us consistently (eg: older glibtool do not pass -stdlib=... during the link phase, which breaks LibcxxOnOlderSystems, and older glibtool also does not pass -fsanitize= which breaks ASan builds, etc).

[RJVB (René Bertin)]

Can't concerned ports simply regenerate the configure script using autoreconf?

FWIW, if that step becomes usual it might be useful to record it in the statefile instead of doing it systematically each time one repeats the configure step.

[jeremyhu (Jeremy Huddleston Sequoia)]

It's very annoying to go in and add 'use_autoreconf ...' to Portfiles after they've been updated to a release where the release manager used an older version of autoconf. Additionally, those options usually persist long after they're needed (once upstream starts using newer tools), and even if they get removed, we might need to re-add it if someone later does a release with older tools...

The best option is to get upstream to use newer autotools.

[mojca (Mojca Miklavec)]

I wonder if MacPorts should be detecting problems like too old tools used to generate the configure script and warn about it.