Multiple versions of emutls in the same process (via different libgcc_s.dylib or libstdc++.dylib) is problematic

[howarth@…]

The current libstdcxx subport links in libgcc_eh.a. This causes the FSF gcc unwinder to be used within the libstdc++ dynamic lib code but the system unwinder in other code generated by the g++-mp-4.x compilers. This is forbidden behavior as only a single unwinder should be in use and this should always be the system unwinder. In a perfect world, the static libgcc_eh would be rewritten to use the same code used for the compatibility unwinder in libSystem or the compiler-rt compact unwinder. Given the current situation, the best solution possible is to avoid the hack of linking in -static-libgcc for the libstdc++ shared library but instead add the libgcc*dylib files to the libstdcxx subport so that they are always available for the libstdc++ shared lib from FSF gcc. These libgcc_ext stubs have been carefully crafted to only provide those symbols not defined in libSystem on darwin and will insure a single unwinder (the system one) is always in use. Note that the spec generated by -static-gcc always passes libstdc++.a to the linker when the g++ compiler is used. This behavior is broken in MacPorts due to the absence of libstdc++.a in the packaged gcc4x. If you are worried about that feature, each gcc4x package should also have its own libstdc++.a retained.

[howarth@…]

testcase for gcc bugzilla exposing libstdcxx packaging flaw

[howarth@…]

The test.cc testcase from ​http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56835 shows the use of two different unwinders in the current libstdcxx packaging which links in libgcc_eh.a to libstdc++'s shared lib. This flaw can be reproduce with either gcc47 or gcc48 using 'g++-mp-4.7 -std=c++11 test.cc' or 'g++-mp-4.8 -std=c++11 test.cc' and then executing the resultiing binary which will segfault. This particular testcase won't compiler on earlier gcc46 or earlier but the underlying problem of two unwinders being in use will still exist.

[jeremyhu (Jeremy Huddleston Sequoia)]

Replying to howarth@…:

The current libstdcxx subport links in libgcc_eh.a. This causes the FSF gcc unwinder to be used within the libstdc++ dynamic lib code but the system unwinder in other code generated by the g++-mp-4.x compilers.

Within libstdc++, the host's libunwind (from the host's libSystem) is being used:

~ $ nm -m /opt/local/lib/libstdc++.6.dylib | grep _Unwind_
                 (undefined) external __Unwind_DeleteException (from libSystem)
                 (undefined) external __Unwind_GetDataRelBase (from libSystem)
                 (undefined) external __Unwind_GetIPInfo (from libSystem)
                 (undefined) external __Unwind_GetLanguageSpecificData (from libSystem)
                 (undefined) external __Unwind_GetRegionStart (from libSystem)
                 (undefined) external __Unwind_GetTextRelBase (from libSystem)
                 (undefined) external __Unwind_RaiseException (from libSystem)
                 (undefined) external __Unwind_Resume (from libSystem)
                 (undefined) external __Unwind_Resume_or_Rethrow (from libSystem)
                 (undefined) external __Unwind_SetGR (from libSystem)
                 (undefined) external __Unwind_SetIP (from libSystem)
...

I am not disputing that libstdc++.6.dylib may be using the FSF unwinder in other places, but I have yet to be proven that.

In a perfect world, the static libgcc_eh would be rewritten to use the same code used for the compatibility unwinder in libSystem ... Given the current situation, the best solution possible is to avoid the hack of linking in -static-libgcc for the libstdc++ shared library but instead add the libgcc*dylib files to the libstdcxx subport so that they are always available for the libstdc++ shared lib from FSF gcc.

If your analysis is correct, this will simply fix the issue for libstdc++, but the issue will remain for anything else that is linked using -static-libgcc. Therefore, we should fix the issue with -static-libgcc rather than doing a 1-off fix for libstdc++.

These libgcc_ext stubs have been carefully crafted to only provide those symbols not defined in libSystem on darwin and will insure a single unwinder (the system one) is always in use. Note that the spec generated by -static-gcc always passes libstdc++.a to the linker when the g++ compiler is used. This behavior is broken in MacPorts due to the absence of libstdc++.a in the packaged gcc4x. If you are worried about that feature, each gcc4x package should also have its own libstdc++.a retained.

Yes, I agree. We should probably maintain libstdc++.a for each compiler. But let's tackle that separately.

This ticket is about getting libstdc++.dylib to be built properly. I would much prefer to see a fix that involves gutting the offending code from libgcc_eh.a that should be resolved to the host, but I don't see any:

$ nm /usr/lib/system/libcompiler_rt.dylib | grep -v ' U ' | awk '{print $3}' | sort -u > /tmp/host_compiler_rt
$ nm /usr/lib/system/libunwind.dylib | grep -v ' U ' | awk '{print $3}' | sort -u > /tmp/host_unwind
$ cat /tmp/host_compiler_rt /tmp/host_unwind | sort -u > /tmp/host
$ nm /opt/local/lib/libstdc++.6.dylib | grep -v ' U ' | awk '{print $3}' | sort -u > /tmp/libstdcxx
$ comm -12 /tmp/libstdcxx /tmp/host

Since your issue is resolved by changing the linking, I'm waiting for you to tell me what is being resolved incorrectly into libgcc_eh.a which should be resolved into libSystem.

---

Furthermore, in another ticket, you said that the issue was about bootstrapping and not multiple unwinders. That seems to make more sense given the evidence here so far. Did you yet try reverting r103047 from #36116?

[howarth@…]

standalone testcase for failing libstdc++ testcase

[howarth@…]

I'll ping Nick on the uploaded test case and have him look at in with a debug unwinder. Simply looking at the symbols in nm is unlikely to be sufficient.

[howarth@…]

The actual crash seem to occur at

0x00007fff92dfc168 in pthread_getspecific ()
0x00007fff92dfc171 in pthread_getspecific ()
0x000000010009df00 in __emutls_get_address ()
0x000000010009df03 in __emutls_get_address ()
0x000000010009df06 in __emutls_get_address ()
0x000000010009df0c in __emutls_get_address ()
0x000000010009df0f in __emutls_get_address ()
0x000000010009df12 in __emutls_get_address ()
0x000000010009df18 in __emutls_get_address ()
0x000000010009df1d in __emutls_get_address ()
0x000000010009df21 in __emutls_get_address ()
0x000000010009df24 in __emutls_get_address ()
0x000000010009df26 in __emutls_get_address ()
0x000000010009df29 in __emutls_get_address ()
0x000000010009df2a in __emutls_get_address ()
0x000000010009df2b in __emutls_get_address ()
0x000000010009df2d in __emutls_get_address ()
0x000000010009df2f in __emutls_get_address ()
0x000000010009df31 in __emutls_get_address ()
0x000000010008db90 in __once_proxy ()

I'll reattempt building the previous packaging with a full bootstrap to see if that helps eliminate the problem.

[howarth@…]

I've verified that test case still crashes when libstdc++.6.dylib is built with a full bootstrap so that the issue seem to be coupled to your use of -static-libgcc when linking libstdc++.

[jeremyhu (Jeremy Huddleston Sequoia)]

Right, so the issue seems to be the emutls symbols which are being linked statically instead of from libgcc_s.dylib

$ nm  /opt/local/lib/gcc48/libgcc_ext.10.5.dylib  | grep -v ' U ' | awk '{print $3}' | sort -u > /tmp/gccext
$ comm -12 /tmp/gccext /tmp/libstdcxx 
___emutls_get_address
___emutls_register_common

which seems to have nothing to do with the host's libunwind.

[jeremyhu (Jeremy Huddleston Sequoia)]

Without being able to look at how FSF implements emutls (due to it being GPL3), I can only assume that the issue here is that one copy registers some data in its TLD and the other copy is trying to find that data in its TLD. It therefore misses, and the application crashes.

This would similarly be the case if you tried to cross between gcc46's libgcc_s.dylib and gcc47's libgcc_s.dylib's emutls support.

Thus I think the best solution is probably to have libstdc++.dylib link against a dynamic libgcc runtime instead of statically.

That being said, I don't think that libgcc should be provided by the libstdcxx port. It should be provided by a libgcc subport.

I will wait to do this work until after gcc48 is stable and gcc49 is added to MacPorts.

[howarth@…]

Placing the libgcc*dylib in a separate subport seems rather excessive considering. To properly FSF gcc, full bootstraps should be used and you are now tripling the number of bootstraps required to install gcc48.

[jeremyhu (Jeremy Huddleston Sequoia)]

Replying to howarth@…:

Placing the libgcc*dylib in a separate subport seems rather excessive considering. To properly FSF gcc, full bootstraps should be used and you are now tripling the number of bootstraps required to install gcc48.

As you've mentioned already:

• The failure to bootstrap was not the issue.
• Many users are getting these as binary packages and thus the bootstrapping issue is not such a concern for them if we decided to go that route.

If we do want to put these both into one port, I'd prefer calling the ports something like gcc-runtime and gcc-runtime-devel (and obsoleting the libstdcxx ports for them).

[cooljeanius (Eric Gallager)]

Cc Me!

[akimd (Akim Demaille)]

Cc Me!

[jeremyhu (Jeremy Huddleston Sequoia)]

libgcc.patch

[jeremyhu (Jeremy Huddleston Sequoia)]

libgcc.patch attached. Please test. I've also sent an email to macports-dev requesting comments before pushing.

[jeremyhu (Jeremy Huddleston Sequoia)]

r109291