openssl: Undefined symbols for architecture x86_64: _ERR_load_OBJ_strings in parallel builds

[smastracci@…]

Hi Team,

Today running port upgrade outdated for my system, and openssl display error. Attach log debug.

Error Display from command line :

bash-3.2# port upgrade outdated
--->  Computing dependencies for openssl
--->  Building openssl
Error: org.macports.build for port openssl returned: command execution failed
Please see the log file for port openssl for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_openssl/openssl/main.log
Error: Unable to upgrade port: 1

Thank.

[neverpanic (Clemens Lang)]

Please use WikiFormatting and preview your post when using Trac. When reporting bugs against ports, please Cc the maintainer, if any.

[neverpanic (Clemens Lang)]

The log is not from a clean attempt. Please run sudo port clean openssl and try again. If that fails again, please attach the main.log again.

You can also try sudo port clean openssl; sudo port -t upgrade openssl.

[smastracci@…]

Hi,

I'm try sudo port clean openssl and sudo port -t upgrade openssl, the same error. Attach new main.log.

Thank.

[neverpanic (Clemens Lang)]

I tried to reproduce your problem but couldn't. The only difference between your log and mine is that I have -arch x86_64 in OBJCXXFLAGS. I believe this is because I use a development version of MacPorts and unrelated to the problem at hand.

Can you attach the output of nm $(port work openssl)/openssl-1.0.1k-x86_64/libcrypto.a? What does port file openssl print on your system? Do you have anything in /usr/local?

[neverpanic (Clemens Lang)]

Has duplicate #46782, which is not +universal, so I'm assuming that doesn't matter in this case.

[turkowski]

i am bitten by this bug too. I still have some old x86 stuff, so I compile with +universal.

[dwhs@…]

After trouble upgrading openssl, I ended up here. I had +universal because I installed wine last in a recent macports reinstall, and therefore ended up with lots of packages in both +universal and not, and hadn’t gotten around to cleaning that up yet. When nothing I saw here helped the +universal version, I decided to see if I could upgrade the non-universal version.

The following managed to work for me:

1. sudo port clean openssl
2. sudo port activate openssl @1.0.1k (i.e., the non-universal variant)
3. sudo port upgrade openssl

It pulled openssl as a binary, then when checking for broken ports, found 2 (python27 @2.7.9 +universal, gnome-vfs @2.24.4 +universal). It rebuilt those (including bringing in the outdated ports that were requirements, and, in the process, openssl @1.0.2 +universal). Completed without error. I finished my sudo port upgrade outdated with no problems.

[neverpanic (Clemens Lang)]

#46782 suggests this can be fixed by opening and closing Xcode, did you all try that? If that works, it sounds like this is a misconfiguration and not a problem with the Portfile.

[michaelthomassullivan]

I encountered a somewhat similar failure while trying to upgrade to openssl-1.0.2, although the specific undefined symbols were different, starting with _CRYPTO_THREADID_cmp, although I don't think the precise list is pertinent (I can provide the full log if anyone cares).

This was actually on the second build attempt; the first time failed with:

:info:build ar: creating archive ../libcrypto.a
:info:build /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../libcrypto.a(ebcdic.o) has no symbols
:info:build /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../libcrypto.a(fips_ers.o) has no symbols
:info:build error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../../libcrypto.a is not an archive
:info:build /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar: internal ranlib command failed

Since after the first failure I tried sudo port clean openssl; sudo port upgrade openssl and got quite different errors, I suspected a race condition, so I appended use_parallel_build no to the Portfile, repeated the same commands, and this time it built and installed cleanly.

I didn't try to fully analyze the root cause of the failure with a parallel make, but it seems to be due to simultaneously running the ar command on the same libcrypto.a from both the Makefile in the crypto directory and the Makefile of one of its many subdirectories, which is likely to lead to non-deterministic behavior.

[neverpanic (Clemens Lang)]

That sounds reasonable. Can everybody who experiences this problem please try again using sudo port clean openssl; sudo port upgrade openssl -- build.jobs=1 and report back if that worked?

[neverpanic (Clemens Lang)]

Cc'ing jeremyhu, who re-enabled parallel building in r130893. Jeremy, does this sound plausible?

[jeremyhu (Jeremy Huddleston Sequoia)]

Given the info in comment 12, it certainly sounds plausible that there are more parallel races than the ones that I uncovered in r130893. I built parallel successfully on multiple machines, but that certainly can't be a thorough confirmation that all the dependencies were added to eliminate the race. I suggest we re-add the 'use_parallel_build no' and keep this ticket open to figure out what's remaining (or upstream it if you prefer).

[michaelthomassullivan]

Cc Me!

[bpanulla (Brian Panulla)]

Cc Me!

[neverpanic (Clemens Lang)]

Disabled parallel builds in r133158. Please report back whether this fixes the problem for you.

[bpanulla (Brian Panulla)]

Fixed for me.... thanks! (10.8.5 with Xcode 5.1.1)

[jeremyhu (Jeremy Huddleston Sequoia)]

Let's leave this open (feel free to assign to me). I want to track down what was really causing this. I loathe slow -j1 builds ;)

[neverpanic (Clemens Lang)]

Reassigning, thanks for offering to work on this, because I wouldn't have the time to do it. Larry, if you want the ticket back, feel free to reclaim it :)

[jeremyhu (Jeremy Huddleston Sequoia)]

I've moved over to using libressl, so I'm not planning on addressing this. Larry, do you want to keep it or call this wontfix?

[neverpanic (Clemens Lang)]

I'm calling this wontfix until upstream openssl gets their act together for proper parallel builds.

[mascguy (Christopher Nielsen)]

It looks like parallel builds are still disabled, and that makes a noticeable impact for folks running on older hardware.

Does anyone know if this is still an issue?

[kencu (Ken)]

With the openssl 3.0.0 update in the PR, I have been building it over and over on many systems with parallel building enabled locally, and have never seen an issue.

It was on my list to re-enable parallel builds once 3.0.0 was pushed.

[Christopher Nielsen <mascguy@…>]

In 02db4e7a363141c67f384f1f29db50fb2f70e53e/macports-ports (master):

openssl3: blacklist Xcode clang < 1001; enable parallel builds

• Fix build failures related to AVX-512 instruction set:

Fixes: #63622

• Parallel builds discussed via the following ticket:

See: #46719

[mascguy (Christopher Nielsen)]

FYI, a formal fix for OpenSSL parallel builds was finally committed on 2018-11-09, for versions 1.1 and later. That's per the following PR:

​https://github.com/openssl/openssl/pull/7583#issuecomment-437254341

So I'll re-enable parallel builds for openssl11 as well.

[Christopher Nielsen <mascguy@…>]

In 3cffa9f1c370507a126f791b590b088cf17db2f5/macports-ports (master):

openssl11: Enable parallel builds

• Parallel builds discussed via the following ticket:

See: #46719