Opened 10 years ago

Closed 9 years ago

Last modified 3 years ago

#46719 closed defect (wontfix)

openssl: Undefined symbols for architecture x86_64: _ERR_load_OBJ_strings in parallel builds

Reported by: smastracci@… Owned by: larryv (Lawrence Velázquez)
Priority: Normal Milestone:
Component: ports Version: 2.3.3
Keywords: Cc: jeremyhu (Jeremy Huddleston Sequoia), neverpanic (Clemens Lang), benshimo@…, michaelthomassullivan, bpanulla (Brian Panulla), mascguy (Christopher Nielsen)
Port: openssl

Description (last modified by neverpanic (Clemens Lang))

Hi Team,

Today running port upgrade outdated for my system, and openssl display error. Attach log debug.

Error Display from command line :

bash-3.2# port upgrade outdated
--->  Computing dependencies for openssl
--->  Building openssl
Error: org.macports.build for port openssl returned: command execution failed
Please see the log file for port openssl for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_openssl/openssl/main.log
Error: Unable to upgrade port: 1

Thank.

Attachments (1)

main.log (481.4 KB) - added by smastracci@… 10 years ago.

Download all attachments as: .zip

Change History (30)

comment:1 Changed 10 years ago by neverpanic (Clemens Lang)

Cc: cal@… added
Description: modified (diff)
Owner: changed from macports-tickets@… to mww@…

Please use WikiFormatting and preview your post when using Trac. When reporting bugs against ports, please Cc the maintainer, if any.

comment:2 Changed 10 years ago by neverpanic (Clemens Lang)

Type: updatedefect

comment:3 Changed 10 years ago by neverpanic (Clemens Lang)

The log is not from a clean attempt. Please run sudo port clean openssl and try again. If that fails again, please attach the main.log again.

You can also try sudo port clean openssl; sudo port -t upgrade openssl.

Changed 10 years ago by smastracci@…

Attachment: main.log added

comment:4 Changed 10 years ago by smastracci@…

Hi,

I'm try sudo port clean openssl and sudo port -t upgrade openssl, the same error. Attach new main.log.

Thank.

comment:5 Changed 10 years ago by neverpanic (Clemens Lang)

Summary: Error Upgrade for port opensslError Upgrade for port openssl +universal

comment:6 Changed 10 years ago by neverpanic (Clemens Lang)

I tried to reproduce your problem but couldn't. The only difference between your log and mine is that I have -arch x86_64 in OBJCXXFLAGS. I believe this is because I use a development version of MacPorts and unrelated to the problem at hand.

Can you attach the output of nm $(port work openssl)/openssl-1.0.1k-x86_64/libcrypto.a? What does port file openssl print on your system? Do you have anything in /usr/local?

comment:7 Changed 10 years ago by neverpanic (Clemens Lang)

Owner: changed from mww@… to larryv@…
Summary: Error Upgrade for port openssl +universalopenssl: Undefined symbols for architecture x86_64: _ERR_load_OBJ_strings

Has duplicate #46782, which is not +universal, so I'm assuming that doesn't matter in this case.

comment:8 Changed 10 years ago by neverpanic (Clemens Lang)

Cc: benshimo@… added

comment:9 Changed 10 years ago by turkowski

i am bitten by this bug too. I still have some old x86 stuff, so I compile with +universal.

comment:10 Changed 10 years ago by dwhs@…

After trouble upgrading openssl, I ended up here. I had +universal because I installed wine last in a recent macports reinstall, and therefore ended up with lots of packages in both +universal and not, and hadn’t gotten around to cleaning that up yet. When nothing I saw here helped the +universal version, I decided to see if I could upgrade the non-universal version.

The following managed to work for me:

  1. sudo port clean openssl
  2. sudo port activate openssl @1.0.1k (i.e., the non-universal variant)
  3. sudo port upgrade openssl

It pulled openssl as a binary, then when checking for broken ports, found 2 (python27 @2.7.9 +universal, gnome-vfs @2.24.4 +universal). It rebuilt those (including bringing in the outdated ports that were requirements, and, in the process, openssl @1.0.2 +universal). Completed without error. I finished my sudo port upgrade outdated with no problems.

comment:11 Changed 10 years ago by neverpanic (Clemens Lang)

#46782 suggests this can be fixed by opening and closing Xcode, did you all try that? If that works, it sounds like this is a misconfiguration and not a problem with the Portfile.

comment:12 Changed 10 years ago by michaelthomassullivan

I encountered a somewhat similar failure while trying to upgrade to openssl-1.0.2, although the specific undefined symbols were different, starting with _CRYPTO_THREADID_cmp, although I don't think the precise list is pertinent (I can provide the full log if anyone cares).

This was actually on the second build attempt; the first time failed with:

:info:build ar: creating archive ../libcrypto.a
:info:build /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../libcrypto.a(ebcdic.o) has no symbols
:info:build /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../libcrypto.a(fips_ers.o) has no symbols
:info:build error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: ../../libcrypto.a is not an archive
:info:build /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar: internal ranlib command failed

Since after the first failure I tried sudo port clean openssl; sudo port upgrade openssl and got quite different errors, I suspected a race condition, so I appended use_parallel_build no to the Portfile, repeated the same commands, and this time it built and installed cleanly.

I didn't try to fully analyze the root cause of the failure with a parallel make, but it seems to be due to simultaneously running the ar command on the same libcrypto.a from both the Makefile in the crypto directory and the Makefile of one of its many subdirectories, which is likely to lead to non-deterministic behavior.

Last edited 10 years ago by michaelthomassullivan (previous) (diff)

comment:13 Changed 10 years ago by neverpanic (Clemens Lang)

That sounds reasonable. Can everybody who experiences this problem please try again using sudo port clean openssl; sudo port upgrade openssl -- build.jobs=1 and report back if that worked?

comment:14 Changed 10 years ago by neverpanic (Clemens Lang)

Cc: jeremyhu@… added

Cc'ing jeremyhu, who re-enabled parallel building in r130893. Jeremy, does this sound plausible?

comment:15 Changed 10 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Given the info in comment 12, it certainly sounds plausible that there are more parallel races than the ones that I uncovered in r130893. I built parallel successfully on multiple machines, but that certainly can't be a thorough confirmation that all the dependencies were added to eliminate the race. I suggest we re-add the 'use_parallel_build no' and keep this ticket open to figure out what's remaining (or upstream it if you prefer).

comment:16 Changed 10 years ago by michaelthomassullivan

Cc: MichaelThomasSullivan@… added

Cc Me!

comment:17 Changed 10 years ago by bpanulla (Brian Panulla)

Cc: macports@… added

Cc Me!

comment:18 Changed 10 years ago by neverpanic (Clemens Lang)

Disabled parallel builds in r133158. Please report back whether this fixes the problem for you.

comment:19 Changed 10 years ago by bpanulla (Brian Panulla)

Fixed for me.... thanks! (10.8.5 with Xcode 5.1.1)

comment:20 Changed 10 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Let's leave this open (feel free to assign to me). I want to track down what was really causing this. I loathe slow -j1 builds ;)

comment:21 Changed 10 years ago by neverpanic (Clemens Lang)

Cc: larryv@… added
Owner: changed from larryv@… to jeremyhu@…
Summary: openssl: Undefined symbols for architecture x86_64: _ERR_load_OBJ_stringsopenssl: Undefined symbols for architecture x86_64: _ERR_load_OBJ_strings in parallel builds

Reassigning, thanks for offering to work on this, because I wouldn't have the time to do it. Larry, if you want the ticket back, feel free to reclaim it :)

comment:22 Changed 9 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Cc: larryv@… removed
Owner: changed from jeremyhu@… to larryv@…

I've moved over to using libressl, so I'm not planning on addressing this. Larry, do you want to keep it or call this wontfix?

comment:23 Changed 9 years ago by neverpanic (Clemens Lang)

Resolution: wontfix
Status: newclosed

I'm calling this wontfix until upstream openssl gets their act together for proper parallel builds.

comment:24 Changed 3 years ago by mascguy (Christopher Nielsen)

It looks like parallel builds are still disabled, and that makes a noticeable impact for folks running on older hardware.

Does anyone know if this is still an issue?

comment:25 Changed 3 years ago by mascguy (Christopher Nielsen)

Cc: mascguy added

comment:26 Changed 3 years ago by kencu (Ken)

With the openssl 3.0.0 update in the PR, I have been building it over and over on many systems with parallel building enabled locally, and have never seen an issue.

It was on my list to re-enable parallel builds once 3.0.0 was pushed.

comment:27 Changed 3 years ago by Christopher Nielsen <mascguy@…>

In 02db4e7a363141c67f384f1f29db50fb2f70e53e/macports-ports (master):

openssl3: blacklist Xcode clang < 1001; enable parallel builds

  • Fix build failures related to AVX-512 instruction set:

Fixes: #63622

  • Parallel builds discussed via the following ticket:

See: #46719

comment:28 Changed 3 years ago by mascguy (Christopher Nielsen)

FYI, upstream fixed OpenSSL parallel builds on 2018-11-09, for versions 1.1 and later. That's per the following PR:

https://github.com/openssl/openssl/pull/7583#issuecomment-437254341

So I'll re-enable parallel builds for openssl11 as well.

Last edited 3 years ago by mascguy (Christopher Nielsen) (previous) (diff)

comment:29 Changed 3 years ago by Christopher Nielsen <mascguy@…>

In 3cffa9f1c370507a126f791b590b088cf17db2f5/macports-ports (master):

openssl11: Enable parallel builds

  • Parallel builds discussed via the following ticket:

See: #46719

Note: See TracTickets for help on using tickets.