openssl: Build failure: invalid instruction mnemonic 'adcxq'

[frozencemetery (Robbie Harwood)]

On 10.9.5 server, on a non-root installation with gcc 4.9.2 and clang 3.5.1, openssl fails to upgrade:

robbie@osxserver:~$ port upgrade openssl
--->  Computing dependencies for openssl
--->  Fetching distfiles for openssl
--->  Verifying checksums for openssl
--->  Extracting openssl
--->  Applying patches to openssl
--->  Configuring openssl
--->  Building openssl
Error: org.macports.build for port openssl returned: command execution failed
Please see the log file for port openssl for details:
    /Network/Servers/osxserver.b-aassoc.edu/Volumes/ServerDrive/NetUsers/robbie/var/macports/logs/_Volumes_ServerDrive_NetUsers_robbie_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_openssl/openssl/main.log
Error: Unable to upgrade port: 1
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets
robbie@osxserver:~$ 

I've attached main.log. Thanks!

[frozencemetery (Robbie Harwood)]

main.log - openssl upgrade failure

[frozencemetery (Robbie Harwood)]

And I messed up the formatting... we're off to a great start here. Sorry, all.

[ryandesign (Ryan Carsten Schmidt)]

What is the output of "/usr/bin/clang -v" and "xcodebuild -version"?

[neverpanic (Clemens Lang)]

This was also reported in #46874, and the problem apparently were outdated Command Line Tools. Try updating them.

[neverpanic (Clemens Lang)]

Did updating the Command Line Tools help?

[frozencemetery (Robbie Harwood)]

Hi cal, thanks for the prompt response. Unfortunately, I see no change after updating command line tools.

[lechris@…]

Replying to rharwood@…:

Hi cal, thanks for the prompt response. Unfortunately, I see no change after updating command line tools.

I stumbled into the same issue today with Xcode 5 and its command line tools. Upgrading to Xcode 6.2 (and its command line tools) solved the issue in my case.

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.9.5
BuildVersion:	13F1066

$ clang --version
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix

$ xcodebuild -version
Xcode 6.2
Build version 6C131e

$ sudo port upgrade openssl
--->  Computing dependencies for openssl
--->  Fetching distfiles for openssl
--->  Verifying checksums for openssl
--->  Extracting openssl
--->  Applying patches to openssl
--->  Configuring openssl
--->  Building openssl
--->  Staging openssl into destroot
--->  Installing openssl @1.0.2a_0
--->  Cleaning openssl
--->  Computing dependencies for openssl
--->  Deactivating openssl @1.0.1j_0
--->  Cleaning openssl
--->  Activating openssl @1.0.2a_0
--->  Cleaning openssl
--->  Updating database of binaries
--->  Scanning binaries for linking errors               
--->  No broken files found.  

[lechris@…]

Cc Me!

[GregoryEAllen (Greg Allen)]

Updating the command line tools fixed it for me.

$ sw_vers 
ProductName:	Mac OS X
ProductVersion:	10.8.5
BuildVersion:	12F2501

$ xcodebuild -version
Xcode 5.1.1
Build version 5B1008

$ clang -v
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin12.6.0
Thread model: posix

[frozencemetery (Robbie Harwood)]

Replying to lechris@…:

I stumbled into the same issue today with Xcode 5 and its command line tools. Upgrading to Xcode 6.2 (and its command line tools) solved the issue in my case.

For me as well. Thanks!

[larryv (Lawrence Velázquez)]

Well, let’s tentatively close this as “invalid” (since it appears to have been a toolchain issue) and hope no one else runs into it.

[mojca (Mojca Miklavec)]

Cc Me!

[mojca (Mojca Miklavec)]

I'm reopening the issue. I never had problems so far, I have

  openssl @1.0.2d_0 (active)

installed, but it now fails to work for me:

:info:build x86_64-mont.s:807:2: error: invalid instruction mnemonic 'adcxq'
:info:build  adcxq %rax,%r13
:info:build  ^~~~~
:info:build x86_64-mont.s:808:2: error: invalid instruction mnemonic 'adcxq'
:info:build  adcxq %rbp,%r14
:info:build  ^~~~~

I have

> sw_vers
ProductName:	Mac OS X
ProductVersion:	10.7.5
BuildVersion:	11G63b

> xcodebuild -version
Xcode 4.6.3
Build version 4H1503

> clang -v
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix

However I'm unable to update my toolchain (without changing the OS version as well, that is).

[su-v]

Build failure of openssl 1.0.2e_0 on OS X 10.7.5 confirmed (openssl 1.0.2d_0 built and installed ok).

[su-v]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Will those of you experiencing the problem with 1.0.2e please let us know what version of clang and what version of Xcode you have installed?

[su-v]

Replying to ryandesign@…:

Will those of you experiencing the problem with 1.0.2e please let us know what version of clang and what version of Xcode you have installed?

I use the same versions as mojca in comment:12 (latest Xcode available for Lion):

$ sw_vers 
ProductName:	Mac OS X
ProductVersion:	10.7.5
BuildVersion:	11G63b
$ 
$ xcodebuild -version
Xcode 4.6.3
Build version 4H1503
$ 
$ clang --version
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix
$ 

[McDutchie (Martijn Dekker)]

Replying to ryandesign@…:

Will those of you experiencing the problem with 1.0.2e please let us know what version of clang and what version of Xcode you have installed?

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.7.5
BuildVersion:	11G63
$ xcodebuild -version
Xcode 4.6.3
Build version 4H1503
$ clang --version
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix
$ 

[persramd@…]

Cc Me!

[beremour]

Problem with:

$sw_vers
ProductName:    Mac OS X
ProductVersion: 10.7.5
BuildVersion:   11G63

$xcodebuild -version
Xcode 4.6.3
Build version 4H1503

$ clang --version
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix

[andrew.long@…]

Here's a 'me too!'

ProductName:	Mac OS X
ProductVersion:	10.7.5
BuildVersion:	11G63

Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix

Xcode 4.6.3
Build version 4H1503

[someuser12]

Cc Me!

[neverpanic (Clemens Lang)]

I think the problem here is Apple's particularly stupid clang --version output. The relevant code is in crypto/bn/asm/x86_64-mont.pl, line 71:

`$ENV{CC} -v 2>&1` =~ /((?:^clang|LLVM) version|based on LLVM) ([3-9])\.([0-9]+)/)

Now, which version number do you think the regex will match… "Apple LLVM version 4.2" or "based on LLVM 3.2svn"?

Can anybody provide me with a MacPorts (or otherwise open source) clang --version output so I can come up with a regex that will avoid this problem?

[mojca (Mojca Miklavec)]

> clang-mp-3.4 --version
clang version 3.4.2 (tags/RELEASE_34/dot2-final)
Target: x86_64-apple-darwin11.4.2
Thread model: posix

[mojca (Mojca Miklavec)]

Upstream report:

• ​https://github.com/openssl/openssl/issues/494

The commit that broke the build:

• ​https://github.com/openssl/openssl/commit/76eba0d94bb418325be6409b272eac5e2bd4a0a9

Relevant report that lead to that change:

• ​https://mta.openssl.org/pipermail/openssl-bugs-mod/2015-November/000248.html
• ​https://rt.openssl.org/Ticket/Display.html?user=guest&pass=guest&id=4142

The reason is Apple since Xcode 7.0 removed string "based on LLVM x.y.z" from version information. Now the clang -v command returns:

Apple LLVM version 7.0.0 (clang-700.1.76)
Target: x86_64-apple-darwin15.0.0
Thread model: posix

[mojca (Mojca Miklavec)]

A temporary workaround could be to use configure.compiler macports-clang-3.[anything]. But we should try to eventually fix the problem without blacklisting the system compiler (unless we don't want to bother fixing the regexp and want to wait for upstream to fix it; and blacklist the compiler just in the meantime).

[su-v]

Why not - as temporary workaround in MacPorts - revert that one upstream commit which broke detecting Lion's clang correctly if port is installed on Lion (*), instead of requiring the user to compile a custom clang from source?

(*)

    if {${os.major} == 11} {
        patchfiles-append   patch-revert-clang-detection-on-Lion.diff
    }

[su-v]

upstream commit reversed, patch level adjusted

[neverpanic (Clemens Lang)]

Please try the attached fix-Apple-clang-version-detection.patch and report back. My build on 10.11 is unchanged with this patch, but the real question is: does it fix your issue?

[mojca (Mojca Miklavec)]

It compiles fine. (Whether it also works properly I'm unable to tell.)

[mojca (Mojca Miklavec)]

PS: This is all Perl code. Wouldn't it be better to create a single Perl function that would return the version of clang and then the same code wouldn't have to be repeated/patched at zillion places?

[neverpanic (Clemens Lang)]

Yes, this is all Perl code that compares compiler version numbers to find out whether AVX or the addx instruction are available. Why the OpenSSL project has literally (at least) 13 copies of the same code is beyond me. I also don't understand why they compare version numbers rather than just, say, run an AVX or addx instruction through the toolchain to see if it works.

However, I'm not in the business of refactoring OpenSSL code in what seems to be rather large endeavor.

You can use the OpenSSL test suite to check whether the code miscompiles with the change. Additionally, compare the output of openssl speed with that of 1.0.2d on 10.7, that should give you a rough idea whether it is using the same assembler optimizations.

[mojca (Mojca Miklavec)]

I tried port -v test openssl which completes. There is too much output to check whether everything went fine: I assume that it would complain at the end or even break in case of problems.

[McDutchie (Martijn Dekker)]

Cc Me!

[McDutchie (Martijn Dekker)]

I can confirm that applying fix-Apple-clang-version-detection.patch​ to the build directory makes openssl @1.0.2e compile and work fine on Lion. The speed also doesn't seem to have suffered any, judging by the output of "openssl speed".

[neverpanic (Clemens Lang)]

Committed the patch in r143276.

[osth@…]

Replying to cal@…:

Committed the patch in r143276.

RESOLVED: had other clang version in /usr/bin than in XCode 5.1.1 app folder.

Solution: copy contents of /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/* to /usr/
clang -v gives then clang-503.0.40

Same problem on mountain lion (edit: only with outdated clang-500.2.79):

sh-3.2# sw_vers
ProductName:	Mac OS X
ProductVersion:	10.8.5
BuildVersion:	12F2560
sh-3.2# clang -v
Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin12.6.0
Thread model: posix

Will attach my patch which fixes it for me

[osth@…]

additional fix needed for mountain lion

[neverpanic (Clemens Lang)]

Can anybody confirm this patch is required on Mountain Lion with clang-500.2.79?

[osth@…]

Replying to cal@…:

Can anybody confirm this patch is required on Mountain Lion with clang-500.2.79?

Had installed my Apple clang as command line tools package (clang in /usr/bin/). Did not realize, that XCode 5.1.1 app folder contained a newer version of clang. - I did edit my first comment..

[neverpanic (Clemens Lang)]

Ah, so the check didn't run the compiler that ended up being used? We should probably try to avoid this…

[barracuda156]

Replying to neverpanic:

Can anybody confirm this patch is required on Mountain Lion with clang-500.2.79?

Yes, openssl3 is broken on 10.8.5 now.