nss fails to build for x86_64 on 10.6.8 when forcing the build to use gcc11 instead of the default clang compiler: no such instruction: `pclmulqdq $16, %xmm2,%xmm0'

[barracuda156]

/opt/x86_64/bin/gcc-mp-11 -Os -std=c99 -arch x86_64 -o Output.OBJD/Darwin_SINGLE_SHLIB/gcm-x86.o -c -std=c99 -Os -fPIC -Dppc -fno-common -pipe -DDARWIN -DHAVE_STRERROR -DHAVE_BSD_FLOCK  -Wall -Wshadow -DNSS_NO_GCC48 -DXP_UNIX -DSHLIB_SUFFIX=\"dylib\" -DSHLIB_PREFIX=\"lib\" -DSHLIB_VERSION=\"3\" -DSOFTOKEN_SHLIB_VERSION=\"3\" -DRIJNDAEL_INCLUDE_TABLES -UDEBUG -DNDEBUG -DNSS_NO_INIT_SUPPORT -DUSE_UTIL_DIRECTLY -DNO_NSPR_10_SUPPORT -DSSL_DISABLE_DEPRECATED_CIPHER_SUITE_NAMES -DNSS_USE_64 -DNSS_X86_OR_X64 -DNSS_X64 -DUSE_HW_SHA2 -DMPI_AMD64 -DMP_IS_LITTLE_ENDIAN -DMP_ASSEMBLY_MULTIPLY -DNSS_USE_COMBA -DHAVE_INT128_SUPPORT -DMP_API_COMPATIBLE -I/opt/x86_64/include/nspr -I../../../dist/Output.OBJD/include -I../../../dist/public/nss -I../../../dist/private/nss -Impi -Iecl -Iverified -Iverified/kremlin/include -Iverified/kremlin/kremlib/dist/minimal -Ideprecated  -mpclmul -maes gcm-x86.c
{standard input}:55:no such instruction: `pclmulqdq $16, %xmm2,%xmm0'
{standard input}:56:no such instruction: `pclmulqdq $17, %xmm2,%xmm4'
{standard input}:57:no such instruction: `pclmulqdq $1, %xmm2,%xmm1'
{standard input}:61:no such instruction: `pclmulqdq $0, %xmm2,%xmm3'
make[4]: *** [Output.OBJD/Darwin_SINGLE_SHLIB/gcm-x86.o] Error 1
make[4]: Leaving directory `/opt/x86_64/var/macports/build/_opt_x86_64_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_nss/nss/work/nss-3.78/nss/lib/freebl'
make[3]: *** [freebl_FREEBL_BUILD_SINGLE_SHLIB] Error 2
make[3]: Leaving directory `/opt/x86_64/var/macports/build/_opt_x86_64_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_nss/nss/work/nss-3.78/nss/lib/freebl'
make[2]: *** [freebl] Error 2
make[2]: Leaving directory `/opt/x86_64/var/macports/build/_opt_x86_64_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_nss/nss/work/nss-3.78/nss/lib'
make[1]: *** [lib] Error 2
make[1]: Leaving directory `/opt/x86_64/var/macports/build/_opt_x86_64_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_nss/nss/work/nss-3.78/nss'
make: *** [all] Error 2

This may be related to failures of nettle and gnutls: #65147 ​https://github.com/macports/macports-ports/pull/14613

[barracuda156]

Apparently the same error, but no solution in the thread: ​https://cryptopp-users.narkive.com/eDmBiQVq/problem-building-cryptopp562-on-mac-osx10-8-using-gcc4-8-2

[barracuda156]

UPD. And while in theory there are options to disable unsupported instructions, nss ignores them.

Tried configure.cflags-append -DNSS_DISABLE_PCLMUL -DNSS_DISABLE_HW_AES, tried build.args-append NSS_DISABLE_PCLMUL=1 NSS_DISABLE_HW_AES=1, tried to cheat it into thinking it is on Nehalem where these instructions do not exist: configure.cflags-append -mtune=nehalem -mno-pclmul -mno-vpclmulqdq, nothing works. It is hopelessly broken.

[barracuda156]

Apparently this is where it has been broken:

    -  Bug 805604 - Support for AES-NI and AVX accelerated AES-GCM was contributed by Shay Gueron of
      Intel. If compiled on Linux systems in 64-bit mode, NSS will include runtime detection to
      check if the platform supports AES-NI and PCLMULQDQ. If so, NSS uses the optimized code path,
      reducing the CPU cycles per byte to 1/20 of what was required before the patch
      (https://bugzilla.mozilla.org/show_bug.cgi?id=805604 and
      https://crypto.stanford.edu/RealWorldCrypto/slides/gueron.pdf). Support for other platforms,
      such as Windows, will follow in a future NSS release.
      (https://bugzilla.mozilla.org/show_bug.cgi?id=540986)

[kencu (Ken)]

these newer AVX assembly instructions are unknown to the 2004-vintage assembler that comes with cctools, or that comes with the system.

To get around this, MacPorts cctools will use a newish clang to assemble instead, if it is available. So we dealt with this error some years ago.

So to fix this, use MacPorts cctools assembler, and make sure a newish clang like clang-11 is installed.

If that still doesn’t work, you’re likely not actually using the MacPorts assembler, or something has broken the mechanism.

[kencu (Ken)]

once again though, MacPirts is set up to use clang on SnowLeopard, which works, and there is no expectation or requirement that people who want to force their own off-the-mainstream installation paths should expect to meet success, or expect anyone else to fix this for them

[barracuda156]

Replying to kencu:

once again though, MacPorts is set up to use clang on SnowLeopard, which works, and there is no expectation or requirement that people who want to force their own off-the-mainstream installation paths should expect to meet success, or expect anyone else to fix this for them

The port does not even have a maintainer, and I generally do not expect someone to fix things for me. For one thing though, most of the errors I report are potentially relevant for others (aside of Rosetta, that no one gonna build), so there may be an interest to fix them unrelated to assisting with specific ticket. On the other hand, suggestions can make a big deal. It is not always obvious what to try, especially when standard options were tried and failed.

[barracuda156]

nss is horribly designed though. It should be quite obvious that cpu-specific stuff should have a setting to disable it (and it is obvious to most other developers, as other ports has such options), and compilers should not be hard-wired. Not only nss fails this, it also disregards its own settings and gcc flags. This problem should have been a matter of a trivial fix in the portfile. Besides, as you have seen in another ticket, it also failed for ppc64, where there is no clang at all. It is just a bad design IMO.

[barracuda156]

Replying to kencu:

By the way, can't we use nasm for cases where as fails (on x86_64)? I have seen some ports use it, but unsure about overall compatibility.

[kencu (Ken)]

but we have no need for nasm, as clang works much better!

Now -- if you're off on a tangent trying to build x86_64 assembly on some system that clang will not properly build or run on, maybe I guess...

[kencu (Ken)]

regarding nss's build idiosyncrasies... no argument. Software should be well designed and have toggles for things that might not be broadly compatible.

My experience with taking such issues upstream is -- spotty. People are busy. If their code "coverage" encompasses the systems that are commonly in use, they are moving on to new features and bug fixes rather than adding support for ancient systems.

But it varies. Feel free to find the upstream for nss and open an issue with them!

[mouse07410 (Mouse)]

I'm a latecomer here, and don't have the full context. But, as I've done a lot with compiling optimized crypto code on Mac (outside of Macports):

In my experience, Clang requires flags -maesni -mpcmul for code like this. GCC, in addition to that, requires a different assembler, like env variable AS_INTEGRATED_ASSEMBLER=1 and CFLAGS += -Wa,-q -Wa,-march=native (re. -march: maybe -maesni -mpcmul instead).

[kencu (Ken)]

Replying to mouse07410:

In my experience, Clang requires flags -maesni -mpcmul for code like this.

I haven't needed that, but we can explore some examples to prove or disprove the point on godbolt.org if you like.

GCC, in addition to that, requires a different assembler, like env variable AS_INTEGRATED_ASSEMBLER=1 and CFLAGS += -Wa,-q -Wa,-march=native

Those things used to be needed to force the standard cctools assembler to send the code to clang to assemble, but those are no longer needed for the past several years in MacPorts when our MacPorts cctools assembler was changed to "do the right thing" by default.

See this part of the cctools Portfile for our modifications that makes this not needed for MacPorts:

	        # List of clang versions to search for at runtime internally by 'as'
	        # Build list from llvm variants, excluding clang < 5 and devel version
	        # (unless devel variant is used in which case it is included first)
	        set as_comps ""
	        foreach variantname ${all_llvm_variants} {
	            set ver $llvm_variant_version($variantname)
	            # Use all non-devel and clang > 4 versions
	            if { ${ver} ne "devel" && ${ver} > 4.999 } {
	                if { ${as_comps} ne "" } {
	                    set as_comps ",${as_comps}"
	                }
	                set as_comps "\"clang-mp-${ver}\"${as_comps}"
	            }
	        }
	        # If llvmdev variant is active, include at start of list
	        if {[variant_isset llvmdev]} {
	            set as_comps "\"clang-mp-devel\",${as_comps}"
	        }
	        ui_debug "as compiler list ${as_comps}"
	        reinplace "s:__MP_CLANG_NAMES__:${as_comps}:" ${worksrcpath}/as/driver.c
	
	        if {${os.major} >= 11} {
	            set try_system_clang 1
	        } else {
	            # clang's integrated assembler may not work well on 10.6 and doesn't
	            # exist on older OS versions.
	            set try_system_clang 0
	        }
	        reinplace "s:__TRY_SYSTEM_CLANG__:${try_system_clang}:" ${worksrcpath}/as/driver.c
	

[mouse07410 (Mouse)]

Hard to tell. I don't recall every details, but sometimes -march=native provided for -maes etc. on the appropriate CPU.

Yes, if you could explore it on goldbots, I think it would be great. Thanks!

[barracuda156]

Replying to mouse07410:

I'm a latecomer here, and don't have the full context. But, as I've done a lot with compiling optimized crypto code on Mac (outside of Macports):

In my experience, Clang requires flags -maesni -mpcmul for code like this. GCC, in addition to that, requires a different assembler, like env variable AS_INTEGRATED_ASSEMBLER=1 and CFLAGS += -Wa,-q -Wa,-march=native (re. -march: maybe -maesni -mpcmul instead).

It builds on PPC with an old assembler with no problems (I think Macports as, correct me if I am wrong). So it should build on Intel with the same too. The problem is that certain cpu-specific improvements were hard-wired into nss code without thinking through that they gonna break the code on old Intel systems.

Solution should be to make cpu-specific instructions optional, not to force everyone use an external assembler from a specific vendor. IMO.

16. S. As a side result, on Rosetta such code is completely broken, since Intel instructions are applied when the build is for PPC.

[barracuda156]

Replying to kencu:

but those are no longer needed for the past several years in MacPorts when our MacPorts cctools assembler was changed to "do the right thing" by default.

In my humble opinion, the baseline case should be using Apple own assembler as it exists for each OS. Using clang may be desirable generally on Intel, but every system should be able to function with OS cctools + Macports cctools + gcc.

[barracuda156]

For the record, same failure with gcc12:

/opt/x86_64/bin/gcc-mp-12 -Os -std=c99 -arch x86_64 -o Output.OBJD/Darwin_SINGLE_SHLIB/gcm-x86.o -c -std=c99 -Os -fPIC -Dppc -fno-common -pipe -DDARWIN -DHAVE_STRERROR -DHAVE_BSD_FLOCK  -Wall -Wshadow -DNSS_NO_GCC48 -DXP_UNIX -DSHLIB_SUFFIX=\"dylib\" -DSHLIB_PREFIX=\"lib\" -DSHLIB_VERSION=\"3\" -DSOFTOKEN_SHLIB_VERSION=\"3\" -DRIJNDAEL_INCLUDE_TABLES -UDEBUG -DNDEBUG -DNSS_NO_INIT_SUPPORT -DUSE_UTIL_DIRECTLY -DNO_NSPR_10_SUPPORT -DSSL_DISABLE_DEPRECATED_CIPHER_SUITE_NAMES -DNSS_USE_64 -DNSS_X86_OR_X64 -DNSS_X64 -DUSE_HW_SHA2 -DMPI_AMD64 -DMP_IS_LITTLE_ENDIAN -DMP_ASSEMBLY_MULTIPLY -DNSS_USE_COMBA -DHAVE_INT128_SUPPORT -DMP_API_COMPATIBLE -I/opt/x86_64/include/nspr -I../../../dist/Output.OBJD/include -I../../../dist/public/nss -I../../../dist/private/nss -Impi -Iecl -Iverified -Iverified/kremlin/include -Iverified/kremlin/kremlib/dist/minimal -Ideprecated  -mpclmul -maes gcm-x86.c
{standard input}:55:no such instruction: `pclmulqdq $16, %xmm2,%xmm0'
{standard input}:56:no such instruction: `pclmulqdq $17, %xmm2,%xmm4'
{standard input}:57:no such instruction: `pclmulqdq $1, %xmm2,%xmm1'
{standard input}:61:no such instruction: `pclmulqdq $0, %xmm2,%xmm3'
make[4]: *** [Output.OBJD/Darwin_SINGLE_SHLIB/gcm-x86.o] Error 1
make[4]: Leaving directory `/opt/x86_64/var/macports/build/_opt_x86_64_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_nss/nss/work/nss-3.84/nss/lib/freebl'
make[3]: *** [freebl_FREEBL_BUILD_SINGLE_SHLIB] Error 2
make[3]: Leaving directory `/opt/x86_64/var/macports/build/_opt_x86_64_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_nss/nss/work/nss-3.84/nss/lib/freebl'
make[2]: *** [freebl] Error 2
make[2]: Leaving directory `/opt/x86_64/var/macports/build/_opt_x86_64_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_nss/nss/work/nss-3.84/nss/lib'
make[1]: *** [lib] Error 2
make[1]: Leaving directory `/opt/x86_64/var/macports/build/_opt_x86_64_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_nss/nss/work/nss-3.84/nss'
make: *** [all] Error 2

Submitted the bug to upstream: ​https://bugzilla.mozilla.org/show_bug.cgi?id=1799377

[kencu (Ken)]

these newer AVX assembly instructions are unknown to the 2004-vintage assembler that comes with cctools, or that comes with the system.

To get around this, MacPorts cctools will use a newish clang to assemble instead, if it is available. So we dealt with this error some years ago.

So to fix this, use MacPorts cctools assembler, and make sure a newish clang like clang-11 is installed.

Regarding:

In my experience, Clang requires flags -maesni -mpcmul for code like this. GCC, in addition to that, requires a different assembler, like env variable AS_INTEGRATED_ASSEMBLER=1 and CFLAGS += -Wa,-q -Wa,-march=native (re. -march: maybe -maesni -mpcmul instead).

The assembler forcing should not be needed on MacPorts, because of the above modification we made to cctools. I'm not sure about the assembly-enabling flags, though.

Regarding:

In my humble opinion, the baseline case should be using Apple own assembler as it exists for each OS. Using clang may be desirable generally on Intel, but every system should be able to function with OS cctools + Macports cctools + gcc.

Then you would have to instead find a way to disable that troublesome assembly on older systems where the system toolchain can't understand it. Some ports have done that. For the general case, MacPorts decided to go a different way, and support the assembly, instead of that.

Summary:

1. on Intel systems, blacklist all clangs that don't support that assembly. A little trial and error should show you, but you can google the details of the which-clang-supported-which assembly-instructions support if you want. Let the build build with clang. All will be well.

2. On other Intel systems where gcc is the default compiler (10.4 and 10.5 Intel at present), you have to have a new enough clang installed as above that can understand that assembly. You may need to "depends_build-append" this compiler.

3. On PPC, you should be fine and this file should not be part of the build anyway. If it is part of the build, might have to either patch it out or do as per point 2.

4. On arm, the only clangs that work are already new enough to handle this assembly, so there should be no issues at present.