OpenBLAS @0.3.7: build error when build_arch != host arch: ../kernel/x86/amax.S:64:2: error: ../kernel/x86/amax.S:instruction requires: Not 64-bit mode

[kencu (Ken)]

Something not working right with the openblas update - this is on 10.7.5:

/opt/local/bin/clang-mp-5.0 -c -O3 -DMAX_STACK_ALLOC=2048 -DEXPRECISION -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX -DNO_AVX512 -DSMP_SERVER -DMAX_CPU_NUMBER=2 -DMAX_PARALLEL_NUMBER=1 -DVERSION=\"1\" -DASMNAME=_samin_k -DASMFNAME=_samin_k_ -DNAME=samin_k_ -DCNAME=samin_k -DCHAR_NAME=\"samin_k_\" -DCHAR_CNAME=\"samin_k\" -DNO_AFFINITY -I.. -UDOUBLE  -UCOMPLEX -UCOMPLEX -UDOUBLE -DUSE_ABS  -DUSE_MIN ../kernel/x86/amax.S -o samin_k.o
../kernel/x86/amax.S:64:2: error: ../kernel/x86/amax.S:instruction requires: Not 64-bit mode
 pushl %esi
 ^
../kernel/x86/amax.S:65:2: error: instruction requires: Not 64-bit mode
 pushl %ebx
 ^
../kernel/x86/amax.S:311:2: error: instruction requires: Not 64-bit mode
 popl %ebx
 ^
../kernel/x86/amax.S:312:2: error: instruction requires: Not 64-bit mode
 popl %esi
 ^
64:2: error: instruction requires: Not 64-bit mode
 pushl %esi
 ^
../kernel/x86/amax.S:65:2: error: instruction requires: Not 64-bit mode
 pushl %ebx
 ^
../kernel/x86/amax.S:311:2: error: instruction requires: Not 64-bit mode
 popl %ebx
 ^
../kernel/x86/amax.S:312:2: error: instruction requires: Not 64-bit mode
 popl %esi
 ^
make[1]: *** [samin_k.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [samax_k.o] Error 1
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/OpenBLAS-0.3.7/kernel'
make: *** [libs] Error 1

[kencu (Ken)]

I have helpfully downloaded the prebuilt binary from the buildbot. However, it appears I still can't build openblas myself on this machine. Always a bit hard to figure those kinds of errors out...

[kencu (Ken)]

best I can figure at present, it seems to be trying to build the 32bit version of amax.S when it is building the 64bit library on this machine, instead of building the 64bit version.

[kencu (Ken)]

Same error on 10.6.8.

[kencu (Ken)]

<​https://build.macports.org/builders/ports-10.6_x86_64_legacy-builder/builds/109235/steps/install-port/logs/stdio>

[jmroot (Joshua Root)]

Could be doing something silly like using uname -m to determine the arch?

[kencu (Ken)]

for now, I pegged openblas at OpenBLAS @0.3.6_1 until I get some time to sort this out.

[kencu (Ken)]

maybe we need to look at this area in the Portfile:

        if {${build_arch} eq "x86_64" || ${build_arch} eq "ppc64"} {
            puts $makeINC "BINARY = 64"
        } else {
            puts $makeINC "BINARY = 32"
        }

[rmottola (Riccardo)]

I encountered exactly the same error on 10.7 (obviously 64bit)

[kencu (Ken)]

Replying to jmroot:

Could be doing something silly like using uname -m to determine the arch?

Indeed, clever sir, that is just exactly what it is doing:

# If ARCH is not set, we use the host system's architecture.
ifndef ARCH
ARCH := $(shell uname -m)
endif

and fixing that by specifically hacking in a setting for ARCH to x86_64:

   if { ${os.major} <= 10 } {
     puts $makeINC "NO_AVX2 = 1"
+    puts $makeINC "ARCH = x86_64"
   }

produces the expected happiness:

$ port -v installed openblas
The following ports are currently installed:
  OpenBLAS @0.3.6_1+gcc8+lapack+native platform='darwin 10' archs='x86_64' date='2019-05-26T18:25:37-0700'
  OpenBLAS @0.3.7_0+gcc8+lapack+native (active) platform='darwin 10' archs='x86_64' date='2019-10-02T19:52:08-0700'

Looks like it came in here <​https://github.com/xianyi/OpenBLAS/pull/2110> and it may already be fixed in git master <​https://github.com/xianyi/OpenBLAS/issues/2231>, so perhaps I'll just try that instead of messing about with the whole build.

[kencu (Ken)]

So yes, this issue is already fixed in the current OpenBLAS-devel port, in which OpenBLAS has more cleverness about ARCH setting.

[fvaccari]

I just tested on 10.6.8 that after adjusting

puts $makeINC "ARCH = x86_64"

another error pops up later in the compilation:

/opt/local/bin/clang-mp-9.0 -c -O3 -DMAX_STACK_ALLOC=2048 -DEXPRECISION -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX2 -DNO_AVX512 -DSMP_SERVER -DMAX_CPU_NUMBER=2 -DMAX_PARALLEL_NUMBER=1 -DVERSION=\"1\" -DASMNAME=_ssymv_U -DASMFNAME=_ssymv_U_ -DNAME=ssymv_U_ -DCNAME=ssymv_U -DCHAR_NAME=\"ssymv_U_\" -DCHAR_CNAME=\"ssymv_U\" -DNO_AFFINITY -I.. -UDOUBLE  -UCOMPLEX -UCOMPLEX -UDOUBLE -ULOWER ../kernel/x86_64/symv_U_sse.S -o ssymv_U.o
../kernel/x86_64/symv_U_sse.S:425:2: error: 32-bit absolute addressing is not supported in 64-bit mode
 PREFETCH PREFETCHSIZE(%rbx)
 ^
../kernel/x86_64/symv_U_sse.S:444:2: error: 32-bit absolute addressing is not supported in 64-bit mode
 PREFETCH PREFETCHSIZE(%r13)
 ^
../kernel/x86_64/symv_U_sse.S:469:2: error: 32-bit absolute addressing is not supported in 64-bit mode
 PREFETCH PREFETCHSIZE(%rbx, %rcx, 1)
 ^
../kernel/x86_64/symv_U_sse.S:510:2: error: 32-bit absolute addressing is not supported in 64-bit mode
 PREFETCH PREFETCHSIZE(%rbp)
 ^
../kernel/x86_64/symv_U_sse.S:529:2: error: 32-bit absolute addressing is not supported in 64-bit mode
 PREFETCHW PREFETCHSIZE(%r14)
 ^
../kernel/x86_64/symv_U_sse.S:554:2: error: 32-bit absolute addressing is not supported in 64-bit mode
 PREFETCH PREFETCHSIZE(%rbp, %rcx, 1)
 ^
make[1]: *** [ssymv_U.o] Error 1
make[1]: *** Waiting for unfinished jobs....

which actually points to ticket #58528

[kencu (Ken)]

Yes, true. I actually have this all sorted out, but just have been too distracted with all the other things going on.

The YONAH processor specified in the Portfile will not work on any 64bit OS, as YONAH is 32 bit only.

So if I manually change the processor to CORE2, or let the software work out the processor, it builds.

So -- this was a bug in OpenBLAS, and then our forcing YONAH is just not going to work. I have a fancier fix coming, using sysctl and testing for a 64-bit capable processor (and if so, forcing CORE2) but in truth, recreating all the OpenBLAS logic to sort out the processor in Portfile logic is a bit of a pain.

[rmottola (Riccardo)]

I read the portfile and it says:

#OS 10.7-10.11 supports down to iMac 7,1, with Intel Core 2 Duo architecture
#OS 10.12-13 supports down to iMac 10,1, with Intel Core 2 Duo architecture
if {${os.major} >= 11 && ${os.major} <= 17} {
    set blas_arch "CORE2"
}

So, it appears that the CPU is set correctly for 10.7 !

10.6 is:

#OS 10.6 supports down to Intel Core Solo architecture
if {${os.major} == 10} {
    set blas_arch "YONAH"
}

for 10.5 we have:

#Define target based on version of OS
#OS 10.5 supports down to Intel Core Solo (Intel) or PowerPC G4
if {${os.major} == 9} {
    if {${os.arch} eq "i386" || [variant_isset universal]} {
        set blas_arch "YONAH"
    } else {
        set blas_arch "PPCG4"
    }
}

since it is setting "down to" if you are correct that CORE2 is required, it is wrong for 10.6 and 10.5 when in universal mode!

But... why does it fail even on 10.7 with that error, I don't fully understand.

[kencu (Ken)]

There are two different errors. See also #58528.

#58528 is fixed in OpenBLAS-devel, I believe (should be, as I saw that commit roll by a while ago), but this one is our bug, and will never be fixed until we fix it.

Here is my WIP <​https://github.com/kencu/macports-ports/commit/462e990eb0208187c5abdcfa29d9507945106a81>, not quite done yet.

[rmottola (Riccardo)]

I just tried openblas-devel on 10.7 and it appears to work. on 10.5 and 10.6 I did not try yet... but I am stll puzzled by YONAH vs CORE2 and forcing it.

[kencu (Ken)]

we are forcing YONAH on systems that are sometimes 64bit, and it breaks in a 64bit build. We have to force CORE2 on 64bit systems instead.

The trick is to solve this generally for the buildbots.

I have been waiting for a new openblas release, as the other error was in openblas itself, and is fixed in openblas-devel, as you noticed.

[kencu (Ken)]

[7beaccfb4e499312fb95e5c7af7becd6b3c9cf46/macports-ports]