unable to install py39-numpy because of OpenBLAS build failure

[giovannicantele]

Hi all, I'm trying to sudo port install py39-numpy Python was working until two days ago, when I unfortunately decided to port upgrade outdated. Since everything was broken, I decided to uninstall all ports and macports and install them back again.

Up to a certain point everything has worked. But now I need numpy to run my python scripts. If I try to install it, it fails because OpenBLAS is now broken.

Looking at /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/main.log I get the following errors:

:info:build ld: file not found: loader_path
:info:build collect2: error: ld returned 1 exit status
:info:build make[1]: *** [libopenblas-r1.dylib] Error 1
[...]
:info:build Exit code: 2
:error:build Failed to build OpenBLAS: command execution failed
:debug:build Error code: CHILDSTATUS 83571 2
:debug:build Backtrace: command execution failed
:debug:build     while executing
:debug:build "system {*}$notty {*}$nice $fullcmdstring"
:debug:build     invoked from within
:debug:build "command_exec build"
:debug:build     (procedure "portbuild::build_main" line 8)
:debug:build     invoked from within
:debug:build "$procedure $targetname"

I'm quite desperate, I don't know how to rebuild my python installation.

Can anyone help me?

[reneeotten (Renee Otten)]

just to make sure, first let's try: sudo port selfupdate && sudo port clean --all py39-numpy openblas, then try to install NumPy again: sudo port install py39-numpy. If that still fails, please attach the full log file to this tickets as the message will tell you to do.

[NicosPavlov]

The error might also be a duplicate of #62856. If the suggestion above does not work, you could check which gcc* and libgcc* ports you have installed.

[giovannicantele]

/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/main.log

[giovannicantele]

Hi all, thank you for prompt replay. I tried as suggested with sudo port selfupdate && sudo port clean --all py39-numpy openblas, then try to install NumPy again: sudo port install py39-numpy, but unfortunately the OpenBLAS issue is still there. The installation exits with the error

Error: Failed to build OpenBLAS: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.
Error: Processing of port py39-numpy failed

and the mentioned main.log file is attached.

As requested, here the list of gcc* and libgcc* that have been previously installed:

  gcc11 @11.1.0_1 (active)
  gcc_select @0.1_9 (active)
  libgcc @4.0_0 (active)
  libgcc11 @11.1.0_1 (active)

[giovannicantele]

now things are going worse. I tried again by uninstalling all ports and install them back again. Everything works fine until I try to install py39-numpy. The new error is now

Error: Failed to install libgcc11: no destroot found at: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc11/libgcc11/work/destroot
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc11/libgcc11/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.
Error: Processing of port py39-numpy failed

[NicosPavlov]

This last issue often happens when the port is not properly cleaned, you could try to clean libgcc11:

sudo port clean libgcc11

For the initial issue, gcc11 is a rather new port, and issues with @rpath are known, you can also refer to #62727. A possible workaround is to install OpenBLAS with the gcc10 variant.

[giovannicantele]

Thank you. I'm sorry to bother you. I tried sudo port install openblas +gcc10 but got the error

:info:build Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/OpenBLAS-0.3.15" && /usr/bin/make -j8 -w all AR=/opt/local/bin/ar RANLIB=/opt/local/bin/ranlib CC="/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/compwrap/cc/usr/bin/clang" CXX="/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/compwrap/cxx/usr/bin/clang++" OBJC="/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/compwrap/objc/usr/bin/clang" OBJCXX="/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/compwrap/objcxx/usr/bin/clang++" FC="/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/compwrap/fc/opt/local/bin/gfortran-mp-10" INSTALL="/usr/bin/install -c"
:info:build Exit code: 2
:error:build Failed to build OpenBLAS: command execution failed
:debug:build Error code: CHILDSTATUS 2316 2
:debug:build Backtrace: command execution failed
:debug:build     while executing
:debug:build "system {*}$notty {*}$nice $fullcmdstring"
:debug:build     invoked from within
:debug:build "command_exec build"
:debug:build     (procedure "portbuild::build_main" line 8)
:debug:build     invoked from within
:debug:build "$procedure $targetname"
:error:build See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/main.log for details.

OpenBLAS seems to be definitely broken. Has anyone managed to build it successfully? In the case, could you please share the full list of commands?

Thank you again.

[NicosPavlov]

Replying to giovannicantele:

OpenBLAS seems to be definitely broken. Has anyone managed to build it successfully? In the case, could you please share the full list of commands?

OpenBLAS builds fine on most system, apart from ARM where there are some issues recently (see ​https://ports.macports.org/port/OpenBLAS/summary).

Please post the main.log file as instructed instead of a copy of part of the build log, this would make determining the error much easier. Ideally, you should run

sudo port clean openblas
sudo port install openblas +gcc10

to ensure having a log from a clean build.

[giovannicantele]

I'm going to attach the log of

sudo port clean openblas
sudo port install openblas +gcc10

What is a bit strange is that everything was working perfectly until few days ago, and what broke everything was the {{sudo port selfupdate}} of yesterday or the day before yesterday.

[giovannicantele]

log file for sudo port clean openblas, sudo port install openblas +gcc10

[cjones051073 (Chris Jones)]

OpenBLAS indeed has issues with its build setup that have been brought to light with recent changes in the gcc11 and newer branches used for arm (and not, yet, intel).

The loader_path issue above is something the OpenBLAS needs to address itself, but for now I have removed it from the list used with arm

​https://github.com/macports/macports-ports/commit/aa09913a124491a4016e853f2095c577e8cb969e

[giovannicantele]

Replying to cjones051073:

OpenBLAS indeed has issues with its build setup that have been brought to light with recent changes in the gcc11 and newer branches used for arm (and not, yet, intel).

The loader_path issue above is something the OpenBLAS needs to address itself, but for now I have removed it from the list used with arm

​https://github.com/macports/macports-ports/commit/aa09913a124491a4016e853f2095c577e8cb969e

So, do I have any chance to make my macports working again (many ports cannot be built due to OpenBLAS, for example gimp) or I should give up using macports and any other related package until this issue is solved? Such an issue has been introduced recently, everything was working fine until few days (weeks?) ago.

[cjones051073 (Chris Jones)]

Look again at my commit.....

I have reverted OpenBLAS back to using gcc10 on arm, which is fine.

[cjones051073 (Chris Jones)]

Hmmm, unfortunately whilst it previously built fine with gcc10, it seems not to any more...

​https://build.macports.org/builders/ports-11_arm64-builder/builds/20453/steps/install-port/logs/stdio

not sure why, and without access to an arm machine I cannot investigate. Someone with one of these machines needs to look into this.

[cjones051073 (Chris Jones)]

See ​https://trac.macports.org/ticket/62727

[giovannicantele]

Replying to cjones051073:

Look again at my commit.....

I have reverted OpenBLAS back to using gcc10 on arm, which is fine.

Unfortunately, it seems that when installing other ports, just like gimp, they request OpenBLAS with gcc11 variant, end this conflicts with OpenBLAS reverted back to using gcc10:

:error:clean Requested variants "+gcc11+lapack+native" do not match those the build was started with: "+gcc10+lapack+native".
:error:clean Please use the same variants again, or run 'port clean OpenBLAS' first to remove the existing partially completed build.

After sudo port clean openblas}} and then again, for example, {{{sudo port install gimp, the loader_path error comes back again:

:info:build ld: file not found: loader_path
:info:build collect2: error: ld returned 1 exit status
:info:build make[1]: *** [libopenblas-r1.dylib] Error 1

Finally, after sudo port selfupdate, sudo port upgrade outdated, sudo port clean openblass, then sudo port install gimp again, still another error is found:

:info:build OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 ./sblat2 < ./sblat2.dat
:info:build dyld: Library not loaded: @rpath/libgfortran.5.dylib
:info:build   Referenced from: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/xianyi-OpenBLAS-904f9a2/test/./sblat1
:info:build   Reason: image not found
:info:build make[1]: *** [level1] Abort trap: 6
:info:build make[1]: *** Waiting for unfinished jobs....
:info:build dyld: Library not loaded: @rpath/libgfortran.5.dylib
:info:build   Referenced from: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/xianyi-OpenBLAS-904f9a2/test/./sblat2
:info:build   Reason: image not found
:info:build /bin/sh: line 1: 78272 Abort trap: 6           OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 ./sblat2 < ./sblat2.dat
:info:build make[1]: *** [level2] Error 134
:info:build make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/xianyi-OpenBLAS-904f9a2/test'
:info:build make: *** [tests] Error 2
:info:build make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_so

[cjones051073 (Chris Jones)]

See

​https://github.com/xianyi/OpenBLAS/issues/3222

looks like I might be able to pull a patch from there...

[cjones051073 (Chris Jones)]

​https://github.com/macports/macports-ports/commit/50d3620eda7ab77ecc3d3867fc37f7a4eac15709

I cannot test on arm directly, but it built fine on intel.

[cjones051073 (Chris Jones)]

just built fine on arm in the buildbots, so closing this.

[giovannicantele]

Replying to cjones051073:

just built fine on arm in the buildbots, so closing this.

It doesn't work to me

:info:build dyld: Library not loaded: @rpath/libgfortran.5.dylib

(just trying to sudo port install openblass after cleaning and then uninstalling all installed ports).

[NicosPavlov]

Did you run

sudo port self update

to synchronize the new changes before trying to install?