py38-scipy @1.10.1_1+gfortran+openblas: Error: 'numpy' must be installed before running the build.

[odysseus9672 (Sean Lake)]

The relevant line of the build log seems to be:

:info:build Error: 'numpy' must be installed before running the build.

I have numpy installed, so maybe it depends on a specific version? What I have installed, specifically, is: py38-numpy @1.24.1

I think this also affects py27-scipy, but I uninstalled it rather than trying to get the build to work for that one.

Attached, please find the build log.

[odysseus9672 (Sean Lake)]

scipy build log.

[erikbs]

It happens with py311-scipy too. The Portfile was changed a couple of days ago, but it was only a revision bump (to trigger rebuild/update, I suppose). My NumPy version is 1.26.1. NumPy 1.24.1 came out almost a year ago, so I am pretty sure that NumPy itself is not responsible. Therefore I tried import numpy to verify that it actually worked. It did not.

The NumPy libraries fail to load because the symbol _xerbla_ is missing in OpenBLAS, which seems to be related to the lapack variant. Apparently OpenBLAS is broken without lapack too (spews out a ton of errors when I try to install it), so now I try installing numpy with +atlas instead of OpenBLAS. ATLAS takes forever to build (it must have been running for at almost an hour whilst OpenBLAS only took a couple of minutes), but if it works that is a temporary solution until OpenBLAS is fixed. Unfortunately, according to some benchmarks I found, ATLAS is slower than OpenBLAS. On the other hand there is another option, the non-free MKL library, which apparently beats the other two at performance, but I did not find out that until the ATLAS build had been running for quite a while.

[ryandesign (Ryan Carsten Schmidt)]

The atlas portfile indicates that it is only compatible with Darwin versions less than 20 (i.e. macOS versions less than 11).

[erikbs]

I am on 10.9, so that is no problem for me. The goal now is simply to find out if OpenBLAS is to blame. If the problem persist even when switching to ATLAS, something else must be the problem. Unfortunately Numpy does not want my ATLAS installation, so I am no closer to solving this.

ATLAS supports Darwin < 20, but MKL only supports Darwin > 15, so there should be temporary solutions for all platforms – if the major changes to the OpenBLAS port indeed turn out to be the cause of the problem.

I will try to see if I can make Numpy accept my ATLAS installation. If not, I will roll back the OpenBLAS port to the previous version and see what happens.

[erikbs]

It looks like the latest NumPy version does not support ATLAS at all, so I went with the backup plan and manually edited in allow_noblas = true in NumPy’s Meson file, just to get NumPy installed. Not having BLAS support is of course not acceptable, but now I can import NumPy without errors, which was the goal this time. My assumption was that once NumPy actually worked, SciPy would install. And it did. It even imported successfully, until I deactivated the BLAS-less NumPy and re-installed the one linked to OpenBLAS. Then it spews out the same import errors as import numpy.

The actual problem here therefore seems to be that the OpenBLAS port is broken after the migration to CMake, apparently by not providing all the symbols NumPy needs. Since NumPy cannot be imported successfully, SciPy thinks it is not installed and refuses to build.

Edit: I created a new ticket on OpenBLAS: #68810

[erikbs]

I believe the issue will disappear if ​https://github.com/macports/macports-ports/pull/21650 is merged.

[reneeotten (Renee Otten)]

Replying to erikbs:

I believe the issue will disappear if ​https://github.com/macports/macports-ports/pull/21650 is merged.

That PR was merged - did it fix the issue?

[cooljeanius (Eric Gallager)]

Replying to reneeotten:

Replying to erikbs:

I believe the issue will disappear if ​https://github.com/macports/macports-ports/pull/21650 is merged.

That PR was merged - did it fix the issue?

I just tested, and it seems that the answer is that no, it didn't

[jmroot (Joshua Root)]

[7800bd904376c43300d4b1e19a2e9018e75fc554/macports-ports]