lapack @3.12.0_1: cmake config files can't be found automatically. They should be moved into ${prefix}/lib/cmake where other ports put their cmake config files.

[lepus2589 (Tim Kaune)]

The pull request updating LAPACK from 3.9.0 to 3.10.0 introduced a new location of the cmake files provided by the library (​https://github.com/macports/macports-ports/pull/12106/commits). They moved from ${destroot}${prefix}/lib/${name}/cmake to ${destroot}${cmake_share_module_dir} .

This would be correct (I guess), if the cmake files provided by LAPACK were find-modules. But LAPACK provides config-modules (see ​https://cmake.org/cmake/help/latest/command/find_package.html for the difference).

As far as I can see, the old location was correct for config modules. In fact, with the latest port version, cmake cannot find LAPACK or LAPACKE on my machine using find_package(LAPACK CONFIG REQUIRED) (the CONFIG keyword forces cmake to look for config modules and skips the cmake-shipped find module findLAPACK, which does not use these cmake files).

[lepus2589 (Tim Kaune)]

Also, moving these files post-build is a bad idea, because they make assumptions about the location of their dependents (LAPACKE->LAPACK->BLAS). lapacke-config.cmake for example sets the LAPACK_DIR explicitly to let the cmake find_package() function find the LAPACK package immediately without search.

I would suggest to simply revert that one commit from the above PR.

[lepus2589 (Tim Kaune)]

I tested the discovery of LAPACK by cmake with the macports commit [3cfffa50a7a405392eddd54fdb5ffb5ac6e91f6a/macports-ports], right before the movement of the cmake files.

It still doesn't work. The paths were

/opt/local/lib/lapack/cmake/cblas-3.10.0/cblas-config-version.cmake
/opt/local/lib/lapack/cmake/cblas-3.10.0/cblas-config.cmake
/opt/local/lib/lapack/cmake/cblas-3.10.0/cblas-targets-release.cmake
/opt/local/lib/lapack/cmake/cblas-3.10.0/cblas-targets.cmake
/opt/local/lib/lapack/cmake/lapack-3.10.0/lapack-config-version.cmake
/opt/local/lib/lapack/cmake/lapack-3.10.0/lapack-config.cmake
/opt/local/lib/lapack/cmake/lapack-3.10.0/lapack-targets-release.cmake
/opt/local/lib/lapack/cmake/lapack-3.10.0/lapack-targets.cmake
/opt/local/lib/lapack/cmake/lapacke-3.10.0/lapacke-config-version.cmake
/opt/local/lib/lapack/cmake/lapacke-3.10.0/lapacke-config.cmake
/opt/local/lib/lapack/cmake/lapacke-3.10.0/lapacke-targets-release.cmake
/opt/local/lib/lapack/cmake/lapacke-3.10.0/lapacke-targets.cmake

The paths traversed by cmake's find_package() (​https://cmake.org/cmake/help/latest/command/find_package.html#id7) are

<prefix>/
<prefix>/(cmake|CMake)/
<prefix>/<name>*/
<prefix>/<name>*/(cmake|CMake)/
<prefix>/(lib/<arch>|lib*|share)/cmake/<name>*/
<prefix>/(lib/<arch>|lib*|share)/<name>*/
<prefix>/(lib/<arch>|lib*|share)/<name>*/(cmake|CMake)/
<prefix>/<name>*/(lib/<arch>|lib*|share)/cmake/<name>*/
<prefix>/<name>*/(lib/<arch>|lib*|share)/<name>*/
<prefix>/<name>*/(lib/<arch>|lib*|share)/<name>*/(cmake|CMake)/

<prefix> is a list containing /opt/local for the cmake port. The LAPACK paths don't fit any of these globbing patterns.

The lapack subdirectory would have to be omitted from the paths like so:

/opt/local/lib/cmake/cblas-3.10.0/cblas-config-version.cmake
/opt/local/lib/cmake/cblas-3.10.0/cblas-config.cmake
/opt/local/lib/cmake/cblas-3.10.0/cblas-targets-release.cmake
/opt/local/lib/cmake/cblas-3.10.0/cblas-targets.cmake
/opt/local/lib/cmake/lapack-3.10.0/lapack-config-version.cmake
/opt/local/lib/cmake/lapack-3.10.0/lapack-config.cmake
/opt/local/lib/cmake/lapack-3.10.0/lapack-targets-release.cmake
/opt/local/lib/cmake/lapack-3.10.0/lapack-targets.cmake
/opt/local/lib/cmake/lapacke-3.10.0/lapacke-config-version.cmake
/opt/local/lib/cmake/lapacke-3.10.0/lapacke-config.cmake
/opt/local/lib/cmake/lapacke-3.10.0/lapacke-targets-release.cmake
/opt/local/lib/cmake/lapacke-3.10.0/lapacke-targets.cmake

Can we do that by adjusting LAPACK build variables or does this have to be fixed upstream? Again, we cannot simply move the cmake files after the build.

[lepus2589 (Tim Kaune)]

The CMAKE_INSTALL_LIBDIR variable is responsible for this. Unfortunately, the LAPACK project does not provide a way to install the cmake files only into a different subdirectory. Also, it does not provide a way to install files belonging to the three libraries (BLAS, LAPACK, LAPACKE) into different folders. The project does this for the cmake files, but not for the header files or library files.

The only solution I can see to make this work is to remove the port name from the CMAKE_INSTALL_INCLUDEDIR, the CMAKE_INSTALL_LIBDIR and the CMAKE_INSTALL_RPATH. As long as the three libraries provided by the LAPACK project are put under a common lapack folder, the cmake autodiscovery won't work.

The CMAKE_INSTALL_NAME_DIR can be removed, as far as I can see. It is not used in the LAPACK build process.

[lepus2589 (Tim Kaune)]

A patch removing the port name 'lapack' from the install paths. Also removes the unnecessery variable CMAKE_INSTALL_NAME_DIR.

[lepus2589 (Tim Kaune)]

With the patch, the +openblas variant has a conflict. openblas also provides the LAPACKE headers in the default location /opt/local/include. At least the default variant of openblas+gcc11+lapack does. Can we somehow depend on a non default variant like openblas+gcc11-lapack?

[michaelld (Michael Dickens)]

​https://github.com/macports/macports-ports/pull/17207 should address this

[ryandesign (Ryan Carsten Schmidt)]

The PR was merged in January.

[lepus2589 (Tim Kaune)]

Sorry for the late reply. Unfortunately, the linked PR did not fix this! The problematic lines in the Portfile are ​math/lapack/Portfile#L77-L78. They move the CMake scripts to the MacPorts CMake Module folder post-build!

TL;DR: Don't move the CMake scripts after build!

This is wrong in two ways: First, the CMake scripts provided by the reference lapack project are config modules. They don't belong into the CMake Modules folder. They belong exactly, where the project places them. Which leads to the second problem: Config module scripts don't need to be and, in fact, are often not relocatable. They may contain hard-coded paths specific to the environment the package was configured in. Moving them post-build just breaks them.

The intended install location of these CMake scripts is (for LAPACKE) ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}/cmake/${LAPACKELIB}-${LAPACK_VERSION} , which should be /opt/local/lib/cmake/lapacke-3.12.0/ for the latest version, which is also a location find_package(lapacke) would search by default (see default search paths above). Moving this to /opt/local/share/cmake/Modules/lapacke-3.12.0/ , which is the MacPorts {cmake_share_module_dir} , breaks them. The same applies to cblas and lapack, of course.

---

Also, the Portfile overwrites CMAKE_INSTALL_LIBDIR to be lib/lapack instead of lib. I can see, why this is being done (to place the binaries into a subfolder), but unfortunately, this also places the CMake scripts into a non-default location. This is a separate issue. It is also something, that can be worked around and also happens at CMake configure time, at least, so the generated CMake config files are consistent. The LAPACK project doesn't offer a separate variable to configure the install location of the CMake scripts, so I can't really think of a solution to this.

[kencu (Ken)]

can you point us to an example where the files are not being properly found, so we can use that to verify a proper fix for this?

[lepus2589 (Tim Kaune)]

Any minimal CMakeLists.txt file will do:

cmake_minimum_required(VERSION 3.20)

project(DiscoverLAPACKE VERSION 1.0.0 LANGUAGES C)

find_package(lapacke CONFIG REQUIRED)

Calling CMake with this file like this:

$ cmake -B ./build .
CMake Error at CMakeLists.txt:5 (find_package):
  Could not find a package configuration file provided by "lapacke" with any
  of the following names:

    lapackeConfig.cmake
    lapacke-config.cmake

  Add the installation prefix of "lapacke" to CMAKE_PREFIX_PATH or set
  "lapacke_DIR" to a directory containing one of the above files.  If
  "lapacke" provides a separate development package or SDK, be sure it has
  been installed.

OK, it's in a non-standard location. Let's help CMake:

$ cmake -B ./build -D "lapacke_DIR=/opt/local/share/cmake/Modules/lapacke-3.12.0" .
-- Could NOT find LAPACK (missing: LAPACK_DIR)

It doesn't throw an error, but doesn't discover lapack in the adjacent directory. Linking an actual project with this configuration would be impossible. Additionally providing the lapack_DIR variable on the command line won't help, because the lapacke-config.cmake overwrites it with an absolute path, which is wrong because it was moved.

[kencu (Ken)]

ok, thanks, that will give us a start.

I think the different handling that appears to be required for config modules is throwing off the situation here.

[kencu (Ken)]

Ok, well I know we don't care about or mix in homebrew here, but just as a point test, out-of-the-box homebrew does no better with that CMakeLists.txt files:

% cmake .
-- The C compiler identification is AppleClang 15.0.0.15000309
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
CMake Error at CMakeLists.txt:5 (find_package):
  Could not find a package configuration file provided by "lapacke" with any
  of the following names:

    lapackeConfig.cmake
    lapacke-config.cmake

  Add the installation prefix of "lapacke" to CMAKE_PREFIX_PATH or set
  "lapacke_DIR" to a directory containing one of the above files.  If
  "lapacke" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!

% brew ls -v lapack
/opt/homebrew/Cellar/lapack/3.12.0/INSTALL_RECEIPT.json
/opt/homebrew/Cellar/lapack/3.12.0/LICENSE
/opt/homebrew/Cellar/lapack/3.12.0/.brew/lapack.rb
/opt/homebrew/Cellar/lapack/3.12.0/include/lapacke_utils.h
/opt/homebrew/Cellar/lapack/3.12.0/include/lapacke.h
/opt/homebrew/Cellar/lapack/3.12.0/include/lapacke_config.h
/opt/homebrew/Cellar/lapack/3.12.0/include/lapack.h
/opt/homebrew/Cellar/lapack/3.12.0/include/lapacke_mangling.h
/opt/homebrew/Cellar/lapack/3.12.0/README.md
/opt/homebrew/Cellar/lapack/3.12.0/lib/liblapacke.3.dylib
/opt/homebrew/Cellar/lapack/3.12.0/lib/liblapack.dylib
/opt/homebrew/Cellar/lapack/3.12.0/lib/pkgconfig/blas.pc
/opt/homebrew/Cellar/lapack/3.12.0/lib/pkgconfig/lapacke.pc
/opt/homebrew/Cellar/lapack/3.12.0/lib/pkgconfig/lapack.pc
/opt/homebrew/Cellar/lapack/3.12.0/lib/cmake/lapack-3.12.0/lapack-targets-release.cmake
/opt/homebrew/Cellar/lapack/3.12.0/lib/cmake/lapack-3.12.0/lapack-config.cmake
/opt/homebrew/Cellar/lapack/3.12.0/lib/cmake/lapack-3.12.0/lapack-config-version.cmake
/opt/homebrew/Cellar/lapack/3.12.0/lib/cmake/lapack-3.12.0/lapack-targets.cmake
/opt/homebrew/Cellar/lapack/3.12.0/lib/cmake/lapacke-3.12.0/lapacke-config-version.cmake
/opt/homebrew/Cellar/lapack/3.12.0/lib/cmake/lapacke-3.12.0/lapacke-config.cmake
/opt/homebrew/Cellar/lapack/3.12.0/lib/cmake/lapacke-3.12.0/lapacke-targets.cmake
/opt/homebrew/Cellar/lapack/3.12.0/lib/cmake/lapacke-3.12.0/lapacke-targets-release.cmake
/opt/homebrew/Cellar/lapack/3.12.0/lib/libblas.3.dylib
/opt/homebrew/Cellar/lapack/3.12.0/lib/liblapacke.3.12.0.dylib
/opt/homebrew/Cellar/lapack/3.12.0/lib/liblapack.3.12.0.dylib
/opt/homebrew/Cellar/lapack/3.12.0/lib/liblapack.3.dylib
/opt/homebrew/Cellar/lapack/3.12.0/lib/libblas.dylib
/opt/homebrew/Cellar/lapack/3.12.0/lib/libblas.3.12.0.dylib
/opt/homebrew/Cellar/lapack/3.12.0/lib/liblapacke.dylib

and no combination of setting the CMAKE_PREFIX_PATH or LAPACK_DIR makes any difference:

% cmake -DLAPACK_DIR=/opt/homebrew/Cellar/lapack/3.12.0/lib/cmake/lapack-3.12.0/  .
CMake Error at CMakeLists.txt:5 (find_package):
  Could not find a package configuration file provided by "lapacke" with any
  of the following names:

    lapackeConfig.cmake
    lapacke-config.cmake

  Add the installation prefix of "lapacke" to CMAKE_PREFIX_PATH or set
  "lapacke_DIR" to a directory containing one of the above files.  If
  "lapacke" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!

[lepus2589 (Tim Kaune)]

For me, the Homebrew installation works. The files are sym-linked into /opt/homebrew/opt/lapack. Also, if you try to discover lapacke in your CMakeLists.txt, use the lapacke_DIR CMake variable, not LAPACK_DIR.

With my example CMakeLists.txt from above and the following command line, the Homebrew lapack install works:

$ cmake -B ./build -D "lapacke_DIR=/opt/homebrew/opt/lapack/lib/cmake/lapacke-3.12.0" .
-- The C compiler identification is Clang 17.0.6
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/local/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Configuring done (1.3s)
-- Generating done (0.0s)
-- Build files have been written to: ./build

CMake succeeds without further comment. In my ./build/CMakeCache.txt, I have the line lapacke_DIR:UNINITIALIZED=/opt/homebrew/opt/lapack/lib/cmake/lapacke-3.12.0.

I can even use the more general form

$ cmake -B ./build -D "CMAKE_PREFIX_PATH=/opt/homebrew/opt/lapack" .

So, I still have to help CMake discover the initial config file, but the config files are consistent and can discover each other automatically.

[kencu (Ken)]

looks like your homebrew example is discovering things in your macports installation.

Anyway the homebrew thing was just a brief aside to indicate they don’t have it nailed out of the box either. I have lapacke installed there exactly as homebrew installs it, unaltered.

We’ll keep working on this to try to get it sorted out.

[lepus2589 (Tim Kaune)]

Yes, I know. The compiler is from MacPorts, as well as CMake itself. As far as I'm aware, this should be irrelevant for this specific issue.

Which CMake distribution and which compiler I'm using does have an impact on the default install prefixes that will be searched by the respective CMake executable. But as both the MacPorts and the Homebrew lapack are in non-default locations anyway, this should be irrelevant. I can still test, if the lapack config files are consistent among themselves. The Homebrew ones are, the MacPorts ones are not.

[kencu (Ken)]

It could be relevant. The cmake in MacPorts is specifically altered to look in specific places, and it is not the same as the generic cmake from upstream.

And the cmake port in MacPorts also has a flaw wherein it will look in /opt/homebrew for some files to use (hardcoded). I patched that flaw out in the cmake-devel port but it still exists in the cmake port.

[lepus2589 (Tim Kaune)]

Good thing I use the cmake-devel port, then. :-D Should've mentioned that.

[kencu (Ken)]

So a bit of confusion with lapack and lapacke.

It looks like both names refer to the same thing, but when the dir is being set, you have to use the dir name (with e or no-e) that matches the package you are searching for (with e or no-e). Which is obvious once you see what is going on, but wasn't so obvious when I was first testing it out.

Paying attention to that, setting the dir manually allows the files to be found.

And moving the config files into /opt/local/lib/cmake/lapack-3.12.0 and /opt/local/lib/cmake/lapacke-3.12.0 and /opt/local/share/cmake/Modules/cblas-3.12.0 allows cmake to find them without any prompting or directory setting, which is surely better.

In /opt/local/lib/cmake I also see other projects that have put their cmake config files in there, for the same reason no doubt.

And that is indeed the location where the cmake documentation says they should be put:

​https://cmake.org/cmake/help/book/mastering-cmake/cmake/Help/guide/using-dependencies/index.html#id3

So -- looks like no reason to put them anywhere else, as AFAICT they won't be found without manual intervention.

I don't think there is anything in the cmake port or the cmake PortGroup that needs repair here. It would seem the fix is to have lapack(e) put the cmake config files in /opt/local/lib/cmake.

Also -- the cmake port has been updated now and is as current and as patched as cmake-devel, so that issue is gone now too.

[kencu (Ken)]

It looks like prior to the commit moving the cmake files in a post-destroot block, the lapack cmake config files were being installed like this:

x ./opt/local/lib/lapack/cmake/lapacke-3.12.0/lapacke-config-version.cmake
x ./opt/local/lib/lapack/cmake/lapacke-3.12.0/lapacke-config.cmake
x ./opt/local/lib/lapack/cmake/lapacke-3.12.0/lapacke-targets.cmake
x ./opt/local/lib/lapack/cmake/lapacke-3.12.0/lapacke-targets-macports.cmake
x ./opt/local/lib/lapack/cmake/lapack-3.12.0/lapack-config.cmake
x ./opt/local/lib/lapack/cmake/lapack-3.12.0/lapack-config-version.cmake
x ./opt/local/lib/lapack/cmake/lapack-3.12.0/lapack-targets.cmake
x ./opt/local/lib/lapack/cmake/lapack-3.12.0/lapack-targets-macports.cmake
x ./opt/local/lib/lapack/cmake/cblas-3.12.0/cblas-config.cmake
x ./opt/local/lib/lapack/cmake/cblas-3.12.0/cblas-targets.cmake
x ./opt/local/lib/lapack/cmake/cblas-3.12.0/cblas-config-version.cmake
x ./opt/local/lib/lapack/cmake/cblas-3.12.0/cblas-targets-macports.cmake

so the problem is they are not installed in:

/opt/local/lib/cmake

they are being installed in:

/opt/local/lib/lapack/cmake

and in there, they can't be found by cmake.

The reason they are being installed in that subdirectory is that the Portfile forces them to be installed there by setting this:

 -DCMAKE_INSTALL_LIBDIR=${prefix}/lib/${name}

and so the cmake config files wind up tucked away in a special place with the default build where cmake can't find them. I presume the Portfile does that so the libraries are sequestered and other ports can also supply similar libraries without conflicting.

I can't see an elegant solution immediately, other than manually moving the files in a corrected post-destroot block...

[kencu (Ken)]

a PR that implements the manual movement of the files to the needed location is here:

​https://github.com/macports/macports-ports/pull/23271

[kencu (Ken)]

of course, the OP says above that manually moving the config files breaks them.

and that openblas may conflict.

All this will need more testing I think.

[kencu (Ken)]

I closed the PR as this is still not right.

We can't set up the build to put the cmake files in the right place, and apparently we can't move them after either.

The way the build system locates the files is wrong without ​https://github.com/macports/macports-ports/pull/17207/commits/2024f790c61c7da0d3cf8285035fc9d95dd1d1f9 and wrong with it too.

OK -- my time here is done. Moving on and will leave this headache for someone who actually uses these software packages to properly sort out and generate a PR if they ever do.

For now, users of this will have to continue to pass in the proper search directory path, as they have been doing the past few years anyway.

[lepus2589 (Tim Kaune)]

Yes, the current LAPACK CMake script does not allow you, to customize the binary install location without affecting the CMake package scripts install location. I raised an issue with upstream LAPACK: ​https://github.com/Reference-LAPACK/lapack/issues/1008

I would still strongly suggest not to move the CMake package scripts to a different location after installing them! This breaks their internal consistency. They are in a non-default discoverable location before and after the move, anyway. So, moving them fixes nothing and just breaks them.

[lepus2589 (Tim Kaune)]

The PR in Reference LAPACK has been merged to master. The next release of LAPACK will provide the LAPACK_BINARY_PATH_SUFFIX option, which allows a consumer, to install all binaries to a subfolder of the lib directory without impacting the CMake scripts.

With -DLAPACK_BINARY_PATH_SUFFIX=lapack, the LAPACK port could then achieve the exact thing that was intended all along.

From past releases, I would assume the next release to be around November 2024.