qgis3 @3.32.3 build failure on macOS 14

[edmcnierney (Ed McNierney)]

I think this is the same problem reported in #68209, which was closed due to a potential conflict with Homebrew. The compilation error isn't the original error reported there, however, and I'm having the same problem. The report for that ticket shows the same error I'm getting (MacOS 14.0, Apple Silicon):

$ sudo port install qgis3
--->  Computing dependencies for qgis3
--->  Fetching archive for qgis3
--->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from https://packages.macports.org/qgis3
--->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/qgis3
--->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from http://mirror.fcix.net/macports/packages/qgis3
--->  Building qgis3

There is no /qgis3 directory at all at any of those archive sites, so the qgis3 archive cannot be downloaded (that's why all three are tried). The pertinent part of my main.log is

:info:archivefetch --->  qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 doesn't seem to exist in /opt/local/var/macports/incoming/verified
:msg:archivefetch --->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from https://packages.macports.org/qgis3
:debug:archivefetch Fetching archive failed: The requested URL returned error: 404
:msg:archivefetch --->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/qgis3
:debug:archivefetch Fetching archive failed: The requested URL returned error: 404
:msg:archivefetch --->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from http://mirror.fcix.net/macports/packages/qgis3
:debug:archivefetch Fetching archive failed: The requested URL returned error: 404

[jmroot (Joshua Root)]

That's not the pertinent part of the log; please attach the whole thing.

[edmcnierney (Ed McNierney)]

Thanks very much for the quick response. There are, of course, later compilation errors in the log. In this ticket I'm reporting on what seem to be missing archives (e.g. ​https://packages.macports.org/qgis3 returns a 404 error).

[nilason (Nicklas Larsson)]

Thanks for the log file, that is usually what make it possible to find out the problem. I couldn't find any traces of Homebrew in the log, this is something else.

There is an issue with a port libunwind and Xcode 15, that perhaps is behind this problem. You need to uninstall libunwind, rebuild PDAL and then QGIS.

Try the following, (I haven't tried this step-by-step so there may be deviations):

sudo port selfupdate
sudo port uninstall libunwind

Say yes to rebuild ports that were dependents to libunwind. Make sure PDAL is rebuilt, if not rebuild it. Install QGIS.

[jmroot (Joshua Root)]

It looks to me like the problem is this:

:info:build /bin/sh: ccache: command not found

The log starts with the configure phase already completed, so maybe it was configured with ccache installed and then ccache was uninstalled. Try just running sudo port clean qgis3 and then try the install again. Attach the new log if it fails.

[nilason (Nicklas Larsson)]

Replying to jmroot:

It looks to me like the problem is this:

:info:build /bin/sh: ccache: command not found

The log starts with the configure phase already completed, so maybe it was configured with ccache installed and then ccache was uninstalled. Try just running sudo port clean qgis3 and then try the install again. Attach the new log if it fails.

You are probably right. But libunwind must be uninstalled anyway.

[jmroot (Joshua Root)]

Yes, libunwind is a problem unfortunately.

[edmcnierney (Ed McNierney)]

I think I followed a reasonable combination of those steps. I started with

sudo port selfupdate
sudo port uninstall libunwind

That did not prompt me to rebuild PDAL, so I forced a rebuild with sudo port upgrade --force PDAL. That succeeded, but sudo port install qgis3 failed again in apparently the same way. I then tried

sudo port clean qgis3
sudo port install qgis3

That produced essentially the same console output

$ sudo port install qgis3
--->  Computing dependencies for qgis3
--->  Fetching archive for qgis3
--->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from https://packages.macports.org/qgis3
--->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/qgis3
--->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from http://mirror.fcix.net/macports/packages/qgis3
--->  Fetching distfiles for qgis3
--->  Verifying checksums for qgis3
--->  Extracting qgis3
--->  Applying patches to qgis3
--->  Configuring qgis3
--->  Building qgis3
Error: Failed to build qgis3: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_qgis3/qgis3/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug.
Error: Processing of port qgis3 failed

I have attached the new main.log file from that attempt.

I'm puzzled, though. Isn't the fact that none of the package repositories - ​https://packages.macports.org/qgis3, ​https://ywg.ca.packages.macports.org/mirror/macports/packages/qgis3, and ​http://mirror.fcix.net/macports/packages/qgis3 - exist a problem itself, or is that a separate issue (or non-issue)?

[nilason (Nicklas Larsson)]

The log file looks pretty much the same. And I see libunwind there again...

What result does port installed libunwind give?

[edmcnierney (Ed McNierney)]

$ port installed libunwind
The following ports are currently installed:
  libunwind @5.0.1_0+universal (active)

[nilason (Nicklas Larsson)]

How did that come back again?

Try:

sudo port uninstall libunwind PDAL
# do **not** re-build
sudo port install PDAL
sudo port install qgis

[nilason (Nicklas Larsson)]

You have to uninstall libunwind-headers too.

[edmcnierney (Ed McNierney)]

No apparent difference - new main.log.3.tgz is attached. Install is for qgis3 rather than qgis.

$ sudo port uninstall libunwind libunwind-headers PDAL
--->  Deactivating PDAL @2.5.6_1+postgresql15
--->  Cleaning PDAL
--->  Uninstalling PDAL @2.5.6_1+postgresql15
--->  Cleaning PDAL
--->  Deactivating libunwind @5.0.1_0+universal
--->  Cleaning libunwind
--->  Uninstalling libunwind @5.0.1_0+universal
--->  Cleaning libunwind
--->  Deactivating libunwind-headers @5.0.1_0
--->  Cleaning libunwind-headers
--->  Uninstalling libunwind-headers @5.0.1_0
--->  Cleaning libunwind-headers
$ sudo port install PDAL
--->  Computing dependencies for PDAL
The following dependencies will be installed:
 libunwind
 libunwind-headers
Continue? [Y/n]: Y
--->  Fetching archive for libunwind-headers
--->  Attempting to fetch libunwind-headers-5.0.1_0.darwin_23.noarch.tbz2 from https://packages.macports.org/libunwind-headers
--->  Attempting to fetch libunwind-headers-5.0.1_0.darwin_23.noarch.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/libunwind-headers
--->  Attempting to fetch libunwind-headers-5.0.1_0.darwin_23.noarch.tbz2 from http://mirror.fcix.net/macports/packages/libunwind-headers
--->  Fetching distfiles for libunwind-headers
--->  Verifying checksums for libunwind-headers
--->  Extracting libunwind-headers
--->  Applying patches to libunwind-headers
--->  Configuring libunwind-headers
--->  Building libunwind-headers
--->  Staging libunwind-headers into destroot
--->  Installing libunwind-headers @5.0.1_0
--->  Activating libunwind-headers @5.0.1_0
--->  Cleaning libunwind-headers
--->  Fetching archive for libunwind
--->  Attempting to fetch libunwind-5.0.1_0+universal.darwin_23.arm64-x86_64.tbz2 from https://packages.macports.org/libunwind
--->  Attempting to fetch libunwind-5.0.1_0+universal.darwin_23.arm64-x86_64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/libunwind
--->  Attempting to fetch libunwind-5.0.1_0+universal.darwin_23.arm64-x86_64.tbz2 from http://mirror.fcix.net/macports/packages/libunwind
--->  Fetching distfiles for libunwind
--->  Verifying checksums for libunwind
--->  Extracting libunwind
--->  Applying patches to libunwind
--->  Configuring libunwind
--->  Building libunwind
--->  Staging libunwind into destroot
--->  Installing libunwind @5.0.1_0+universal
--->  Activating libunwind @5.0.1_0+universal
--->  Cleaning libunwind
--->  Fetching archive for PDAL
--->  Attempting to fetch PDAL-2.5.6_1+postgresql15.darwin_23.arm64.tbz2 from https://packages.macports.org/PDAL
--->  Attempting to fetch PDAL-2.5.6_1+postgresql15.darwin_23.arm64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/PDAL
--->  Attempting to fetch PDAL-2.5.6_1+postgresql15.darwin_23.arm64.tbz2 from http://mirror.fcix.net/macports/packages/PDAL
--->  Fetching distfiles for PDAL
--->  Verifying checksums for PDAL
--->  Extracting PDAL
--->  Applying patches to PDAL
--->  Configuring PDAL
--->  Building PDAL
--->  Staging PDAL into destroot
--->  Installing PDAL @2.5.6_1+postgresql15
--->  Activating PDAL @2.5.6_1+postgresql15
--->  Cleaning PDAL
--->  Scanning binaries for linking errors
--->  No broken files found.
--->  No broken ports found.
ndau$ sudo port install qgis3
--->  Computing dependencies for qgis3
--->  Fetching archive for qgis3
--->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from https://packages.macports.org/qgis3
--->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/qgis3
--->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from http://mirror.fcix.net/macports/packages/qgis3
--->  Building qgis3
Error: Failed to build qgis3: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_qgis3/qgis3/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug.
Error: Processing of port qgis3 failed

[nilason (Nicklas Larsson)]

Of course, qgis3 it is. The workaround fix for pdal was committed some 8 hrs ago, should be available.

sudo port selfupdate
sudo port uninstall libunwind libunwind-headers PDAL
sudo port clean PDAL qgis3
sudo port install PDAL

[ryandesign (Ryan Carsten Schmidt)]

Replying to edmcnierney:

I'm puzzled, though. Isn't the fact that none of the package repositories - ​https://packages.macports.org/qgis3, ​https://ywg.ca.packages.macports.org/mirror/macports/packages/qgis3, and ​http://mirror.fcix.net/macports/packages/qgis3 - exist a problem itself, or is that a separate issue (or non-issue)?

Absence of pre-compiled archives is not a problem. (We haven't started making them for Darwin 23 yet.) When an archive is not available, MacPorts builds from source on your system, which is expected to work; if it does not, it is a bug that should be reported along with the main.log file.

[ryandesign (Ryan Carsten Schmidt)]

Replying to nilason:

The workaround fix for pdal was committed some 8 hrs ago, should be available.

The build system, which is also responsible for updating the master rsync server, was offline for most of the day due to a power outage, so the PDAL update from 10 hours ago may not have been available on the rsync server at the time. Updates to the rsync server should have resumed by now.

[edmcnierney (Ed McNierney)]

I'm always careful to report if I don't follow the instructions exactly ;-)

I don't see any difference.

$ sudo port selfupdate
--->  Updating MacPorts base sources using rsync
MacPorts base version 2.8.1 installed,
MacPorts base version 2.8.1 downloaded.
--->  Updating the ports tree
--->  MacPorts base is already the latest version

The ports tree has been updated. To upgrade your installed ports, you should run
  port upgrade outdated
ndau$ sudo port uninstall libunwind libunwind-headers PDAL
--->  Deactivating PDAL @2.5.6_1+postgresql15
--->  Cleaning PDAL
--->  Uninstalling PDAL @2.5.6_1+postgresql15
--->  Cleaning PDAL
--->  Deactivating libunwind @5.0.1_0+universal
--->  Cleaning libunwind
--->  Uninstalling libunwind @5.0.1_0+universal
--->  Cleaning libunwind
--->  Deactivating libunwind-headers @5.0.1_0
--->  Cleaning libunwind-headers
--->  Uninstalling libunwind-headers @5.0.1_0
--->  Cleaning libunwind-headers
ndau$ sudo port clean PDAL qgis3
Password:
--->  Cleaning PDAL
--->  Cleaning qgis3
ndau$ sudo port install PDAL
--->  Computing dependencies for PDAL
The following dependencies will be installed:
 libunwind
 libunwind-headers
Continue? [Y/n]: Y
--->  Fetching archive for libunwind-headers
--->  Attempting to fetch libunwind-headers-5.0.1_0.darwin_23.noarch.tbz2 from https://packages.macports.org/libunwind-headers
--->  Attempting to fetch libunwind-headers-5.0.1_0.darwin_23.noarch.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/libunwind-headers
--->  Attempting to fetch libunwind-headers-5.0.1_0.darwin_23.noarch.tbz2 from http://mirror.fcix.net/macports/packages/libunwind-headers
--->  Fetching distfiles for libunwind-headers
--->  Verifying checksums for libunwind-headers
--->  Extracting libunwind-headers
--->  Applying patches to libunwind-headers
--->  Configuring libunwind-headers
--->  Building libunwind-headers
--->  Staging libunwind-headers into destroot
--->  Installing libunwind-headers @5.0.1_0
--->  Activating libunwind-headers @5.0.1_0
--->  Cleaning libunwind-headers
--->  Fetching archive for libunwind
--->  Attempting to fetch libunwind-5.0.1_0+universal.darwin_23.arm64-x86_64.tbz2 from https://packages.macports.org/libunwind
--->  Attempting to fetch libunwind-5.0.1_0+universal.darwin_23.arm64-x86_64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/libunwind
--->  Attempting to fetch libunwind-5.0.1_0+universal.darwin_23.arm64-x86_64.tbz2 from http://mirror.fcix.net/macports/packages/libunwind
--->  Fetching distfiles for libunwind
--->  Verifying checksums for libunwind
--->  Extracting libunwind
--->  Applying patches to libunwind
--->  Configuring libunwind
--->  Building libunwind
--->  Staging libunwind into destroot
--->  Installing libunwind @5.0.1_0+universal
--->  Activating libunwind @5.0.1_0+universal
--->  Cleaning libunwind
--->  Fetching archive for PDAL
--->  Attempting to fetch PDAL-2.5.6_1+postgresql15.darwin_23.arm64.tbz2 from https://packages.macports.org/PDAL
--->  Attempting to fetch PDAL-2.5.6_1+postgresql15.darwin_23.arm64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/PDAL
--->  Attempting to fetch PDAL-2.5.6_1+postgresql15.darwin_23.arm64.tbz2 from http://mirror.fcix.net/macports/packages/PDAL
--->  Fetching distfiles for PDAL
--->  Verifying checksums for PDAL
--->  Extracting PDAL
--->  Applying patches to PDAL
--->  Configuring PDAL
--->  Building PDAL
--->  Staging PDAL into destroot
--->  Installing PDAL @2.5.6_1+postgresql15
--->  Activating PDAL @2.5.6_1+postgresql15
--->  Cleaning PDAL
--->  Scanning binaries for linking errors
--->  No broken files found.
--->  No broken ports found.
ndau$ sudo port install qgis3
--->  Computing dependencies for qgis3
--->  Fetching archive for qgis3
--->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from https://packages.macports.org/qgis3
--->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/qgis3
--->  Attempting to fetch qgis3-3.32.3_1+plugin_support311+postgresql15+proj9+python311.darwin_23.arm64.tbz2 from http://mirror.fcix.net/macports/packages/qgis3
--->  Fetching distfiles for qgis3
--->  Verifying checksums for qgis3
--->  Extracting qgis3
--->  Applying patches to qgis3
--->  Configuring qgis3
--->  Building qgis3
Error: Failed to build qgis3: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_qgis3/qgis3/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug.
Error: Processing of port qgis3 failed

New log is attached.

[edmcnierney (Ed McNierney)]

Replying to ryandesign:

Replying to edmcnierney:

I'm puzzled, though. Isn't the fact that none of the package repositories - ​https://packages.macports.org/qgis3, ​https://ywg.ca.packages.macports.org/mirror/macports/packages/qgis3, and ​http://mirror.fcix.net/macports/packages/qgis3 - exist a problem itself, or is that a separate issue (or non-issue)?

Absence of pre-compiled archives is not a problem. (We haven't started making them for Darwin 23 yet.) When an archive is not available, MacPorts builds from source on your system, which is expected to work; if it does not, it is a bug that should be reported along with the main.log file.

Thanks very much for that explanation. I'm surprised that even without the Darwin 23 builds the top-level directories don't exist. But now I can stop wondering whether that's the cause of this problem.

[nilason (Nicklas Larsson)]

Ed, did you take another shot at this, now with the rsync server up again?

[edmcnierney (Ed McNierney)]

Thanks for checking in again. My day job got in the way a bit, but I have better info. I experimented with building/installing several of the intermediate dependencies, some of which worked, but together did not succeed. I got to the point where even if I did succeed I could no longer accurately explain why.

So I took out a big hammer and completely uninstalled MacPorts. I then built and installed MacPorts 2.8.1. As there is not yet an installer package explicitly labeled for Ventura I thought this was safer but it probably wasn't necessary. The only port commands I then issued were

sudo port selfupdate
sudo port install PDAL
sudo port install qgis3

There were no problems with PDAL. Several hours later (QGIS has quite a dependency tree!) the QGIS3 build failed in a different place. The package py311-scipy would not build, and the problems appear to be related to the lapack and/or openblas libraries. Last time around I tried building lapack and that failed, but at this point I'm in a very well-defined state and I'd rather not mess around without further guidance.

I've attached the py311-scipy log. I'll be able to spend more time on this tonight (US EDT) and over the weekend.

[nilason (Nicklas Larsson)]

I suspect you meant to say Sonoma, there is installer for that https://www.macports.org/install.php.

[edmcnierney (Ed McNierney)]

I did mean MacOS 14 Sonoma (I'm getting a little tired of this California stuff). And I was looking instead at ​https://github.com/macports/macports-base/releases and neglected to click the "Show all 40 assets" link, so I didn't see it. The build and install went smoothly so I'm not going to worry about it. It's pretty clear that the installation isn't related to my problem. Thanks for the correction.

[bal-agates]

Yesterday my fresh build of qgis3 @3.32.3_1 also failed because port py311-scipy failed to build. See #68329. This is on a MacBook Pro 2021 (M1) with macOS 14.0 (Sonoma) and Xcode 15.0.

[nilason (Nicklas Larsson)]

Replying to bal-agates:

Yesterday my fresh build of qgis3 @3.32.3_1 also failed because port py311-scipy failed to build. See #68329. This is on a MacBook Pro 2021 (M1) with macOS 14.0 (Sonoma) and Xcode 15.0.

You can try with -plugin_support311, to at least get QGIS going. That excludes a whole bunch of dependancy ports (mainly added for simplifying installing plugins, thus not needed for core QGIS).

[MaintenanceCosts]

Building with -plugin_support311 produced the first successful qgis3 build for me since the Sonoma upgrade. Core QGIS functionality appears to work at least for my use case.

I remain unable to build py311-scipy.

[bal-agates]

I was able to build qgis3 on macOS 14.2 arm64 today with variant options "+draco +grass +plugin_support311 +postgresql15 +proj9 +python311". The py311-scipy build problems seem to have been solved by an OpenBLAS change.

Likely this issue can be closed.

[edmcnierney (Ed McNierney)]

I'm unable to reproduce that successful installation (macOS 14.2, M2 Max). I uninstalled then reinstalled OpenBLAS then PDAL:

sudo port install OpenBLAS +native
sudo port install PDAL
sudo port install qgis3 +draco +grass +plugin_support311 +postgresql15 +proj9 +python311

I've attached the new build log.

[nilason (Nicklas Larsson)]

Replying to edmcnierney:

I'm unable to reproduce that successful installation (macOS 14.2, M2 Max). I uninstalled then reinstalled OpenBLAS then PDAL:

The log shows the same ccache issue noted by comment:4 .

(I have QGIS running on an ARM machine with macOS 14.2 so it should work).

[ryandesign (Ryan Carsten Schmidt)]

Replying to edmcnierney:

I'm surprised that even without the Darwin 23 builds the top-level directories don't exist.

That means we believe we're not legally allowed to distribute the archives, because:

% macports-infrastructure/jobs/port_binary_distributable.tcl -v qgis3
"qgis3" is not distributable because its license "gpl" conflicts with license "OpenSSL" of dependency "openssl11"

[edmcnierney (Ed McNierney)]

I'm sorry I've had so little time to follow up on this bug, but it appears to be fixed. I am now running macOS Sonoma (14.5) and I did a simple sudo port install qgis3 without trouble. I installed QGIS 3.36.3-Maidenhead and it's working fine.

I don't know what changed to fix it or why I had trouble in the first place, but this ticket can be closed. It's at least no longer reproducible. Thanks very much for all the attention to it.