gdal @3.0.1_3: Build fails, argument list too long

[Dave-Allured (Dave Allured)]

I am building gdal 3.0.1 from source using Macports. I am building into a custom user directory, and not using sudo. "Port install" proceeds to compile many gdal components without trouble. Then it fails with "Argument list too long" on a very long libtool command. See main.log which I am attaching.

Here is the point of failure from main.log. The offending command line is 252733 characters long. There are 1267 arguments to the libtool command, most of them containing long full paths (only 3 of those shown here). I added line breaks for clarity, and truncated. See main.log for the original, unmodified long command.

:info:build /Library/Developer/CommandLineTools/usr/bin/make libgdal.la
:info:build make[1]: Entering directory `/Volumes/User/dallured/port/262/var/macports/build/_Volumes_User_dallured_port_262_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.0.1/gdal'
:info:build /bin/sh /Volumes/User/dallured/port/262/var/macports/build/_Volumes_User_dallured_port_262_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.0.1/gdal/libtool
        --mode=link
        --silent
        /usr/bin/clang++
        -std=c++14
        -L/Users/dallured/port/262/lib
        -Wl,-headerpad_max_install_names
        -stdlib=libc++
        -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk
        -arch x86_64
        -lcrypto
        -lqhull
        -lqhull
        -L/Users/dallured/port/262/lib
        -lgeos_c
        -lwebp
<snip>
        /Volumes/User/dallured/port/262/var/macports/build/_Volumes_User_dallured_port_262_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.0.1/gdal/alg/contour.lo
        /Volumes/User/dallured/port/262/var/macports/build/_Volumes_User_dallured_port_262_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.0.1/gdal/alg/delaunay.lo
        /Volumes/User/dallured/port/262/var/macports/build/_Volumes_User_dallured_port_262_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.0.1/gdal/alg/gdal_crs.lo
<snip>
:info:build 	    -rpath /Users/dallured/port/262/lib \
:info:build 	    -no-undefined \
:info:build 	    -version-info 26:1:0
:info:build make[1]: /bin/sh: Argument list too long
:info:build make[1]: *** [libgdal.la] Error 1

I think this is a corner case, convergence of two factors:

1. Major version update from gdal 2.x to 3.x. Version 2.x built fine for me recently.
2. Custom install prefix which is longer than the usual /opt/local.

What is the best approach to fix this? Devise a Macports patch, or request a fix upstream? Is there a Macports approach to reduce duplication of the same 1267 very long file prefixes?

[Dave-Allured (Dave Allured)]

Main.log from build failure, Argument list too long, October 28

[ryandesign (Ryan Carsten Schmidt)]

Replying to Dave-Allured:

I think this is a corner case, convergence of two factors:

1. Major version update from gdal 2.x to 3.x. Version 2.x built fine for me recently.
2. Custom install prefix which is longer than the usual /opt/local.

You're probably right. Version 3 probably has more files than version 2.

What is the best approach to fix this?

Can you shorten your custom MacPorts prefix?

Devise a Macports patch, or request a fix upstream?

We are probably not doing anything special to build this, so if anyone can fix it it's upstream.

Is there a Macports approach to reduce duplication of the same 1267 very long file prefixes?

Nope. It's their build system causing the absolute paths to be used. You could ask them to use relative paths instead of absolute ones, but if they are just using autotools and libtool in the normal way then it may not be under their control.

[ryandesign (Ryan Carsten Schmidt)]

Replying to Dave-Allured:

2. Custom install prefix which is longer than the usual /opt/local.

Not just the length of the prefix, but also the length of the path to the ports tree. You could sync with git, saving the clone in a shorter path like /opt/ports.

[Dave-Allured (Dave Allured)]

Problem solved. There is now a makefile change which will link using much shorter relative paths. I will await the next GDAL release.

​https://github.com/OSGeo/gdal/issues/2075#issuecomment-560321087

[AgilentGCMS]

So what does this mean? The issue has been fixed in gdal, and we wait for that fix to make its way into the macports tree?

[mf2k (Frank Schima)]

Yes. If you want to help move things along, submit a GitHub pull request with the fixed version.

[Dave-Allured (Dave Allured)]

Has duplicate: #60011

[Dave-Allured (Dave Allured)]

GDAL version 3.0.3 was released on 2020 January 8. This version should include the fix for "Argument list too long".

Update request was filed on January 24: #60009

[Veence (Vincent)]

I have committed the new version. Please let me know if that new release fixes this issue

[AgilentGCMS]

Indeed, macports now builds gdal without errors.

[Veence (Vincent)]

Excellent! Then I can close this ticket. Thanks for testing!

[Dave-Allured (Dave Allured)]

The new version (gdal 3.0.3 and portfile) works for me too. Very good to have this problem cleaned up. Thanks, Veence and Even of OSGeo/GDAL.