Failed to build gdal (Catalina 10.15.5 Macports 2.6.2)

[avillasenorh (Antonio Villaseñor)]

I am installing gdal in a new MacBook Pro 16" with latest version of Catalina (10.15.5) and Macports (2.6.2)

I run:

sudo port install gdal +hdf5 +netcdf +openjpeg

And I get the following error:

.
.
.
--->  Building gdal
Error: Failed to build gdal: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.
Error: Processing of port gdal failed

The last lines of the log file are:

:info:build make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/frmts'
:info:build make: *** [frmts-target] Error 2
:info:build make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal'
:info:build Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal" && /usr/bin/make -j16 -w
:info:build Exit code: 2
:error:build Failed to build gdal: command execution failed
:debug:build Error code: CHILDSTATUS 44466 2
:debug:build Backtrace: command execution failed
:debug:build     while executing
:debug:build "system {*}$notty {*}$nice $fullcmdstring"
:debug:build     invoked from within
:debug:build "command_exec build"
:debug:build     (procedure "portbuild::build_main" line 8)
:debug:build     invoked from within
:debug:build "$procedure $targetname"
:error:build See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/main.log for details.

[ryandesign (Ryan Carsten Schmidt)]

Please attach the entire main.log. We cannot see what's wrong based just on the few lines you posted.

[ryandesign (Ryan Carsten Schmidt)]

​gdal @3.1.0 built fine on our Catalina buildbot worker so if you're seeing a build error you'll need to provide us with your main.log file so we can see what went wrong.

[avillasenorh (Antonio Villaseñor)]

main.log file

[kencu (Ken)]

try building it without the +openjpeg variant.

[avillasenorh (Antonio Villaseñor)]

main.log file without the +openjpeg variant

[avillasenorh (Antonio Villaseñor)]

I left out the +openjpeg variant and run:

sudo port install gdal +hdf5 +netcdf 

I still got an error. I attached the new main.log file

[kencu (Ken)]

Ok. I had hoped that turning off the openjpeg variant would stop this, but it didn't

:info:build /bin/sh /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/libtool --mode=compile --silent --tag=CXX /usr/bin/clang++ -std=c++14 -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/port -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/gcore -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/alg -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/ogr -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/ogr/ogrsf_frmts -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/gnm -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/apps -DHAVE_AVX_AT_COMPILE_TIME -DHAVE_SSSE3_AT_COMPILE_TIME -DHAVE_SSE_AT_COMPILE_TIME -pipe -DGDAL_COMPILATION -stdlib=libc++ -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64  -Wall -Wextra -Winit-self -Wunused-parameter -Wformat -Werror=format-security -Wno-format-nonliteral -Wshorten-64-to-32 -Wshadow -Werror=vla -Wdate-time -Wnull-dereference -Wextra-semi -Wcomma -Wfloat-conversion -Wdocumentation -Wno-documentation-deprecated-sync -Wunused-private-field -Wmissing-declarations -Wnon-virtual-dtor -Woverloaded-virtual -fno-operator-names -Wzero-as-null-pointer-constant -Wimplicit-fallthrough  -I..  -I/opt/local/include -I/opt/local/include -I/opt/local/lib/proj6/include  -DGNM_ENABLED -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/port  -DGDAL_COMPILATION -I/opt/local/include -DHAVE_SPATIALITE -DSPATIALITE_412_OR_LATER -DHAVE_PCRE -DSQLITE_HAS_COLUMN_METADATA -c -o ../o/ogrsqliteutility.lo ogrsqliteutility.cpp
:info:build jpeglsdataset.cpp:126:35: error: no member named 'ImageTypeNotSupported' in 'charls::jpegls_errc'
:info:build         case CharlsApiResultType::ImageTypeNotSupported: return "ImageTypeNotSupported";
:info:build              ~~~~~~~~~~~~~~~~~~~~~^
:info:build jpeglsdataset.cpp:127:35: error: no member named 'UnsupportedBitDepthForTransform' in 'charls::jpegls_errc'; did you mean 'UnsupportedColorTransform'?
:info:build         case CharlsApiResultType::UnsupportedBitDepthForTransform: return "UnsupportedBitDepthForTransform";
:info:build              ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
:info:build                                   UnsupportedColorTransform
:info:build /opt/local/include/CharLS/public_types.h:360:5: note: 'UnsupportedColorTransform' declared here
:info:build     UnsupportedColorTransform = color_transform_not_supported,
:info:build     ^
:info:build jpeglsdataset.cpp:125:14: error: duplicate case value: 'ParameterValueNotSupported' and 'InvalidCompressedData' both equal '5'
:info:build         case CharlsApiResultType::InvalidCompressedData: return "InvalidCompressedData";
:info:build              ^
:info:build jpeglsdataset.cpp:121:14: note: previous case defined here
:info:build         case CharlsApiResultType::ParameterValueNotSupported: return "ParameterValueNotSupported";
:info:build              ^
:info:build jpeglsdataset.cpp:129:14: error: duplicate case value 'UnsupportedColorTransform'
:info:build         case CharlsApiResultType::UnsupportedColorTransform: return "UnsupportedColorTransform";
:info:build              ^
:info:build jpeglsdataset.cpp:127:14: note: previous case defined here
:info:build         case CharlsApiResultType::UnsupportedBitDepthForTransform: return "UnsupportedBitDepthForTransform";
:info:build              ^
:info:build /bin/sh /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/libtool --mode=compile --silent --tag=CXX /usr/bin/clang++ -std=c++14 -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/port -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/gcore -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/alg -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/ogr -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/ogr/ogrsf_frmts -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/gnm -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/apps -DHAVE_AVX_AT_COMPILE_TIME -DHAVE_SSSE3_AT_COMPILE_TIME -DHAVE_SSE_AT_COMPILE_TIME -pipe -DGDAL_COMPILATION -stdlib=libc++ -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64  -Wall -Wextra -Winit-self -Wunused-parameter -Wformat -Werror=format-security -Wno-format-nonliteral -Wshorten-64-to-32 -Wshadow -Werror=vla -Wdate-time -Wnull-dereference -Wextra-semi -Wcomma -Wfloat-conversion -Wdocumentation -Wno-documentation-deprecated-sync -Wunused-private-field -Wmissing-declarations -Wnon-virtual-dtor -Woverloaded-virtual -fno-operator-names -Wzero-as-null-pointer-constant -Wimplicit-fallthrough  -DGNM_ENABLED -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/port  -DGDAL_COMPILATION -I/opt/local/include -DHAVE_HDF5 -DNETCDF_HAS_NC4   -c -o ../o/netcdf_sentinel3_sral_mwr.lo netcdf_sentinel3_sral_mwr.cpp
:info:build 4 errors generated.
:info:build make[2]: *** [../o/jpeglsdataset.lo] Error 1
:info:build make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/frmts/jpegls'
:info:build make[1]: *** [jpegls-install-obj] Error 2
:info:build make[1]: *** Waiting for unfinished jobs....

gdal has a cornucopia of variants, and a little quick factorial math will show you the number of potential combinations of them is astronomical. To be frank, the only build of any port that is ever really tested is the default build, and then usually only on the port maintainer's system. So lots of room for trouble.

Would you consider going with the default build options, which apparently does build according to the buildbot, and then once you are at least in business, look at these variants carefully and see if you really need one or the other of them.

If you do really need one or the other, and that variant won't build, then that is a better place for someone to start to help you.

On the broader front, other package managers on macOS have headed completely away from all variants for this reason. There is one, single, default build available, and no variants are allowed. Less flexible, but much less headache.

[kencu (Ken)]

on another note -- looking at the log, it is surprisingly not very easy for me to confirm the actual variant selection you typed into the command line.

I'm sure it is in that 6MB file somewhere -- maybe -- but it's not obvious (like right at the top of the file, for example, where you would expect it to be) and searching for "variant" in the log shows dozens of hits for everything but what variants you typed into the command line.

[kencu (Ken)]

I think I see what might have gone on.

charls was updated several weeks ago. gdal might need some tweaking to build against the new charls headers.

[avillasenorh (Antonio Villaseñor)]

Many thanks for your help.

I appreciate very much your quick responses and dedication.

I use that combination of gdal because it is in the instructions of the software that I am really interested in (gmt6):

​https://github.com/GenericMappingTools/gmt/blob/master/INSTALL.md#install-via-macports

I have installed recently a slightly older version in another Mac with Catalina without any problems. This is the version in the other Mac:

$ port list gdal
gdal                           @3.0.3          gis/gdal

The current version I think it is 3.1.0

Thanks again!!!

[reneeotten (Renee Otten)]

Replying to kencu:

I think I see what might have gone on.

charls was updated several weeks ago. gdal might need some tweaking to build against the new charls headers.

perhaps the changes from ​this upstream PR are needed? Let's hope that the maintainer can take a look at this.

[Dave-Allured (Dave Allured)]

Replying to reneeotten:

Replying to kencu:

I think I see what might have gone on.

charls was updated several weeks ago. gdal might need some tweaking to build against the new charls headers.

perhaps the changes from ​this upstream PR are needed? Let's hope that the maintainer can take a look at this.

That upstream PR predated the gdal 3.1.0 release by a few days, so it should have been included. The 3.1.0 release notes say something to this effect, "add support for CharLS 2.1".

[Dave-Allured (Dave Allured)]

There may be insights in #60696. Quoting:

What appears to have happened is that charls changed the capitalization of its include directory and library between 2.0.0 and 2.1.0. In 2.0.0, it's CharLS; in 2.1.0, it's charls.

[reneeotten (Renee Otten)]

well, how about this ​upstream PR then ;) Feel free to give that a try and if it works please submit a PR.

Again, I am not using this software - just attempted to point to a possible solution. But yes, on closer inspection you are right that the first PR I listed was indeed already part of the 3.1.0-release.

[Dave-Allured (Dave Allured)]

Replying to reneeotten:

well, how about this ​upstream PR then ;)

I missed that, sorry. Good catch. Indeed, that will gum up the works on case insensitive file systems.

@avillasenorh, by any chance, is /opt/local in a case insensitive file system on your new MacBook Pro?

Feel free to give that a try and if it works please submit a PR.

Again, I am not using this software - just attempted to point to a possible solution. But yes, on closer inspection you are right that the first PR I listed was indeed already part of the 3.1.0-release.

GDAL 3.1.1 will include this new fix. When released, I will update on Macports, if someone else does not get to it first.

​https://lists.osgeo.org/pipermail/gdal-dev/2020-June/052314.html

[avillasenorh (Antonio Villaseñor)]

I use an APFS file system in my MacBook Pro. I assume that /opt/local is case insensitive, since that is the default installation, and I haven't done anything to change it to case sensitive.

Thanks again!

Antonio

[ryandesign (Ryan Carsten Schmidt)]

Has duplicates #60693 and #60694.

Vince committed [bcc04bf03c5f6579a5a1a85ff795eb9785b2642c/macports-ports] to gdal. Did that fix it?

[ryandesign (Ryan Carsten Schmidt)]

Replying to avillasenorh:

$ port list gdal
gdal                           @3.0.3          gis/gdal

"port list" doesn't tell you what you think it tells you.

[Veence (Vincent)]

I have committed a patch that should solve this issue. As a matter of fact, I just compiled/installed GDAL on my brand new MacBook Air with the default FS (Case insensitive), and it worked like a charm, so there’s no reason it should fail.

[avillasenorh (Antonio Villaseñor)]

I run first: sudo port selfupdate sudo port upgrade outdated

And then the command that gave me problems: sudo port install gdal +hdf5 +netcdf +openjpeg

It now compiles without problems and installs gdal!!!

Thanks everybody! Antonio

[Veence (Vincent)]

Perfetto ! :) I close this issue, then.