gildas @201911b: does not build for the right C++ stdlib (undefined symbols)

[ryandesign (Ryan Carsten Schmidt)]

gildas fails to build for a C++ standard library that is not the compiler's default. For example, ever since MacPorts 2.6.0, this happens on Mac OS X 10.6-10.8:

The error is ​a bunch of undefined symbols, like this:

(cd ./built/x86_64-darwin-gfortran && /opt/local/bin/gfortran-mp-9  -dynamiclib   -L/opt/local/lib -Wl,-headerpad_max_install_names -shared -o /opt/local/lib/gildas/x86_64-darwin-gfortran/lib/libatm2009.so -install_name /opt/local/lib/gildas/x86_64-darwin-gfortran/lib/libatm2009.so ATMOpacity.o ATMLength.o ATMInverseLength.o ATMPressure.o ATMTemperature.o ATMAngle.o ATMFrequency.o ATMHumidity.o ATMMassDensity.o ATMNumberDensity.o ATMPercent.o ATMWaterVaporRadiometer.o ATMWVRMeasurement.o ATMType.o ATMProfile.o ATMRefractiveIndex.o ATMRefractiveIndexProfile.o ATMSkyStatus.o ATMSpectralGrid.o ATMError.o  \
	-o /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/integ/x86_64-darwin-gfortran/lib/libatm2009.so -L/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/integ/x86_64-darwin-gfortran/lib -L/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib  -lc++  -lm -ldl -install_name @loader_path/../lib/libatm2009.so)
Undefined symbols for architecture x86_64:
  "std::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare(char const*) const", referenced from:
      atm::Opacity::Opacity(double, std::basic_string<char, std::char_traits<char>, std::allocator<char> >) in ATMOpacity.o
      atm::Opacity::sput(double, std::basic_string<char, std::char_traits<char>, std::allocator<char> >) in ATMOpacity.o
      atm::Opacity::sget(double, std::basic_string<char, std::char_traits<char>, std::allocator<char> >) in ATMOpacity.o
      atm::Opacity::get(std::basic_string<char, std::char_traits<char>, std::allocator<char> >) const in ATMOpacity.o
      atm::Length::sput(double, std::basic_string<char, std::char_traits<char>, std::allocator<char> >) in ATMLength.o
      atm::Length::sget(double, std::basic_string<char, std::char_traits<char>, std::allocator<char> >) in ATMLength.o
      atm::InverseLength::sput(double, std::basic_string<char, std::char_traits<char>, std::allocator<char> >) in ATMInverseLength.o
      ...

This is happening because the port is supplying the C++ standard library flag (-lc++) to the Fortran compiler, but is not supplying that flag (or better yet the -stdlib=libc++ flag) to the C++ compiler, so the parts that were built with the C++ compiler were built with the default C++ standard library (libstdc++) which is not compatible with libc++.

The Portfile goes to some effort to put the right flags into the build system with a bunch of patchfiles and reinplaces, including cflags and ldflags. However I note it is not making any attempt to also do that for the cxxflags, which would contain the required stdlib flag. In addition, the reinplaces that are supposed to replace the placeholder @STDCXXLIB@ with the real value are failing, because the placeholder does not exist in any of the patchfiles. In fact many of the reinplaces are failing. Warnings are printed about this, which should have been heeded and corrected before the changes were committed:

Warning: reinplace s|@CC@|/usr/bin/clang|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/Makefile.def
Warning: reinplace s|@CC@|/usr/bin/clang|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/Makefile.python
Warning: reinplace s|@CC@|/usr/bin/clang|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/gildas-env.sh
Warning: reinplace s|@CCFLAGS@|-Os|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/Makefile.python
Warning: reinplace s|@CCFLAGS@|-Os|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/gildas-env.sh
Warning: reinplace s|@CCFLAGS@|-Os|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/define-system.sh
Warning: reinplace s|@FC@|/opt/local/bin/gfortran-mp-9|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/Makefile.def
Warning: reinplace s|@FC@|/opt/local/bin/gfortran-mp-9|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/Makefile.python
Warning: reinplace s|@FC@|/opt/local/bin/gfortran-mp-9|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/gildas-env.sh
Warning: reinplace s|@FCFLAGS@|-Os|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/Makefile.python
Warning: reinplace s|@FCFLAGS@|-Os|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/gildas-env.sh
Warning: reinplace s|@FCFLAGS@|-Os|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/define-system.sh
Warning: reinplace s|@CXX@|/usr/bin/clang++|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/Makefile.def
Warning: reinplace s|@CXX@|/usr/bin/clang++|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/Makefile.python
Warning: reinplace s|@CXX@|/usr/bin/clang++|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/gildas-env.sh
Warning: reinplace s|@CPP@|/usr/bin/cpp|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/Makefile.python
Warning: reinplace s|@CPP@|/usr/bin/cpp|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/gildas-env.sh
Warning: reinplace s|@CPPFLAGS@|-I/opt/local/include|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/Makefile.python
Warning: reinplace s|@CPPFLAGS@|-I/opt/local/include|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/gildas-env.sh
Warning: reinplace s|@CPPFLAGS@|-I/opt/local/include|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/define-system.sh
Warning: reinplace s|@LDFLAGS@|-L/opt/local/lib -Wl,-headerpad_max_install_names|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/Makefile.python
Warning: reinplace s|@LDFLAGS@|-L/opt/local/lib -Wl,-headerpad_max_install_names|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/gildas-env.sh
Warning: reinplace s|@LDFLAGS@|-L/opt/local/lib -Wl,-headerpad_max_install_names|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/define-system.sh
Warning: reinplace s|@PYTHON@|/opt/local/bin/python2.7|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/define-system.sh
Warning: reinplace s|@STDCXXLIB@|-lc++|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/Makefile.def
Warning: reinplace s|@STDCXXLIB@|-lc++|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/Makefile.python
Warning: reinplace s|@STDCXXLIB@|-lc++|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/gildas-env.sh
Warning: reinplace s|@STDCXXLIB@|-lc++|g didn't change anything in /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_science_gildas/gildas/work/gildas-src-nov19a/admin/define-system.sh

[bardeau]

Dear Ryan,

to have the complete picture, note that there was an attempt to automatize the detection of libc++ location (file admin/Makefile.def):

ifeq ($(GAG_COMPILER_CXXKIND),clang++)
  # Apple LLVM: we have to link against libc++. No way to find its correct
  # location, assume default.
  GAG_COMPILER_CXXLIBS = -lc++
else
  # GNU: we have to link against libstdc++. Find its correct location (it
  # can be at non standard place in case of custom compiler)
  GAG_COMPILER_CXXLIBS = $(shell $(GAG_COMPILER_CXXEXE) $(GAG_32BITS_SWITCH) -print-file-name=libstdc++.$(LIB_CXX_EXT))
endif

This test is useful for those who compile Gildas by themselves (i.e. not using MacPorts) like developers or specific users. Unfortunately, as quoted above, there is no good solution for the Apple compiler. In particular, it accepts the option --print-file-name=libc++.dylib but fails to returns the full path to its library. This is why the above test finally ends by using the default /usr/lib/libc++.dylib.

For the reason explained just above, I would prefer a solution which works with or without MacPorts. As the link is the responsibility of the Fortran compiler, this solution should use portable options like -L, -l, or just a full path like for the GNU case. In second choice, I can try to fix the reinplaces to use properly the @STDCXXLIB@ placeholder.

[kencu (Ken)]

It appears that the fortran compiler is trying to drive the link.

asking the fortran compiler to do the link phase will not usually work in a complex build, I believe, at least on Apple systems.

If the CXX compiler does the link instead, the proper c++ standard library should be used automatically.

[bardeau]

Using the Fortran compiler is a conscious choice has most of the libraries built in Gildas involve directly or indirectly symbols compiled from Fortran sources. Linking libraries (with part of C++ code) with the C++ compiler will imply finding the right path to the libfortran provided by the Fortran compiler, so we have just moved the issue to another point.

[ryandesign (Ryan Carsten Schmidt)]

As I explained initially, I think the linking with fortran is using the right standard library. (Well, it is hardcoded to use libc++, which is not exactly what we want, but is at least correct now on 10.6 and later; contrast with #51726.) But the compiling with clang++ is not: it is not using the MacPorts-specified CXXFLAGS containing the -stdlib=... flag. That, along with the reinplaces that aren't doing anything, is what needs to be fixed.

[bardeau]

Regarding the reinplaces: in our project there are 5 files patched specifically for MacPorts. Among those 5, 4 contain keys to be reinplaced by proper MacPorts translation:

    reinplace -W ${worksrcpath}/admin "s|@CC@|${configure.cc}|g" Makefile.def Makefile.python gildas-env.sh define-system.sh
    reinplace -W ${worksrcpath}/admin "s|@CCFLAGS@|${configure.cflags}|g" Makefile.def Makefile.python gildas-env.sh define-system.sh
    reinplace -W ${worksrcpath}/admin "s|@FC@|${configure.fc}|g" Makefile.def Makefile.python gildas-env.sh define-system.sh
    reinplace -W ${worksrcpath}/admin "s|@FCFLAGS@|${configure.fcflags}|g" Makefile.def Makefile.python gildas-env.sh define-system.sh
    reinplace -W ${worksrcpath}/admin "s|@CXX@|${configure.cxx}|g" Makefile.def Makefile.python gildas-env.sh define-system.sh
    reinplace -W ${worksrcpath}/admin "s|@CPP@|${configure.cpp}|g" Makefile.def Makefile.python gildas-env.sh define-system.sh
    reinplace -W ${worksrcpath}/admin "s|@CPPFLAGS@|${configure.cppflags}|g" Makefile.def Makefile.python gildas-env.sh define-system.sh
    reinplace -W ${worksrcpath}/admin "s|@LDFLAGS@|${configure.ldflags}|g" Makefile.def Makefile.python gildas-env.sh define-system.sh
    reinplace -W ${worksrcpath}/admin "s|@PYTHON@|${configure.python}|g" Makefile.def Makefile.python gildas-env.sh define-system.sh
    reinplace -W ${worksrcpath}/admin "s|@STDCXXLIB@|${cxxstdlib}|g" Makefile.def Makefile.python gildas-env.sh define-system.sh

There are 10 keys to be reinplaced, which means possibly up to 40 combinations of key+file. It has been chosen to apply all the reinplacements to all files for the sake of completeness, at the time the Portfile was first created, and for later changes. Yes, I could put the proper list of files to each reinplace invocation, but this means keeping an up-to-date list with the risk it becomes inconsistent with what the patched files actually contains.

[bardeau]

Regarding the C++ link: I am a bit confused by all the information given, in particular in #51726. But as far as I understand, you suggest to use the MacPorts variable ${configure.cxxflags} to compile the C++ files. Under my environment, it resolves as "-Os -stdlib=libc++". This is unfortunate as it provides a compilation flag (-Os) together with a link flag (-stdlib=libc++). In Gildas, compilation and link are 2 clearly separated steps. Which configuration variable provides only the C++ compilation flags?

[ryandesign (Ryan Carsten Schmidt)]

Replying to bardeau:

Regarding the C++ link: I am a bit confused by all the information given, in particular in #51726. But as far as I understand, you suggest to use the MacPorts variable ${configure.cxxflags} to compile the C++ files. Under my environment, it resolves as "-Os -stdlib=libc++".

Of course.

This is unfortunate as it provides a compilation flag (-Os) together with a link flag (-stdlib=libc++). In Gildas, compilation and link are 2 clearly separated steps. Which configuration variable provides only the C++ compilation flags?

As far as I know, having the optimization flag in there when linking should not be a problem. Thousands of other ports use it happily. But if it is a problem, you can clear it by clearing configure.optflags; it will then disappear from configure.cxxflags etc. In this case, make sure the build system already sets suitable optimization flags or make other arrangements for supplying them.

[ryandesign (Ryan Carsten Schmidt)]

Replying to bardeau:

There are 10 keys to be reinplaced, which means possibly up to 40 combinations of key+file. It has been chosen to apply all the reinplacements to all files for the sake of completeness, at the time the Portfile was first created, and for later changes. Yes, I could put the proper list of files to each reinplace invocation, but this means keeping an up-to-date list with the risk it becomes inconsistent with what the patched files actually contains.

Every reinplace warning should be eliminated. Usually that's done by making the reinplace replace something. In the unusual circumstance that you want to allow a reinplace that does nothing, add the -q flag to quiet the warning (and be prepared to manually audit your reinplaces periodically to ensure they're still doing what you intended). We would like to turn the warning into a hard error in the future; if we did that today, it would prevent this port from installing at all. See #60844.

The C++ stdlib mismatch issues of this ticket also still need to be resolved.

[bardeau]

For the time being ​this pull request will fix the reinplace warnings.