cmake @3.16.1: error: no matching constructor for initialization of 'const std::vector<ExpectedSpec>'

[letaage]

Hello,

Just trying to update cmake from 3.15.5_0 to 3.16.0_0 and it is failing with the following error message (last 20 lines of log):

:info:build cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.16.0/Tests/RunCMake && /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.16.0/Bootstrap.cmk/cmake -E cmake_link_script CMakeFiles/ctresalloc.dir/link.txt --verbose=ON
:info:build /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++  -pipe -Os -stdlib=libc++ -arch x86_64 -mmacosx-version-min=10.8 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -L/opt/local/lib -Wl,-headerpad_max_install_names -lMacportsLegacySupport -lMacportsLegacySupport -arch x86_64 CMakeFiles/ctresalloc.dir/CTestResourceAllocation/ctresalloc.cxx.o  -o ../../bin/ctresalloc  ../../Source/libCTestLib.a ../../Source/libCMakeLib.a ../../Source/kwsys/libcmsys.a ../../Utilities/std/libcmstd.a /opt/local/lib/libexpat.dylib /opt/local/lib/libz.dylib /opt/local/lib/libarchive.dylib ../../Utilities/cmjsoncpp/libcmjsoncpp.a /opt/local/lib/libuv.dylib ../../Utilities/cmlibrhash/libcmlibrhash.a -framework CoreFoundation -framework CoreServices /opt/local/lib/libcurl.dylib 
:info:build make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.16.0'
:info:build [ 97%] Built target ctresalloc
:info:build make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.16.0'
:info:build make: *** [all] Error 2
:info:build make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.16.0'
:info:build Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.16.0" && /usr/bin/make -j2 -w all VERBOSE=ON 
:info:build Exit code: 2
:error:build Failed to build cmake: command execution failed
:debug:build Error code: CHILDSTATUS 18380 2
:debug:build Backtrace: command execution failed
:debug:build     while executing
:debug:build "system {*}$notty {*}$nice $fullcmdstring"
:debug:build     invoked from within
:debug:build "command_exec build"
:debug:build     (procedure "portbuild::build_main" line 8)
:debug:build     invoked from within
:debug:build "$procedure $targetname"
:error:build See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/main.log for details.

I have cleaned and rebuilt but the problem persists and it is preventing me from upgrading ports with a dependency on cmake 3.16.0.

Complete log attached

[letaage]

Build log file

[jmroot (Joshua Root)]

:info:build /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.16.0/Tests/CMakeLib/testCTestResourceSpec.cxx:14:40: error: no matching constructor for initialization of 'const std::vector<ExpectedSpec>'
:info:build static const std::vector<ExpectedSpec> expectedResourceSpecs = {
:info:build                                        ^                       ~

[jmroot (Joshua Root)]

Looks like the same error as #59782. Seems to have built OK on the buildbot: ​https://build.macports.org/builders/ports-10.8_x86_64-builder/builds/9528

[kencu (Ken)]

Too bad I don't have the c++ skills to tweak that bit of code so that the older compilers can handle it.

[jmroot (Joshua Root)]

You could try using direct initialization instead; it doesn't look like there's any reason to use copy initialization. Basically just delete the =, resulting in

static const std::vector<ExpectedSpec> expectedResourceSpecs{ (…)

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #59833.

3.16.1 failed on the ​10.8 and ​10.9 buildbot workers.

[michaelld (Michael Dickens)]

OK let me get one of my build computers going with 10.9 and updating ... I have the C++XY skillz to do this ... just need an active host & build and mine are a little out of date ...

[RJVB (René Bertin)]

I just built cmake 3.16.1 on 10.9.5; the vector error doesn't occur with port:clang-5 (and newer I'd guess, possibly certain older versions too). Not that that helps during a clean install from source, where you obviously cannot use a MacPorts clang compiler.

Anyway, I had to apply another patch:

diff --git Utilities/std/cm/optional Utilities/std/cm/optional
index 80b0951604ef8688f5a27d6b5a8caab2a70fe5cb..304120a6cb7598af2057fd04ff8a6b388c552a50 100644
--- Utilities/std/cm/optional
+++ Utilities/std/cm/optional
@@ -6,9 +6,9 @@
 #ifndef cm_optional
 #define cm_optional
 
-#if __cplusplus >= 201703L || (defined(_MSVC_LANG) && _MSVC_LANG >= 201703L)
-#  define CMake_HAVE_CXX_OPTIONAL
-#endif
+// #if __cplusplus >= 201703L || (defined(_MSVC_LANG) && _MSVC_LANG >= 201703L)
+// #  define CMake_HAVE_CXX_OPTIONAL
+// #endif
 
 #if defined(CMake_HAVE_CXX_OPTIONAL)
 #  include <optional> // IWYU pragma: export
diff --git Utilities/std/cm/shared_mutex Utilities/std/cm/shared_mutex
index 2ac9447dc7593221b1564490d1aa58493ffa722a..c3419e4509651dcb8b5b3c99d57e5cd47bd7c600 100644
--- Utilities/std/cm/shared_mutex
+++ Utilities/std/cm/shared_mutex
@@ -9,9 +9,9 @@
 #if __cplusplus >= 201402L || defined(_MSVC_LANG) && _MSVC_LANG >= 201402L
 #  define CMake_HAVE_CXX_SHARED_LOCK
 #endif
-#if __cplusplus >= 201703L || defined(_MSVC_LANG) && _MSVC_LANG >= 201703L
-#  define CMake_HAVE_CXX_SHARED_MUTEX
-#endif
+// #if __cplusplus >= 201703L || defined(_MSVC_LANG) && _MSVC_LANG >= 201703L
+// #  define CMake_HAVE_CXX_SHARED_MUTEX
+// #endif
 
 #if defined(CMake_HAVE_CXX_SHARED_LOCK)
 #  include <shared_mutex> // IWYU pragma: export

I get a clear message that std::shared_mutex is available only 10.12 and up (never saw that before!). I'm assuming that the same is true for std::bad_optional_access so I apply the patch above on ${os.major} <= 15.

BTW, Michael, you can really speed things up with another tweak, to the bootstrap script which makes it use the pre-existing cmake command in case of an upgrade or repeated build:

diff --git bootstrap bootstrap
index 1f8eaa50394a6f32a0121ef911ca8b7e6f2f2f35..69687faa7b26d9e9ea8c8d623cd28e23ef0224d6 100755
--- bootstrap
+++ bootstrap
@@ -1644,15 +1644,20 @@ fi
 
 echo "---------------------------------------------"
 
-# Run make to build bootstrap cmake
-if [ "x${cmake_parallel_make}" != "x" ]; then
-  ${cmake_make_processor} ${cmake_make_flags}
+if [ -x ${cmake_prefix_dir}/bin/cmake ] ;then
+  # use the existing cmake that we'll be replacing.
+  ln -s ${cmake_prefix_dir}/bin/cmake ${cmake_bootstrap_dir}/cmake
 else
-  ${cmake_make_processor}
-fi
-RES=$?
-if [ "${RES}" -ne "0" ]; then
-  cmake_error 9 "Problem while running ${cmake_make_processor}"
+  # Run make to build bootstrap cmake
+  if [ "x${cmake_parallel_make}" != "x" ]; then
+    ${cmake_make_processor} ${cmake_make_flags}
+  else
+    ${cmake_make_processor}
+  fi
+  RES=$?
+  if [ "${RES}" -ne "0" ]; then
+    cmake_error 9 "Problem while running ${cmake_make_processor}"
+  fi
 fi
 cd "${cmake_binary_dir}"

[michaelld (Michael Dickens)]

I've done a -lot- of playing around with the default compiler on OSX 10.9.5, which is Xcode 6.2 (6C131e) "Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)". I'm pretty convinced that it can't handle the C++11 std::initializer_list parsing required by the code. MacPorts Clang 3.7 builds this code just fine; it's the oldest I have installed right now. As soon as the updates are complete I'll install clang-3.4 and clang-3.5 to see if either of those work -- guessing 3.4 won't, but 3.5 might.

Anyway, my current believe of our best solution here is to just blacklist this compiler on older OSX (e.g., "clang < 3.7") or however we phrase it. By now, most of the older OSX will have mulitple modern compilers installed already, so I don't see this as a deal-breaker -- at least so long as we can disable testing to bootstrap cmake using the Xcode-provided compiler ... not sure we do that right now for when bootstrapping ... hmmm ... have to look into disabling testing for when we're doing that.

[michaelld (Michael Dickens)]

@RJVB : for the first patch, you found it was required to build on 10.9.5, regardless of the modern-enough Clang compiler selected? I can see in the debug log that even the Tests are being compiled with c++1z ... no idea why since Clang 6.0 cannot support it & I thought I had fixed up the tests for it to really truly fail if c++1z headers were not available ... now I need to go back and revisit what's going on there ...

[michaelld (Michael Dickens)]

@RJVB : for the 2nd patch: I like the idea! My primary wondering is whether CMake internally uses new features to build itself. This would make sense, at least for some features. If/when this is the case, then using an older CMake to build the newer CMake is likely to result in failure. If this is not the case, then I'd be all for doing this since it would -definitely- speed up cmake builds! A good way to test would be to revert cmake back to 3.14 and then try to build 3.16 using 3.14 ... guessing there are enough changes between those version to cause build errors if there will ever be any!

[Ionic (Mihai Moldovan)]

If I remember correctly, cmake is a build-dependency of other (bootstrapping) software like llvm/clang, so be careful not to create a loop.

llvm-3.7 was the last version to use autotools instead of cmake (or it was at least deprecated in 3.8 and might not work for any newer version), which is also one of the reasons clang-3.7 is still around in MacPorts.

Be careful with that stuff.

[Ionic (Mihai Moldovan)]

Also, please don't try to use the already-installed cmake version for building cmake. This will cause all trace mode builds to fail. It also makes cmake builds unreproducible. Yes, it might speed up builds, but only just a bit. The initial bootstrapping is fast enough to not cause huge trouble.

Don't use such a hack.

[Ionic (Mihai Moldovan)]

The initial issue presented here is not (immediately) happening because the system clang version on 10.9 is too old to support initializer lists (which would have been weird in the first place, since it's roughly based upon clang 3.4 and initializer lists have been supported since clang 3.1), but because cmake supplies -std=gnu++1y, which it doesn't understand and just silently ignores. Instead of enabling the C++11 mode, it just stays in the default C++03 mode, leading to the failure seen here.

Now... does cmake actually need C++14? If not, we could switch to something like -std=gnu++11.

[Ionic (Mihai Moldovan)]

Aaaand that was wrong again. Apple-clang 6.0 does support gnu++1y as a parameter to -std.

The issue seems to be tries to explicitly initialize the std::map structure inside of the cmCTestResourceSpec structure and older compilers choke on that, while it works with newer ones.

If you use the universal initializer for that part, it just works with older compilers (i.e., {{{}}} => {}).

Arguably, that's a more elegant solution anyway, but I don't understand why the original specification can't be mapped correctly.

[michaelld (Michael Dickens)]

I tried forcing c++11 with this compiler (via extracting the build command and replacing the gnu++1y with c++11), and it still didn't work ... same issue. Somehow, as you note, the compiler isn't interpreting the initializer_list for std::map correctly. Hence, IMHO the compiler is broken for this specific project's needs, and blacklisting it is the right thing to do.

[michaelld (Michael Dickens)]

I'll be back at work on Monday where by build computers are located. Maybe possibly clang-3.4 will be installed & I can test it with this issue to verify whether it works or not. Then, I can work out the correct blacklist entry. Again: Given the age of the OS (and other impacted OSs with this bug), it is very likely that newer MacPorts' clang will be installed & hence the blacklist is a nominal issue.

[michaelld (Michael Dickens)]

Related to using an already-installed cmake to build a newer one: RJVB's change checks to see whether cmake is installed, and if so uses it. If cmake is not installed then the whole bootstrap process is used. That said, my concern about the old cmake not being able to compile new cmake still stands, and until there is some evidence otherwise (that older can indeed be used to build newer, regardless of feature differences), I'm not going to use the change. For the vast majority of users, the pre-compiled binary will do the trick, and so there is very little incentive to try to speed up the build process in this manner.

[michaelld (Michael Dickens)]

Looking back at my prior blacklist comment & pre-compiled binary: do we need the Xcode clang to be able to build cmake when doing bootstrap, since the pre-compiled binary won't be available yet (somehow)? I'm not clear on how these fit together. blacklisting would, of course, mean that a pre-compiled binary would be required for any OS we target, which might not be the way we want to go here ...

[Ionic (Mihai Moldovan)]

I think so, yes. Newer llvm versions use cmake as their built system so blacklisting a version that is used as a bootstrap compiler will lead to a dead end. Maybe the loop thing was wrong, I don't remember.

I haven't followed this stuff very actively, though, jeremyhu probably knows much better. It also mostly concerns really, really, really old operating systems that do not feature libc++ or C++11 support out of the box and hence need to bootstrap with old llvm/clang versions in order to get a newer one.

I'll see if a very simple patch gets cmake going again on my 10.9 system. That would be great. No promises though.

Related to using an already-installed cmake to build a newer one [...]

Yep, that makes build unreproducible, including the issue of not knowing whether old versions can be used to correctly bootstrap the package. I also don't know how that would pan out in trace mode, because sometimes trace mode doesn't seem to hide the files completely, but merely deny read or write access to them, so such a check would still come out true, but subsequent uses fail.

[kencu (Ken)]

someday we will need to choose a version of cmake to be 'cmake-bootstrap'. Just trying to stave that off a bit.

When that time comes, cmake-3.9 might be a good choice as it builds so easily, being the last that didn't need c++11.

I already have a cmake39 port myself that could become cmake-bootstrap when we need it.

[Ionic (Mihai Moldovan)]

This seems to build fine on my 10.9 machine... and should theoretically also work fine on newer operating systems. Please check: ​https://git.ionic.de/macports-overlay.git/commitdiff/64e93dc5ee21c79cc1c4cf3fa6f551c5613291e7

[kencu (Ken)]

Replying to Ionic:

This seems to build fine on my 10.9 machine... and should theoretically also work fine on newer operating systems. Please check: ​https://git.ionic.de/macports-overlay.git/commitdiff/64e93dc5ee21c79cc1c4cf3fa6f551c5613291e7

That looks encouraging...

[michaelld (Michael Dickens)]

I had worked out the final reduction from {{{}}} to {} ... but I still couldn't get the first 2 sets to work ... maybe I did & somehow overlooked it? Anyway I can test this when I'm back to work Monday. Would be really nice to have a generic solution!

[Ionic (Mihai Moldovan)]

The first set, which explicitly initializes an std::map object with multiple full entries must stay as-is. All others need to be reduced to the universal zero initializer. My patch shows that.

I also believe this to be a compiler bug, but if we can fix it that easily, why not? We probably should also upstream it. At least I don't see any reason for not to: the change is not a hack but adheres nicely to the standard.

[michaelld (Michael Dickens)]

Tested & confirmed @ionic's fix works! I'll fix this up in MP shortly.

[michaelld (Michael Dickens)]

In 16a8f77942fbb97c27a435320a619f554ed489b8/macports-ports (master):

cmake: add patch to fix struct init that breaks builds on older OSs

Fix seems to be universal, not specific for any given OS or compiler.

Closes: #59822

[michaelld (Michael Dickens)]

Thanks @ionic! Fix tested and verified on 10.9 and 10.14 ; builds cleanly on both so merging in the fix. we'll see how it does on the buildbots!

[Ionic (Mihai Moldovan)]

Good that it works overall. But still... that's prrobably something we should upstream, right?

[michaelld (Michael Dickens)]

yes ... working on it ...

[michaelld (Michael Dickens)]

​https://gitlab.kitware.com/cmake/cmake/issues/20097

[michaelld (Michael Dickens)]

FYI this change was just merged: ​https://gitlab.kitware.com/cmake/cmake/commit/e8dbe4bba10dff2ac865b145830c931d1154d8fc ... it'll be in the next CMake 3.16 release, which I'm guessing will be within the next week. I'll update the MP CMake release once this is out & then close this ticket. Go team!

[michaelld (Michael Dickens)]

In 24645a1ba28bb6826f407cf51e35c57a6ef0f33c/macports-ports (master):

cmake: update release to 3.16.2 and devel to 20191219-10fea251

Remove patch that was integrated upstream

Closes: #59822

[michaelld (Michael Dickens)]

In 16a8f77942fbb97c27a435320a619f554ed489b8/macports-ports (dar, master, py38-reproject, revert-6945-rust-1.43.0, wireshark):

cmake: add patch to fix struct init that breaks builds on older OSs

Fix seems to be universal, not specific for any given OS or compiler.

Closes: #59822

[michaelld (Michael Dickens)]

In 24645a1ba28bb6826f407cf51e35c57a6ef0f33c/macports-ports (dar, master, py38-reproject, revert-6945-rust-1.43.0, wireshark):

cmake: update release to 3.16.2 and devel to 20191219-10fea251

Remove patch that was integrated upstream

Closes: #59822