cmake: doesn't recognize macports-clang (No known features for CXX compiler)

[RJVB (René Bertin)]

I just ran into an incomprehensible error where everything always worked just fine before, when using configure.compiler=macports-clang-3.9:

-- Configuring done
CMake Error in src/platformtheme/CMakeLists.txt:
  No known features for CXX compiler

  "Clang"

  version 3.9.0.

After much head scratching and googling, I tracked that down to a line in the Qt5CoreConfigExtras.cmake file shipped by Qt 5.7.1 (I'm testing that Qt version):

set_property(TARGET Qt5::Core PROPERTY INTERFACE_COMPILE_FEATURES cxx_decltype)

From what I understand, CMake only supports compile[r] features with GCC and AppleClang, on Mac. Indeed, outcommenting that line fixes the configure error.

I'm reporting this as a CMake bug in hope Michael can find a proper fix (to be reported upstream). I'm CC'ing the llvm/clang maintainers as a FYI, and Marcus so he's aware of this nasty little breaker too.

​https://bugreports.qt.io/browse/QTBUG-57886

[RJVB (René Bertin)]

This issue can be avoided by adding

cmake_policy(SET CMP0025 NEW)

somewhere at an appropriately early location. I wonder if we cannot activate that policy by default? It's supposed to become the default anyway...

[RJVB (René Bertin)]

I haven't yet figured out a proper way to change the policy default in the C++ code, but there are a few options to set the desired behaviour globally:

• remove the check-and-return from Clang-C.cmake and Clang-CXX.cmake . Blunt, and you cannot go back to the "OLD" policy (should you ever want to)

• following ​https://cmake.org/cmake/help/v3.0/manual/cmake-policies.7.html, add a command line option e.g. to configure.pre_args :
• -DCMAKE_MINIMUM_REQUIRED_VERSION=3.0

or

• -DCMAKE_POLICY_DEFAULT_CMP0025=NEW

I tested the latter in cmake-1.1.tcl and it works.

[RJVB (René Bertin)]

I was a bit too enthusiastic: setting CMP0025 on the commandline can still be undone by a cmake_minimum_required(VERSION x) with x < 3.0 in the CMakeLists.txt file. It turns out it is NOT possible to set the minimum required version on the commandline.

Annoyingly, this can lead to an impressive series of errors that aren't very helpful though I guess that after a while you understand what they imply.

I can think of only 1 way to prevent this:

• create a cache file (akin to QMake's .qmake.cache), say ${workpath}/.cmake.initcache that contains cmake_minimum_required(VERSION 3.0)
• tell CMake to load that file with -C${workpath}/.cmake.initcache

(I'm using ${workpath} because no sane project should be writing anything there, whereas we cannot be sure about ${build.dir} and ${worksrcpath}).

This file can be written in the pre-configure, or when the default configure.pre_args value is fixed.

Thoughts - @michaelld, @larryv?

[neverpanic (Clemens Lang)]

Please do not attempt to change cmake_minimum_required(VERSION) from the command line; it's meant to simplify backwards-incompatible changes in CMake, and setting it to something on random CMakeLists.txt files is begging for problems.

You can probably run cmake_policy(SET CMP0025 NEW) in a CMake Toolchain file passed using -DCMAKE_TOOLCHAIN_FILE. However, this looks a lot like a CMake bug to me, and rather than running in circles around that bug, you should report it to the developers of CMake and have them fix it. If you've done that already, please provide a link to the bug report.

[RJVB (René Bertin)]

Hah, that's you and Qt's Thiago who agree that CMake should be doing something about this, and as I expected the consensus among the CMake devs (in the form of a single person) is that there is no bug, projects just have to do the right thing, know all the little details on all platforms where they might be deployed, etc.

Evidently I would have made either the minimum version configurable, or else the whole init.cache thing optional. But that point is moot: it seems one cannot select a minimum version that way.

So I'll stick to setting -DCMAKE_POLICY_DEFAULT_CMP0025=NEW in configure.pre_args. That is something that can be undone with the usual methods, but a priori it only has the effect that cmake will accept any clang compiler on Mac, not just one identifying as AppleClang. That seems a behaviour we want anyway.

If we agree on that we can also just remove the policy 25 checks in Clang-C.cmake and Clang-CXX.cmake. The effect of that will be that OS X is not treated any different than standard Unix (rather, anything not Win32) in that no more difference will be made between Apple clang, macports-clang or any other home brewn clang ;)

If you've done that already, please provide a link to the bug report.

I can't: bugs are reported on their mailing list, and I'm not yet seeing an archived version. The discussion thread subject is COMPILE_FEATURES, Mac and non-Apple clang versions.

[mojca (Mojca Miklavec)]

Replying to RJVB:

Bugs are reported on their mailing list, and I'm not yet seeing an archived version. The discussion thread subject is COMPILE_FEATURES, Mac and non-Apple clang versions.

I now found an archived version:

• ​http://public.kitware.com/pipermail/cmake-developers/2017-January/029720.html

Maybe there's a lag in updating their mail archive.

These are the relevant files:

/opt/local/share/cmake-3.8/Modules/Compiler/AppleClang-ASM.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/AppleClang-C-FeatureTests.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/AppleClang-C.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/AppleClang-CXX-FeatureTests.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/AppleClang-CXX.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/AppleClang-DetermineCompiler.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/Clang-ASM.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/Clang-C-FeatureTests.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/Clang-C.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/Clang-CXX-FeatureTests.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/Clang-CXX-TestableFeatures.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/Clang-CXX.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/Clang-DetermineCompiler.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/Clang-DetermineCompilerInternal.cmake
/opt/local/share/cmake-3.8/Modules/Compiler/Clang.cmake
/opt/local/share/cmake-3.8/Modules/Platform/Darwin-AppleClang-C.cmake
/opt/local/share/cmake-3.8/Modules/Platform/Darwin-AppleClang-CXX.cmake
/opt/local/share/cmake-3.8/Modules/Platform/Darwin-Clang-C.cmake
/opt/local/share/cmake-3.8/Modules/Platform/Darwin-Clang-CXX.cmake
/opt/local/share/cmake-3.8/Modules/Platform/Darwin-Clang.cmake

My suggestion would be to come up with a set of proper patches for these files and submit the patches upstream.

The problematic part seems to be

cmake_policy(GET CMP0025 appleClangPolicy)
if(WIN32 OR (APPLE AND NOT appleClangPolicy STREQUAL NEW))
  return()
endif()

and I guess that's because of Apple's confusing numbering. I guess all we need to do is distinguish between a stock and Apple clang and simply do the same thing as on any other Unix system when a stock Clang compiler is chosen.

[RJVB (René Bertin)]

Hmmm, evidently I forgot about this ticket.

A patch is being tested for the Qt cmake module that triggers the issue and which got me acquainted with it.

Mojca is right in her assessment of the problematic part, I'll be attaching a port:cmake patch that seems to have "resolved" the problem for me. It removes the whole taking-into-consideration of the CMP25 policy. I still set the policy to NEW on the cmake commandline (in my still-pending modified cmake 1.1 PG) and together with the patch I've been able to build cmake-based projects with any compiler I've thrown at them.

Given KitWare's attitude I think that shunting the whole policy is an acceptable approach for MacPorts. To be really pedantic it could be coupled to an install variant which is made default only on specific platforms, but I don't see any problems making it unconditional. It's a MacPorts goal that one can specify the compiler to use for building a port, and a silly CMake distinction between stock clang and AppleClang shouldn't stand in the way. Not on Mac in any case, where there isn't supposed to be any such difference.

[RJVB (René Bertin)]

Mojca voiced a concern on the ML that my patch could lead to mis-identification of features of the system AppleClang compiler.

I found an equivalence table of sorts: ​https://gist.github.com/yamaya/2924292

which I can complete with

Xcode 3.2.6:

> /Developer/usr/bin/clang --version
Apple clang version 1.7 (tags/Apple/clang-77) (based on LLVM 2.9svn)

and Xcode 4.2:

> /Developer/Xcode4/usr/bin/clang --version
Apple clang version 3.0 (tags/Apple/clang-211.10.1) (based on LLVM 3.0svn)
Target: x86_64-apple-darwin13.4.0

(both are identified as regular Clang by CMake, v1.7 and v3.0.0 respectively)

Indeed, such issues could occur with AppleClang versions based on Clang 3.4 and earlier. Clang 3.5 and later are fine as the corresponding AppleClang have version 6 and up in CMake.

However, it looks like indeed the compilers provided by Xcode >= 4.4 (AppleClang 4.x) and < 6.0 (AppleClang 5.1.x) will be identified as a more recent compilers and thus endowed with features they don't have.

The linked table also shows that setting up an equivalence table in CMake script code will be more complicated than it should be if we don't figure out where CMake generates its AppleClang version strings from what the compiler returns ("6.0.0.6000057" for "Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)").

We'd still need to do something like this (using the "(based on LLVM x.y*)" string?) if we want to get a patch upstreamed, but we could also be consider that we're approaching this from the wrong side for MacPorts purposes. The easy way out here would be to patch the clang ports such that they spoof the corresponding AppleClang version string. As I said, that would only be necessary for clang 3.4 and below, and could be reserved for when the port is to provide the MacPorts default compiler. If someone could add Jeremy and/or Larry to this ticket, they might have feedback on this.

[mojca (Mojca Miklavec)]

Both Jeremy and Larry are subscribed to this ticket. I don't think the patch should worry about equivalence table, but merely do the same thing as it's done on Linux for the stock Clang complier. Worrying about equivalence table should be left to CMake developers when supporting AppleClang. Our task is easier.

Not that I understand this madness about trying to cover the database of all features of all compilers. Testing for features sounds way more useful. But we should primarily make sure that we don't bail out just because "macports-clang-x.y" is used.

[RJVB (René Bertin)]

Replying to mojca:

do the same thing as it's done on Linux for the stock Clang complier.

Eh? Correct me if I'm wrong, but the same compiler module is used everywhere for Clang. Yes, there are AppleClang modules. I'm not certain where/when exactly they come into play, but as far as I can tell they're not where the CMP25 policy is enacted.

Actually, I realise that it's possible that they are used when using an AppleClang version, and the regular Clang-C[XX].cmake modules when using a stock (and MacPorts) clang version. If that's the case there is indeed no need to worry about equivalence tables, and your concern that my patch might interfere with feature detection for the system compiler largely moot. Provided that the system compiler on the OS X versions you are considering is indeed detected as AppleClang.

Worrying about equivalence table should be left to CMake developers when supporting AppleClang. Our task is easier.

Not if we want to be able to present a full-featured patch to make them reconsider their point of view.

Not that I understand this madness about trying to cover the database of all features of all compilers. Testing for features sounds way more useful.

Look at the modules, not all features are concerned, only the important ones that often need to be tested for, and which may not be easy to verify with a true (but still cheap) compile check. A compiler version might have an option to select C++XY mode, but we know from experience that this doesn't guarantee that it has complete and reliable support. That's where this database comes in.

[RJVB (René Bertin)]

Seems my hunch is correct:

1. AppleClang-C[XX].cmake is included when using a recognised Apple system Xcode compiler using an Apple LLVM versioning scheme
2. Clang-C[XX].cmake is included otherwise.

2) also applies to the Clang compilers provided by Xcode 3.2.6 and 4.2, i.e. the system Clang compilers on 10.6 (and earlier, undoubtedly), but that's because they identify themselves as as stock Clang compilers.

To summarise, I think this was a storm in a glass of water and a false alarm in the end. The patch I attached should be safe from what I can tell (= never interfere with feature determination of the system compiler).

If ever you do run into such an issue we can take additional measures but I think we can and should rely on the KitWare guys having done their homework "how to detect AppleClang".

What we can do is add an option that a) is less ugly than the policy option and b) not affected by what the project's cmake files do in terms on minimal required version, policy setting etc. For instance (in Clang-CXX-cmake):

cmake_policy(GET CMP0025 appleClangPolicy)
option(CMAKE_CXX_REJECT_NON_APPLECLANG "Reject Clang++ other than AppleClang++ on Mac unless Policy 25 is set to NEW" OFF)
if(APPLE AND CMAKE_CXX_REJECT_NON_APPLECLANG AND NOT appleClangPolicy STREQUAL NEW)
  return()
endif()

This has the same effect as my earlier patch, but makes it possible to revert to CMake's stock behaviour by adding -DCMAKE_CXX_REJECT_NON_APPLECLANG=ON to the commandline.

[RJVB (René Bertin)]

version for 3.10.0 and up

[RJVB (René Bertin)]

Almost a year now that I run CMake with this patch (and quite heavily so), without any negative side-effects.

The only side-effect of this kind of patch is that you can no longer use cmake-mp for testing stock cmake behaviour. Probably a moot issue in this case but easy to address. Just add a non-default +stock variant to ports that come with non-crucial patches adapting their behaviour to MacPorts "requirements" like "we should be able to use any compiler".

[jeremyhu (Jeremy Huddleston Sequoia)]

Is there an upstream bug report tracking this?

[RJVB (René Bertin)]

No, it was discussed on the cmake-devel ML on Jan 3rd and 4th 2017 (the pipermail link in Mojca's comment above). I haven't re-read the whole exchange but the take-home message was "works as intended" for me, and "just set policy 25 where and as required".

We could do that too of course, on a port-by-port basis, but I see no reason to impose the maintenance cost of that if it's so trivial to bypass the whole policy thing. AFAICT that was introduced only because Apple's clang identifies itself in a different way than stock clang. The policy would be useful if it modified the feature lookup in such a way that it works better with Apple's clang versioning scheme but I never found any evidence of it does. All it seems to do in MacPorts context is get in the way and make it impossible to use Apple/Xcode clang and MacPorts clang (or clang from other sources) interchangeably.

[michaelld (Michael Dickens)]

Is this still an issue for CMake 3.18.0?

[RJVB (René Bertin)]

No idea, TBH. I'm still rocking 3.16.1 and still with the latest patch I attached, and my copy of the cmake-1.1 PG sets policies 25 and 60 to "new". If you hadn't asked I would only have looked into the question if said patch needed refactoring ;)

[cooljeanius (Eric Gallager)]

Bug #53615 is similar