cmake 1.1 PG : ccache support

[RJVB (René Bertin)]

I keep evolving my cmake 1.1 PG (​https://github.com/RJVB/macstrop/blob/master/_resources/port1.0/group/cmake-1.1.tcl); yesterday I figured out how to support ccache:

options                             cmake.ccache

# we support ccache via our own cache variable (which incompatible projects can turn off).
default cmake.ccache                [tbool configure.ccache]

# tell CMake to use ccache via the CMAKE_<LANG>_COMPILER_LAUNCHER variable
# and unset the global configure.ccache option which is not compatible
# with CMake.
# See https://stackoverflow.com/questions/1815688/how-to-use-ccache-with-cmake
proc cmake::ccaching {} {
    global cmake.ccache prefix
    if {[tbool cmake.ccache] && [file exists ${prefix}/bin/ccache]} {
        return [list \
            -DCMAKE_C_COMPILER_LAUNCHER=${prefix}/bin/ccache \
            -DCMAKE_CXX_COMPILER_LAUNCHER=${prefix}/bin/ccache]
    }
}

configure.ccache    no

default configure.pre_args {[list \
                    -DCMAKE_BUILD_TYPE=${cmake.build_type} \
                    -DCMAKE_INSTALL_PREFIX="${cmake.install_prefix}" \
                    -DCMAKE_INSTALL_NAME_DIR="${cmake.install_prefix}/lib" \
                    {*}[cmake::system_prefix_path] \
                    {*}[cmake::ccaching] \
                    {-DCMAKE_C_COMPILER="$CC"} \
                    {-DCMAKE_CXX_COMPILER="$CXX"} \
<snip>

FWIW, I did this thinking it could speed up building llvm and clang from source as the latter seems to rebuild a good part of the former. No such luck, I got barely more than 1% cache hits.

[RJVB (René Bertin)]

FWIW, initial tests activating ccache use via the commandline (configure.ccache=yes) seemed to work but may have the effect that "base" sets CC=ccache ${CC}, on top of cmake's COMPILER_LAUNCHER variable being set. That's not a good idea of course.

I think this is a bug in "base"; until resolved one can use cmake.ccache=yes on the command line. This is a benefit (mostly...) of using options variables that I just discovered.

[kencu (Ken)]

I was wondering why none of my cmake builds are using ccache -- which I would really like to have work, building things like root6 over and over...

[mascguy (Christopher Nielsen)]

So is this working...?

[RJVB (René Bertin)]

I think my current version does something to prevent base from adding ccache to the CC env. variable, but that aside, yes, this works like a charm.

[mascguy (Christopher Nielsen)]

So can this ticket be closed?

[RJVB (René Bertin)]

If you wish...

[kencu (Ken)]

It is in the cmake 1.1 PG <​https://github.com/macports/macports-ports/blob/722e1970778c525e7b85ceb2cdb3a83fc1a9cf31/_resources/port1.0/group/cmake-1.1.tcl#L193>.

but do you actually see it working?

[mascguy (Christopher Nielsen)]

I'll let you folks figure out whether this is resolved or not. I'm simply trying to help reduce our open-ticket backlog...

[RJVB (René Bertin)]

Replying to kencu:

It is in the cmake 1.1 PG <​https://github.com/macports/macports-ports/blob/722e1970778c525e7b85ceb2cdb3a83fc1a9cf31/_resources/port1.0/group/cmake-1.1.tcl#L193>.

but do you actually see it working?

Hah, I did have a recollection about going through the motions to get this in, I should have checked.

Yes, this works for me. There are projects that roll their own implementation in their CMake files which may then interfere with what the PG does. Projects designed to use a central, precompiled headerfile also tend to gain nothing from using ccache. Using LTO will cut your possible gain roughly in half. Barring those there is no reason why it wouldn't work. @kencu: do you have an example where it doesn't?

Of course you won't see much benefit with one-time builds. I also have the impression that ccache won't use the cached code from the same file built in a different (source) directory. The interest is mostly for port maintainers who may rebuild a port many times with minimal changes that would invalidate the ccache caches. Then the gain can be enormous.

[mascguy (Christopher Nielsen)]

On a semi-related note: For our Python ports - and specifically, those that are using Ninja - the CCache variables must be set via the environment to have an effect.

For example, the following was necessary for port 'py-pytorch':

    # TODO: CCache support might be worth adding to Python portgroup
    if {[tbool configure.ccache]} {
        build.env-append \
            CMAKE_C_COMPILER_LAUNCHER=${prefix}/bin/ccache \
            CMAKE_CXX_COMPILER_LAUNCHER=${prefix}/bin/ccache
    }

Ken and I briefly discussed, but neither of us are intimately familiar with the interactions between Ninja, CMake, etc. So the question is, can this be accomplished without adding to the environment?

And if not, is this something we might want to add to the CMake portgroup... or to Python?

Thoughts?

[kencu (Ken)]

In the cmake-1.1 PortGroup, where this is all handled, I don't see that the appropriate configure args are ever actually added to the configure.pre_args list.

Chris, we might just check that somebody actually remembered to add them to the build line, and they are actually there in practice.

There should be no need to add the CMAKE args to the environment. None of the other CMAKE args need to be added to the environment.