RFE: Improve Compiler Management Code

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

I would like to make the following improvements to the MacPorts code:

• Eliminate the cxx11 PorGroups (​1.0 and ​1.1).
• Prevent the creation of a c11 PortGroup.
  • grep -r "support C11" --include=Portfile * | cut -d "/" -f2 | wc -l lists 21 ports.
  • See also #52468.
• Support Fortran compilers without the ​compilers PortGroup, which is the ​current recommendation.
• Simplify (eliminate?) the ​compilers PortGroup.
• Add the latest stable ​GCC and ​Clang compilers to the the compilers ​fallback list.
• Minimize the dependencies when using a MacPorts compiler.
  • When using MacPorts GCC for example, the compiler is added as a library dependency.
  • The compiler should be a build dependency with libgcc as the library dependency.
• Improve the compiler choices for ​PPC systems.
• Turn off universal support when MacPorts GCC is used.

I have created a ​PortGroup to test these changes.
If successful, they can be incorporated into the base code.

[dstrubbe (David Strubbe)]

Could you explain in what sense you think this treatment is superior to the compiler portgroup? It seems like it is simpler by eliminating much of the functionality.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Replying to dstrubbe:

Could you explain in what sense you think this treatment is superior to the compiler portgroup? It seems like it is simpler by eliminating much of the functionality.

Forgive me for not being clear on this point.
I believe the compiler PortGroup could be simplified with the proposed changes.
It could also be eliminated from any ports that just need Fortran and not the full power of the compilers PortGroup.

The compiler PortGroup essentially bypasses portconfigure.tcl (see ​here and ​here).
If portconfigure.tcl were updated, perhaps the code duplication could be eliminated.

With G95 not having been updated ​in quite some time and ​Flang still being in early stages, ​GFortran seems like the only feasible Fortran compiler for the time being.
So for ports that just want a Fortran compiler, adding compiler.require_fortran yes instead of using the compiler PortGroup seems reasonable.

For those ports that still desire the ability to modify the compiler with variants, the compiler PortGroup would remain with full functionality but with a little less code.

[michaelld (Michael Dickens)]

I love the idea here. I'll test out on some of my ports & provide feedback if there is any.

[dstrubbe (David Strubbe)]

Replying to MarcusCalhoun-Lopez:

Replying to dstrubbe:

Could you explain in what sense you think this treatment is superior to the compiler portgroup? It seems like it is simpler by eliminating much of the functionality.

Forgive me for not being clear on this point.
I believe the compiler PortGroup could be simplified with the proposed changes.
It could also be eliminated from any ports that just need Fortran and not the full power of the compilers PortGroup.

What actual changes would happen to the compilers portgroup? I see your proposed changes to base, but what do you propose for the portgroup? The portgroup was largely motivated by providing Fortran compilers, so I'm not sure why you contrast "just need Fortran and not the full power of the compilers PortGroup." g95 works fine as far as I know regardless of whether there were recent updates. Also, my point is that the compiler portgroup does many other things than what your base changes seem to provide. In particular, the ability to track which Fortran compiler was used for a dependency and ensure consistency is important.

[seanfarley (Sean Farley)]

I can try to provide some backstory for the compilers portgroup:

The compilers portgroup serves a (minor-ish) purpose of providing some compatibility between different versions of fortran in combination with gcc or clang/llvm. For instance, as a user I could say, "I want to install superlu and mumps with avx" to which the command would be port install superlu +clang +gfortran and that would make sure the dependencies all match.

This becomes a major problem when considering mpi ports. For those that don't know, both mpich and openmpi are just compiler wrappers. You can see which compiler was used for either mpich or openmpi by mpicc -show. For a port like petsc, all it wants is "give me a mpi library" which, for better or worse, means supplying the correct compiler wrapper. Does the user want mpich with gcc? mpich with clang? openmpi with gcc? The combinatorial explosion is what I tried to wrangle with the compilers and mpi portgroups.

One could argue that the inability to depend on variants is what this boils down to. I dunno. Regardless, keeping in mind the downstream dependents (mpi and friends) is very important when considering changing the compilers portgroup (concrete example: dolfin depends on boost+mpi).

[kencu (Ken)]

My own approach to compiler selection on MacPorts might be somewhat different, I will admit.

Rather than test the oldest compiler that can do the job, for Intel, I would force all systems from 10.6 to 10.9 (or 10.10 perhaps) to build everything, always, with clang-4.0. Basically, use the cxx11 1.1 PortGroup for every build, always, whether c++11 is needed or not, on all older systems. Build failures will be minimized, and there will be toolchain consistency across the OS versions. I would not use the old original compiler for anything. I think it's largely a waste of time to plumb the oldest compiler that can do the job.

At the moment, I think clang-5.0 will generate too many errors to use as the day-to-day compiler, but we'll see how that works out.

Xcode would take some further sorting - you're stuck there unless you're going to do something fancy like SDK overlays with the newer clang compilers (and otools, NM, strings, libtool, etc). I have done this on my system to make xcode build with clang-3.9/libc++ on 10.6.8, but it takes some tinkering. Someone who knows how to make SDK overlays could probably make this work much more easily.

For PPC, it's not so easy. gcc6 can't build some things that apple's gcc4.2 can build, due to objective-C deficiencies. So you're pretty much stuck first trying to see if gcc4.2 can build it, and if not, next trying gcc6.

[kencu (Ken)]

actually, I wonder if I'm right about PPC... perhaps it might be better to flush all the builds towards gcc6, and then spot the ones that gcc6 can't build back to gcc4.2. I'm very close with clang-3.8 on PPC - I think I finish that off and see where it fits into the mix.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

In e8c4772d46ceb8e3f2eac8888097d074da6d6bc2/macports-ports:

languages PG: add more compiler enforcement options

Add:

compiler.fortran_fallback
compiler.openmp_version
compiler.mpi

See #55139

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

I hope that the recent change addresses some of the concerns expressed.
As a very simple example, the following code should allow a port to select the GCC 7 compiler or just the GCC 7 Fortran part.

compiler.require_fortran yes

variant mpich conflicts openmpi {}
if {[variant_isset mpich]} {
    compiler.mpi mpich
}

variant openmpi conflicts mpich {}
if {[variant_isset openmpi]} {
    compiler.mpi openmpi
}

variant gcc7 conflicts gfortran {}
if {[variant_isset gcc7]} {
    compiler.whitelist macports-gcc-7 macports-mpich-gcc-7 macports-openmpi-gcc-7
}

variant gfortran conflicts gcc7 {}
if {[variant_isset gfortran]} {
    compiler.fortran_fallback macports-gcc-7 macports-mpich-gcc-7 macports-openmpi-gcc-7
}

If anyone is interested in testing, I have attached a test Portfile for the reference implementation of ​blas.

[seanfarley (Sean Farley)]

I'm still a bit lost in this discussion. How exactly does fortran_fallback behave? Can a dependent port query for its value?

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

The essential aspects of this ticket have been ​merged.