Opened 3 years ago
Last modified 3 years ago
#62807 closed enhancement
openmpi/mpich: prune target compiler list; refactor duplicated logic, and simplify ports, via new portgroup — at Version 15
Reported by: | mascguy (Christopher Nielsen) | Owned by: | mascguy (Christopher Nielsen) |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | |
Keywords: | Cc: | eborisch (Eric A. Borisch), cjones051073 (Chris Jones), kencu (Ken) | |
Port: | openmpi mpich |
Description (last modified by mascguy (Christopher Nielsen))
I envision this being a very iterative process, with the initial portgroup being something as simple as a few procs to eliminate copy-paste logic.
Also prune the target compiler list significantly, whilst still supporting MacOS releases back to 10.6.
Change History (15)
comment:1 Changed 3 years ago by mascguy (Christopher Nielsen)
comment:2 Changed 3 years ago by mascguy (Christopher Nielsen)
As for other shared logic, the two ports have so many differences - some subtle, and many not-so-subtle - that we might be limited in the refactoring effort.
Although the clist handling - including generation of subports, logic to exclude/fail based on compiler version and macOS release, etc - would be nice to refactor out.
If both ports supported exactly the same set of target compilers, it would be easier...
comment:3 follow-up: 14 Changed 3 years ago by eborisch (Eric A. Borisch)
I'm all for them targeting the same set of compilers; what's the current delta?
Another thought: I've been toying with the idea of having all the non-default (-clang*, etc) flavors use the default libraries and headers rather than build their own, with a defaulted-on +wrapperonly variant.
In this mode, we would just make copies of mpicc
, mpicxx
, etc. with the appropriate changes to CC/CXX etc. in the script (and continue to depend upon the compilers to make sure they are installed.) Executables like mpiexec
could just be a symlink to the -default mpiexec
(likely easier to maintain than having a separate set of select files for the variants).
It's typically rare interested in seeing how the different compilers do at generating the MPI libraries themselves, and you're more interested in using the features of a particular compiler for your (or another port's) code. This would drastically cut down the overhead for having multiple flavors of the wrappers around, while still leaving the options of disabling the variant if you want a full build of MPI libraries by that compiler.
It looks like openmpi uses a binary mpicc (symlinked to a binary, actually), so I'm not sure how easy this would be to implement on openmpi, but it should be fairly easy on mpich.
comment:4 Changed 3 years ago by mascguy (Christopher Nielsen)
Cc: | kencu added |
---|
comment:5 Changed 3 years ago by mascguy (Christopher Nielsen)
This is the current list we support:
openmpi-clang @4.1.1 science/openmpi openmpi-clang10 @4.1.1 science/openmpi openmpi-clang11 @4.1.1 science/openmpi openmpi-clang33 @4.1.1 science/openmpi openmpi-clang34 @4.1.1 science/openmpi openmpi-clang37 @4.1.1 science/openmpi openmpi-clang50 @4.1.1 science/openmpi openmpi-clang60 @4.1.1 science/openmpi openmpi-clang70 @4.1.1 science/openmpi openmpi-clang80 @4.1.1 science/openmpi openmpi-clang90 @4.1.1 science/openmpi openmpi-default @4.1.1 science/openmpi openmpi-gcc5 @4.1.1 science/openmpi openmpi-gcc6 @4.1.1 science/openmpi openmpi-gcc7 @4.1.1 science/openmpi openmpi-gcc8 @4.1.1 science/openmpi openmpi-gcc9 @4.1.1 science/openmpi openmpi-gcc10 @4.1.1 science/openmpi openmpi-gcc49 @4.1.1 science/openmpi mpich-clang @3.4.1 science/mpich mpich-clang10 @3.4.1 science/mpich mpich-clang11 @3.4.1 science/mpich mpich-clang50 @3.4.1 science/mpich mpich-clang60 @3.4.1 science/mpich mpich-clang70 @3.4.1 science/mpich mpich-clang80 @3.4.1 science/mpich mpich-clang90 @3.4.1 science/mpich mpich-default @3.4.1 science/mpich mpich-gcc5 @3.4.1 science/mpich mpich-gcc6 @3.4.1 science/mpich mpich-gcc7 @3.4.1 science/mpich mpich-gcc8 @3.4.1 science/mpich mpich-gcc9 @3.4.1 science/mpich mpich-gcc10 @3.4.1 science/mpich mpich-gcc49 @3.4.1 science/mpich mpich-gccdevel @3.4.1 science/mpich
comment:6 Changed 3 years ago by mascguy (Christopher Nielsen)
Per the earlier discussion on the macports-users list, should we remove all GCC versions earlier than 7?
As for Clang, what's the oldest version we need to support?
comment:7 Changed 3 years ago by kencu (Ken)
I am not aware of any reason to keep support for gcc older than gcc7 in the mpi family.
Likewise, anyone using a macports-clang probably wants at least macports-clang-9.0 by this point, but I don't know if there might be some niche user with an old code base that uses some older macports-clang for something.
maybe deprecate them in some way? Or just bin them?
comment:8 follow-up: 9 Changed 3 years ago by kencu (Ken)
we do need to keep the older gcc versions for other purposes, and probably the older clangs too for similar reasons (bootstrapping or certain older code that needs it and can't/won't be updated).
comment:9 Changed 3 years ago by mascguy (Christopher Nielsen)
Replying to kencu:
we do need to keep the older gcc versions for other purposes, and probably the older clangs too for similar reasons (bootstrapping or certain older code that needs it and can't/won't be updated).
Ken, are you referring to MacPorts as a whole, or just these two ports? I ask, as I'm speaking just for the latter...
comment:10 Changed 3 years ago by cjones051073 (Chris Jones)
yes, we are not suggesting removing the primary clang/gcc ports here Ken, just the mpich/openmpi versions.
comment:12 Changed 3 years ago by mascguy (Christopher Nielsen)
So, would the following set of compilers be sufficient?
- Clang 9 - 11
- GCC 7 - 10
comment:13 Changed 3 years ago by mascguy (Christopher Nielsen)
(Plus the host/Xcode versions, aka, variations "-clang" and "-default")
comment:14 Changed 3 years ago by mascguy (Christopher Nielsen)
Replying to eborisch:
Another thought: I've been toying with the idea of having all the non-default (-clang*, etc) flavors use the default libraries and headers rather than build their own, with a defaulted-on +wrapperonly variant.
In this mode, we would just make copies of
mpicc
,mpicxx
, etc. with the appropriate changes to CC/CXX etc. in the script (and continue to depend upon the compilers to make sure they are installed.) Executables likempiexec
could just be a symlink to the -defaultmpiexec
(likely easier to maintain than having a separate set of select files for the variants).It's typically rare interested in seeing how the different compilers do at generating the MPI libraries themselves, and you're more interested in using the features of a particular compiler for your (or another port's) code. This would drastically cut down the overhead for having multiple flavors of the wrappers around, while still leaving the options of disabling the variant if you want a full build of MPI libraries by that compiler.
It looks like openmpi uses a binary mpicc (symlinked to a binary, actually), so I'm not sure how easy this would be to implement on openmpi, but it should be fairly easy on mpich.
Interesting idea!
Since the default flavor doesn't utilize the buildbot binaries, does that mean an install would always cause a build from source (at least for the default libs)?
comment:15 Changed 3 years ago by mascguy (Christopher Nielsen)
Description: | modified (diff) |
---|---|
Summary: | openmpi/mpich: refactor duplicated logic, and simplify ports, via new portgroup → openmpi/mpich: prune target compiler list; refactor duplicated logic, and simplify ports, via new portgroup |
The first two candidates are as follows. I'm not married to the proc names, but the point is to eliminate copy-paste logic: