Opened 4 years ago

Closed 4 years ago

#62783 closed defect (fixed)

openmpi: validate that buildbot binaries are hardware-independent

Reported by: mascguy (Christopher Nielsen) Owned by: mascguy (Christopher Nielsen)
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc:
Port: openmpi

Description

Presently, the openmpi portfile forces builds from source, as there is a long-standing assumption that OpenMPI builds are specific to the hardware it was built on. While that might have been true for earlier versions, years ago, that doesn't appear to be the case today.

And since the openmpi-* ports build a considerable number of files - and it takes a while, even on modern hardware - it doesn't provide an optimal user experience.

Validate whether the published buildbot-generated binaries run on hardware older than those machines, 2009-era Xserves.

Thankfully, the OpenMPI maintainer is running on a 2008-era MacPro, with CPUs that are at least one major generation behind our Xserves.

Change History (4)

comment:1 Changed 4 years ago by mascguy (Christopher Nielsen)

After re-reviewing the various OpenMPI docs, configure arguments, articles, etc, it doesn't appear that binaries are hardware-dependent: The only real dependence is whether it's a 32-bit vs. 64-bit build. But MacPorts ensures that the appropriate binary - or build flags, if installing from source - is used for the user's machine. So it's not an issue.

I've also validated the buildbot-generated binaries, via several MPI test programs, across the following selection of openmpi-* ports. These were run on my 2008-era MacPro, to ensure the binaries run on older CPUs:

  • openmpi-default
  • openmpi-clang90
  • openmpi-clang10
  • openmpi-clang11
  • openmpi-gcc8
  • openmpi-gcc9
  • openmpi-gcc10

All work without issue.

The only hardware dependency that comes into play, occurs when running across a cluster of machines: If the CPU endiness is different between members, then OpenMPI must be compiled with flag --enable-heterogeneous to work properly. That ensures that any primitive data like integers and floating point values, are properly converted to the proper byte order.

All of this isn't an issue with Intel and ARM-based Macs, as both are little-endian. However, if the cluster contains members with big-endian architectures - like PowerPC - the '--enable-heterogeneous' option must be enabled.

Conclusion: While buildbot binaries are fine to use as-is, we should add a new variant hetereogeneous to support users who want to run as part of a heterogeneous cluster.

comment:2 Changed 4 years ago by mascguy (Christopher Nielsen)

Reviewing sibling port mpich, there is one case where buildbot binaries should not be used: When using the compiler provided by Xcode. That can vary between the buildbot and users' machines, so don't use binaries in that case.

So the following subports will not use a buildbot binary:

  • openmpi-default
  • openmpi-clang
  • openmpi-devel-default
  • openmpi-devel-clang

comment:3 Changed 4 years ago by mascguy (Christopher Nielsen)

PR request submitted to enable buildbot binary use, along with a new variant for heterogeneous functionality. Plus a new variant to cover legacy mpi1 support, requested by issue:59547.

PR 10852 - openmpi: update to 4.1.1; add new variants heterogeneous and mpi1; enable use of published binaries for all compiler-specific subports

Thoughts/comments welcome, as always.

Last edited 4 years ago by mascguy (Christopher Nielsen) (previous) (diff)

comment:4 Changed 4 years ago by Christopher Nielsen <mascguy@…>

Resolution: fixed
Status: assignedclosed

In ed9c96aae95bb1e0a0551f5d315a2b2be3925773/macports-ports (master):

openmpi: update to 4.1.1; add new variants heterogeneous and mpi1

  • Enable use of published binaries for all compiler-specific subports
  • Add lib dependency for zlib, and corresponding configure flag
  • Ensure target compilers are added to depends_lib

Fixes: #62795
Fixes: #62788
Fixes: #62783
Fixes: #62214
Fixes: #62070
Fixes: #60597
Fixes: #59547

Note: See TracTickets for help on using tickets.