New port: mumps 4.10.0 - a library for solving sparse linear systems

[wimmer@…]

Mumps is a library for solving sparse linear systems. All of the major Linux distributions include it (Debian, Redhat, Ubuntu, ...)

Mumps can be built both as a parallel or a sequential library. The parallel version needs scalapack which is as far as I've seen not included in macports yet. Hence, currently the protfile only builds the sequential version (Porting scalapack will take a little longer).

[cooljeanius (Eric Gallager)]

Just a quick note on the license, "{public domain}" parses to "either 'public' or 'domain'". You probably want "public-domain" (with a hypen instead of curly-braces) instead.

[larryv (Lawrence Velázquez)]

Comments:

• This looks like it should be in the “math” category, rather than the “science” one.
• Leave out “revision 0”, which is the default.
• Leave out “use_bzip2 no”, since Portfiles already default to fetching .tar.gz files.
• Leave out the “worksrcdir” line, since it already defaults to the value of “distname”.
• Leave out “build.target all”, which is the default.
• Add a SHA256 checksum.
• Add a variant for GCC 4.8, unless the software does not build correctly with it.
• Leave out the “depends_lib-append” lines for the compilers, since MacPorts 2.2 adds these automatically.
• Using “join” in the “build.env-append” line is redundant. Just substitute “configure.ldflags” directly.
• The reinplaces that merely swap one bit of static text for another bit of static text should be patches instead.

[seanfarley (Sean Farley)]

MUMPS is a bit of a train wreck of a package but I'll comment that I already have MUMPS and its dependents here: ​MUMPS, ​ScaLAPACK, and (the now deprecated) ​BLACS. I plan to add these ports once the multiplecompilers and mpi port groups are in trunk.

A word of warning: getting MUMPS to work in parallel requires ParMETIS and therefore a custom patch series (that I already ​wrote to the MUMPS team) to get it to work with the new version of METIS. Also, I have some custom patches to get shared libraries working.

[wimmer@…]

Replying to sean@…:

MUMPS is a bit of a train wreck of a package but I'll comment that I already have MUMPS and its dependents here: ​MUMPS, ​ScaLAPACK, and (the now deprecated) ​BLACS. I plan to add these ports once the multiplecompilers and mpi port groups are in trunk.

A word of warning: getting MUMPS to work in parallel requires ParMETIS and therefore a custom patch series (that I already ​wrote to the MUMPS team) to get it to work with the new version of METIS. Also, I have some custom patches to get shared libraries working.

As far as I've seen, you have the parallel version of Mumps in the portfile, right? I'm personally more interested in the sequential version anyways.

It is actually not quite clear to me, if a single build of Mumps can be used both in parallel or in sequential - in the end the difference is just that dummy mpi library libmpiseq. Still, in the FAQ they say that one must decide between a parallel or sequential installation, and also Debian has two distinct versions of the library - so this might seem the best strategy for macports, too.

May I inquire in how far the multiplecompilers and mpi port groups you mention are of importance to Mumps, sclapack, etc.?

As far as parmetis is concerned: Wouldn't it make sense anyways to keep a metis4 variant in macports? Many scentific programs use the old API, and although the changes are not big (i.e. patchable), it might be useful. Also, let me mention that I have good experiences with scotch and Mumps with scotch also already being in macports.

Well, I probably digressed too much ;) In any case, what should we do now? Should I go ahead with the sequential Mumps version or wait for you?

[seanfarley (Sean Farley)]

Replying to wimmer@…:

As far as I've seen, you have the parallel version of Mumps in the portfile, right? I'm personally more interested in the sequential version anyways.

They are the same.

It is actually not quite clear to me, if a single build of Mumps can be used both in parallel or in sequential - in the end the difference is just that dummy mpi library libmpiseq. Still, in the FAQ they say that one must decide between a parallel or sequential installation, and also Debian has two distinct versions of the library - so this might seem the best strategy for macports, too.

MUMPS only uses MPI code and loads a sequential version (libmpiseq) when using serial (as you note). I am against having two ports that do the same.

May I inquire in how far the multiplecompilers and mpi port groups you mention are of importance to Mumps, sclapack, etc.?

There is a discussion on the ​macports dev list about it that is (hopefully) winding down.

As far as parmetis is concerned: Wouldn't it make sense anyways to keep a metis4 variant in macports? Many scentific programs use the old API, and although the changes are not big (i.e. patchable), it might be useful.

The changes to get scientific programs using the new METIS 5 api (and 64 bit ints) is pretty trivial. I see no reason to keep an old version around since the patches to fix these packages (MUMPS, SuiteSparse, etc.) are small.

Also, let me mention that I have good experiences with scotch and Mumps with scotch also already being in macports.

Yes, I plan to unify all of this once the port groups are ironed out.

Well, I probably digressed too much ;) In any case, what should we do now? Should I go ahead with the sequential Mumps version or wait for you?

I would very much like to wait on the port groups so that I can finish integrating all the scientific ports I've had in my own repo for a while now.

[wimmer@…]

Sorry, was away for a few days, hence only now the reply (by the way - let me know if you think this discussion is not appropriate in the port submission ticket, although I think it is, as it's relevant for the port under consideration)

Replying to sean@…:

Replying to wimmer@…:

As far as I've seen, you have the parallel version of Mumps in the portfile, right? I'm personally more interested in the sequential version anyways.

They are the same.

It is actually not quite clear to me, if a single build of Mumps can be used both in parallel or in sequential - in the end the difference is just that dummy mpi library libmpiseq. Still, in the FAQ they say that one must decide between a parallel or sequential installation, and also Debian has two distinct versions of the library - so this might seem the best strategy for macports, too.

MUMPS only uses MPI code and loads a sequential version (libmpiseq) when using serial (as you note). I am against having two ports that do the same.

There's two things to note there:

• the sequential mpiseq has its own mpif.h, with definitions that differ from e.g. openmpi. From the Mumps source code it's hard to tell if there is some dependency on that particular mpif.h. Presumably not, but without the developers confirming this ... In any case one has to be careful which mpif.h to use.
• Mumps has the orderings it can use defined at compile-time (with -Dparmetis etc.). For a variant you'd want to use mostly in parallel, this means you want to compile it with the parallel parmetis and ptscotch. Now, assuming you can use the parallel version also in sequential mode by linking mpiseq, you still would have to link against parmetis and ptscotch although in the sequential version you wouldn't ever use it. Now, one could presumably fix this by adding dummy parmetis/ptscotch wrappers to mpiseq, but that's even more patching

What's so bad about having both a sequential and a parallel version installed (from the same Portfile)? This is how Debian does it, so it's kind of a standard (with the sequential version having a trailing _seq in the library name)

As far as parmetis is concerned: Wouldn't it make sense anyways to keep a metis4 variant in macports? Many scentific programs use the old API, and although the changes are not big (i.e. patchable), it might be useful.

The changes to get scientific programs using the new METIS 5 api (and 64 bit ints) is pretty trivial. I see no reason to keep an old version around since the patches to fix these packages (MUMPS, SuiteSparse, etc.) are small.

I was mostly thinking about people wanting to compile software that uses metis v4 - those wouldn't want to patch everything themselves. (even for macports, if you can avoid patching stuff, I think it's better, with every patch you can make a mistake).

[seanfarley (Sean Farley)]

Replying to wimmer@…:

There's two things to note there:

• the sequential mpiseq has its own mpif.h, with definitions that differ from e.g. openmpi. From the Mumps source code it's hard to tell if there is some dependency on that particular mpif.h. Presumably not, but without the developers confirming this ... In any case one has to be careful which mpif.h to use.

Using the correct mpi is always an issue. The goal of mpiuni is to be a drop-in replacement for mpi (with obvious differences, of course). This doesn't affect MUMPS that much, actually. We do this kind of thing all the time within PETSc (which has an optional dependency on MUMPS).

• Mumps has the orderings it can use defined at compile-time (with -Dparmetis etc.). For a variant you'd want to use mostly in parallel, this means you want to compile it with the parallel parmetis and ptscotch. Now, assuming you can use the parallel version also in sequential mode by linking mpiseq, you still would have to link against parmetis and ptscotch although in the sequential version you wouldn't ever use it. Now, one could presumably fix this by adding dummy parmetis/ptscotch wrappers to mpiseq, but that's even more patching

Dummy wrappers for parmet… what? You can compile all the optional orderings and decide on which one to use at run-time. You can also compile MUMPS for parallel and use its sequential algorithm at run-time. As homework, you can try it out:

• running mpi-mumps + parmetis with 'mpiexec -n 1 ./test_prog'
• running mpi-mumps + ptscotch with 'mpiexec -n 1 ./test_prog'
• running mpi-mumps + parmetis with 'mpiexec -n 2 ./test_prog'
• running mpi-mumps + ptscotch with 'mpiexec -n 2 ./test_prog'
• running mpi-mumps + internal ordering with 'mpiexec -n 1 ./test_prog'
• running mpi-mumps + internal ordering with 'mpiexec -n 2 ./test_prog'

And then try all of that again with sequentially built MUMPS.

What's so bad about having both a sequential and a parallel version installed (from the same Portfile)? This is how Debian does it, so it's kind of a standard (with the sequential version having a trailing _seq in the library name)

Well, for one, I don't like having an unnecessary port (sequential MUMPS) since all its functionality would be provided by the parallel version.

I was mostly thinking about people wanting to compile software that uses metis v4 - those wouldn't want to patch everything themselves. (even for macports, if you can avoid patching stuff, I think it's better, with every patch you can make a mistake).

Instead of a hypothetical situation, do you have any examples? I have updated most of the ports (in my side repo) to use the newest version of MeTiS / ParMeTiS but might have missed some. Since it's a very, very simple change (usually two lines of code change), I would rather push others to update their code.

[wimmer@…]

Replying to sean@…:

Replying to wimmer@…:

• Mumps has the orderings it can use defined at compile-time (with -Dparmetis etc.). For a variant you'd want to use mostly in parallel, this means you want to compile it with the parallel parmetis and ptscotch. Now, assuming you can use the parallel version also in sequential mode by linking mpiseq, you still would have to link against parmetis and ptscotch although in the sequential version you wouldn't ever use it. Now, one could presumably fix this by adding dummy parmetis/ptscotch wrappers to mpiseq, but that's even more patching

Dummy wrappers for parmet… what? You can compile all the optional orderings and decide on which one to use at run-time. You can also compile MUMPS for parallel and use its sequential algorithm at run-time. As homework, you can try it out:

• running mpi-mumps + parmetis with 'mpiexec -n 1 ./test_prog'
• running mpi-mumps + ptscotch with 'mpiexec -n 1 ./test_prog'
• running mpi-mumps + parmetis with 'mpiexec -n 2 ./test_prog'
• running mpi-mumps + ptscotch with 'mpiexec -n 2 ./test_prog'
• running mpi-mumps + internal ordering with 'mpiexec -n 1 ./test_prog'
• running mpi-mumps + internal ordering with 'mpiexec -n 2 ./test_prog'

And then try all of that again with sequentially built MUMPS.

Sure you can do that, but this wasn't my point. My point was that after you specified for which orderings MUMPS is built by specifying any combination of "-Dpord -Dmetis -Dparmetis -Dscotch -Dptscotch", you have to always link against all of these libraries.

Now, it is worth noting that the parmetis switch does not include all of the functionality of the metis switch, as MUMPS offers more functionality when only sequential analysis is used (see documentation of the ICNTL(28) parameter). Hence, one would like to include both parmetis and metis at the same time anyways (I noticed that your Portfile only had parmetis included). If a user would want to use the sequential version, there is not reason for him to use the parallel analysis at all (it gives no additional functionality, but even reduces it), but he/she would always need to link against parmetis/ptscotch.

This is, in my opinion, a good reason to have separate sequential and parallel versions.

That in addition to the fact that while I believe that it is probably possible to use the parallel version in sequential mode without mpi (by linking against mpiseq and dealing with different mpi's properly), the MUMPS web page does advocate separate sequential and parallel versions (​http://graal.ens-lyon.fr/MUMPS/index.php?page=faq#5).

What's so bad about having both a sequential and a parallel version installed (from the same Portfile)? This is how Debian does it, so it's kind of a standard (with the sequential version having a trailing _seq in the library name)

Well, for one, I don't like having an unnecessary port (sequential MUMPS) since all its functionality would be provided by the parallel version.

I was mostly thinking about people wanting to compile software that uses metis v4 - those wouldn't want to patch everything themselves. (even for macports, if you can avoid patching stuff, I think it's better, with every patch you can make a mistake).

Instead of a hypothetical situation, do you have any examples? I have updated most of the ports (in my side repo) to use the newest version of MeTiS / ParMeTiS but might have missed some. Since it's a very, very simple change (usually two lines of code change), I would rather push others to update their code.

I don't have a particular example (except MUMPS itself), but suppose I want to install versions of Mumps, SuiteSparse, SuperLU, ... not provided by macports - then I have to do the patching myself (or manually install metis 4). Metis 4 was the standard for more than 10 years when there was no development there; scientific codes are usually conservative in that you don't want to touch what works, so people do not easily change to Metis 5 (look at Mumps and SuiteSparse).

[seanfarley (Sean Farley)]

Replying to wimmer@…:

Now, it is worth noting that the parmetis switch does not include all of the functionality of the metis switch, as MUMPS offers more functionality when only sequential analysis is used (see documentation of the ICNTL(28) parameter). Hence, one would like to include both parmetis and metis at the same time anyways (I noticed that your Portfile only had parmetis included). If a user would want to use the sequential version, there is not reason for him to use the parallel analysis at all (it gives no additional functionality, but even reduces it), but he/she would always need to link against parmetis/ptscotch.

This is, in my opinion, a good reason to have separate sequential and parallel versions.

That in addition to the fact that while I believe that it is probably possible to use the parallel version in sequential mode without mpi (by linking against mpiseq and dealing with different mpi's properly), the MUMPS web page does advocate separate sequential and parallel versions (​http://graal.ens-lyon.fr/MUMPS/index.php?page=faq#5).

[snip]

I don't have a particular example (except MUMPS itself), but suppose I want to install versions of Mumps, SuiteSparse, SuperLU, ... not provided by macports - then I have to do the patching myself (or manually install metis 4). Metis 4 was the standard for more than 10 years when there was no development there; scientific codes are usually conservative in that you don't want to touch what works, so people do not easily change to Metis 5 (look at Mumps and SuiteSparse).

Yes, I am very familiar with the community not wanting to update their own code (even when given the patches!) from being on the PETSc team. Barry has a way of convincing others to not pander to lazy scientists so I don't think it's very persuasive to (from what I can understand) not want to have a longer link line and especially use software outside the macports tree (which is not supported nor stable).

[seanfarley (Sean Farley)]

Fixed in r116391.