OpenCoarrays @1.0

[fanfarillo.gcc@…]

Open-source software project for developing, porting and tuning transport layers that support coarray Fortran compilers. The version provided is based on MPI-3.0 (requires mpich) and it is currently used by GNU Fortran (GCC 5).

[ryandesign (Ryan Carsten Schmidt)]

This port is a prerequisite for #47426.

The use of use_configure no is a cause for concern, because it usually implies that the port is not UsingTheRightCompiler or -arch flags and doesn't have a universal variant so these things have to be done manually in the port. If you're using fsf gcc then that further complicates things, but I'm not sure if you are using gcc, since you've declared a dependency on gcc5 but haven't given any configure arguments or environment variables telling the build to use it.

[rouson (Damian Rouson)]

Replying to ryandesign@…:

This port is a prerequisite for #47426.

The use of use_configure no is a cause for concern, because it usually implies that the port is not UsingTheRightCompiler or -arch flags and doesn't have a universal variant so these things have to be done manually in the port. If you're using fsf gcc then that further complicates things, but I'm not sure if you are using gcc, since you've declared a dependency on gcc5 but haven't given any configure arguments or environment variables telling the build to use it.

I have had a very hard time figuring out the appropriate dependencies so I'm hoping you can help sort this out:

1. Building gcc5 does not require the OpenCoarrays libcaf_mpi.a library, but using gcc5 to build parallel executables from coarray Fortran codes requires linking to libcaf_mpi.a
2. Building libcaf_mpi.a does not require gcc5 (earlier versions of gcc5 can build libcaf_mpi.a), but libcaf_mpi.a will be unusable if mpich-gcc5 is not present.

so either can be built without the other, but each is of limited utility without the utility without the other. Should gcc5 depend on OpenCoarrays or should OpenCoarrays depend on gcc5 or neither?

Damian

[petrrr]

Cc Me!

[cooljeanius (Eric Gallager)]

Cc Me!

[fanfarillo.gcc@…]

Replying to damian@…:

Replying to ryandesign@…:

This port is a prerequisite for #47426.

The use of use_configure no is a cause for concern, because it usually implies that the port is not UsingTheRightCompiler or -arch flags and doesn't have a universal variant so these things have to be done manually in the port. If you're using fsf gcc then that further complicates things, but I'm not sure if you are using gcc, since you've declared a dependency on gcc5 but haven't given any configure arguments or environment variables telling the build to use it.

I have had a very hard time figuring out the appropriate dependencies so I'm hoping you can help sort this out:

1. Building gcc5 does not require the OpenCoarrays libcaf_mpi.a library, but using gcc5 to build parallel executables from coarray Fortran codes requires linking to libcaf_mpi.a
2. Building libcaf_mpi.a does not require gcc5 (earlier versions of gcc5 can build libcaf_mpi.a), but libcaf_mpi.a will be unusable if mpich-gcc5 is not present.

so either can be built without the other, but each is of limited utility without the utility without the other. Should gcc5 depend on OpenCoarrays or should OpenCoarrays depend on gcc5 or neither?

Damian

Starting from a clean system, without gcc5, I expect the user to install gcc5 first. Then, if he/she is interested also in coarray support, will install opencoarrays. This will probably download mpich-devel-gcc5 if not already installed.

I just tested the portfile locally and all the actions I described ran exactly as expected. The opencoarrays library has been saved in /opt/local/lib. I also tried to compile and run a coarray program after the installation process:

mpif90 -fcoarray=lib testCoarray.f90 -L/opt/local/lib -lcaf_mpi -o test

compiles without any problem

$ mpirun -np 4 ./test

 Hello world, I am           1
           1 prev           0 next           2
 Hello world, I am           2
           2 prev           1 next           3
 Hello world, I am           3
           3 prev           2 next           0
 Hello world, I am           4
           4 prev           3 next           1

which means the program runs correctly too.

[rouson (Damian Rouson)]

Could someone please let Alessandro know how to get this port accepted into Macports? We would very much like to notify OpenCoarrays users of the option to install via Macports.

[rouson (Damian Rouson)]

Cc Me!

[rouson (Damian Rouson)]

* PING *

Traffic is picking up on the OpenCoarrays git repository: 60 unique visitors and 14 unique cloners in the past two weeks. Unfortunately, not all users are comfortable with git, CMake, and building software from source so we would really like to offer installation via package management software such as MacPorts. Could someone please reply to Alesandro about how to get his port accepted as part of MacPorts?

[rouson (Damian Rouson)]

Replying to ryandesign@…:

This port is a prerequisite for #47426.

The use of use_configure no is a cause for concern, because it usually implies that the port is not UsingTheRightCompiler or -arch flags and doesn't have a universal variant so these things have to be done manually in the port. If you're using fsf gcc then that further complicates things, but I'm not sure if you are using gcc, since you've declared a dependency on gcc5 but haven't given any configure arguments or environment variables telling the build to use it.

In hopes of responding more directly to your question, gcc5 is required for _using_ OpenCoarrays but it is not required for _building_ OpenCoarrays. Therefore, the depency on gcc5 exists because we assume someone building OpenCoarrays also wants to use OpenCoarrays. However, there is no need for configure arguments or environment variables to tell the build to use gcc5 because earlier versions of gcc are perfectly capable of building OpenCoarrays. We just want to make sure the user has gcc5 installed before they use OpenCoarrays so the easiest way to do that is to make sure it gets installed before they build OpenCoarrays.

[rouson (Damian Rouson)]

* PING *

Could we please get a response regarding this proposed port?

[ryandesign (Ryan Carsten Schmidt)]

I am looking into this now. I did not see your other messages before because I was not Cc'd on the ticket, but I'll Cc myself now since nobody else appears to be working on it.

The INSTALL file says cmake is the preferred build system here. Any reason why you didn't use that in the Portfile?

We still haven't addressed the fact that we're not UsingTheRightCompiler and -arch flags here. Currently it tries to build with "gcc" (which could be anything on a user's system; we want RepeatableBuilds) and "mpicc" (which will not exist on user systems unless they have used "port select", and MacPorts ports are not allowed to use the results of "port select" in their own builds (and, with trace mode, would be prohibited from doing so)).

So opencoarrays appears to require mpich in order to build. You've written the dependency to be on mpich-devel-gcc5, which depends on gcc5. Therefore we would not be able to accommodate #47426 which requests that opencoarrays be added as a dependency of gcc5, because that would introduce a circular dependency, which is not possible in MacPorts.

Any reason why you used mpich-devel-gcc5 (the potentially unstable development version) and not mpich-gcc5 (the stable version)? If either would work, we should use a path:-style dependency (instead of a port:-style one) to allow the user to choose between the development or stable version, with preference for the stable one.

The port claims to download version 1.0 of the software, but the opencoarrays web site does not mention such a version. It just refers users to the github page to download. The github page doesn't have any tags. If this is really version 1.0, why isn't there a 1.0 tag in the repository? If the URL listed in the portfile's master_sites is an official download URL, why isn't it mentioned anywhere on the opencoarrays web site?

I tried adding a universal variant but it failed because the mpich dependency doesn't have a universal variant.

[Ionic (Mihai Moldovan)]

Please understand that this is a complicated situation. For instance, I have no idea how this port is related to gcc5, for instance, and have never used mpich.

I can only reiterate ryans concerns: this port is installing libraries and using use_configure no -- thus, it will need to:

variant universal {}

# Do something to add [get_canonical_archflags cc] and [get_canonical_archflags ld] to CFLAGS/LDFLAGS.
# Because you're not using configure, you're probably required to add it to build.env,
# but I don't know your build system and if it accepts environment variables.
# There's probably also something like this for Fortran, but I have never used Fortran and genuinely no idea how this works...

[ryandesign (Ryan Carsten Schmidt)]

Replying to ionic@…:

variant universal {}

Can't: mpich-devel-gcc5 does not have a universal variant. (Unless that's not required? Maybe mpich-gcc5-devel needs installs_libs no? Or maybe opencoarrays needs depends_skip_archcheck mpich-devel-gcc5?)

# Do something to add [get_canonical_archflags cc] and [get_canonical_archflags ld] to CFLAGS/LDFLAGS.
# Because you're not using configure, you're probably required to add it to build.env,
# but I don't know your build system and if it accepts environment variables.
# There's probably also something like this for Fortran, but I have never used Fortran and genuinely no idea how this works...

I have this done locally already, but am holding off on committing pending answers to other questions. Maybe using cmake will handle it for us.

[Ionic (Mihai Moldovan)]

Replying to ryandesign@…:

I tried adding a universal variant but it failed because the mpich dependency doesn't have a universal variant.

Can't: mpich-devel-gcc5 does not have a universal variant. (Unless that's not required? Maybe mpich-gcc5-devel needs installs_libs no? Or maybe opencoarrays needs depends_skip_archcheck mpich-devel-gcc5?)

Oh, even more fun... I don't know, I don't have any of these ports installed. There's a chance they only install binaries, but I wouldn't bet on that.

[rouson (Damian Rouson)]

Replying to ryandesign@…:

I am looking into this now. I did not see your other messages before because I was not Cc'd on the ticket, but I'll Cc myself now since nobody else appears to be working on it.

Thank you!

The INSTALL file says cmake is the preferred build system here. Any reason why you didn't use that in the Portfile?

Two reasons:

1. We didn't want to add another dependency (on CMake) if there is a way to build with out it.
2. The Portfile was developed by Alessandro, who maintains the Makefile, while I maintain the CMake system. I have not looked into any details of the contents of the Portfile and did not ask Alessandro to learn the CMake set up so the separation of duties led to the choice of Make over CMake for the Portfile.

We still haven't addressed the fact that we're not UsingTheRightCompiler and -arch flags here. Currently it tries to build with "gcc" (which could be anything on a user's system; we want RepeatableBuilds) and "mpicc" (which will not exist on user systems unless they have used "port select", and MacPorts ports are not allowed to use the results of "port select" in their own builds (and, with trace mode, would be prohibited from doing so)).

Alessandro is new to writing a Portfile and I have no experience with it either. If you have a suggestion to address this, please feel free edit the Portfile accordingly.

So opencoarrays appears to require mpich in order to build. You've written the dependency to be on mpich-devel-gcc5, which depends on gcc5. Therefore we would not be able to accommodate #47426 which requests that opencoarrays be added as a dependency of gcc5, because that would introduce a circular dependency, which is not possible in MacPorts.

I am glad to rescind #47426 and will look into whether I can do so.

Any reason why you used mpich-devel-gcc5 (the potentially unstable development version) and not mpich-gcc5 (the stable version)? If either would work, we should use a path:-style dependency (instead of a port:-style one) to allow the user to choose between the development or stable version, with preference for the stable one.

That was a mistake on my part. I believe either will work.

The port claims to download version 1.0 of the software, but the opencoarrays web site does not mention such a version. It just refers users to the github page to download. The github page doesn't have any tags. If this is really version 1.0, why isn't there a 1.0 tag in the repository? If the URL listed in the portfile's master_sites is an official download URL, why isn't it mentioned anywhere on the opencoarrays web site?

Alessandro has been planning to post a version 1.0. He is most likely going to be offline and away from work for the next couple of days, but I'll ask him to respond to these questions as soon as he returns. There is a chance he and I will talk tomorrow. For now, just pushed a commit to github that adds a "VERSION 1.0" argument to "project" statement in the top-level CMakeLists.txt file. This in combination with the subsequent and previously existing CMake write_basic_package_version_file command causes CMake to write package version information in a subdirectory of the "lib" installation directory. If there is more that we should, please explain. In particular, I'm not sure how to what is meant by the phrase "a 1.0 tag in the repository." Is this tag to appear in the README file or in a separate file or some other way?

I tried adding a universal variant but it failed because the mpich dependency doesn't have a universal variant.

I have no clue what this means -- hopefully Alessandro will -- but if you can provide more details, please do. Please keep in mind that neither of us has any experience with writing a Portfile. Andy advice is welcome.

[fanfarillo.gcc@…]

Replying to ryandesign@…:

I am looking into this now. I did not see your other messages before because I was not Cc'd on the ticket, but I'll Cc myself now since nobody else appears to be working on it.

The INSTALL file says cmake is the preferred build system here. Any reason why you didn't use that in the Portfile?

The CMake support has been added after I proposed the Portfile.

We still haven't addressed the fact that we're not UsingTheRightCompiler and -arch flags here. Currently it tries to build with "gcc" (which could be anything on a user's system; we want RepeatableBuilds) and "mpicc" (which will not exist on user systems unless they have used "port select", and MacPorts ports are not allowed to use the results of "port select" in their own builds (and, with trace mode, would be prohibited from doing so)).

I'll check this tomorrow. Just to be clear, opencoarrays does not need gcc5 to build. Gcc5 does not need opencoarrays to compile a coarray Fortran code which uses carries. Gcc5 needs opencoarrays to create the executable of a coarray Fortran code (actually it needs opencoarray only for the multi-image coarray code).

So opencoarrays appears to require mpich in order to build. You've written the dependency to be on mpich-devel-gcc5, which depends on gcc5. Therefore we would not be able to accommodate #47426 which requests that opencoarrays be added as a dependency of gcc5, because that would introduce a circular dependency, which is not possible in MacPorts.

Any reason why you used mpich-devel-gcc5 (the potentially unstable development version) and not mpich-gcc5 (the stable version)? If either would work, we should use a path:-style dependency (instead of a port:-style one) to allow the user to choose between the development or stable version, with preference for the stable one.

We would like to keep the installation process as easy as possible (asking for a choice sounds less easy to me). Let's just use mpich-gcc5.

The port claims to download version 1.0 of the software, but the opencoarrays web site does not mention such a version. It just refers users to the github page to download. The github page doesn't have any tags. If this is really version 1.0, why isn't there a 1.0 tag in the repository? If the URL listed in the portfile's master_sites is an official download URL, why isn't it mentioned anywhere on the opencoarrays web site?

Good questions. The tarball currently available from macport was a candidate version now obsolete. The plan was to release OpenCoarrays 1.0 as soon as this port was accepted. We still plan to do that; I should be able to work on the port tomorrow, after that, if all the changes will be accepted we can post the tarball of OpenCoarrays 1.0 on opencoarrays.org.

I tried adding a universal variant but it failed because the mpich dependency doesn't have a universal variant.

[petrrr]

Well to me it sounds like gcc5 (and maybe mpich-*gcc5) are no real dependencies, in particular not lib dependencies. It may be reasonable to have them as runtime dependencies, as otherwise the whole port is useless.

I just wonder what exactly are the dependencies of this port? I understand that you need MPICH, but which version of MPICH whould you pick? Does it make any sense to link build against any mpich for a different compiler other then gcc5? I guess it does not. So this would make mpich-gcc5 or mpich-devel-gcc5, maybe mpich-*gcc6 the set of possible mpich dependencies.

[fanfarillo.gcc@…]

Replying to petr@…:

Well to me it sounds like gcc5 (and maybe mpich-*gcc5) are no real dependencies, in particular not lib dependencies. It may be reasonable to have them as runtime dependencies, as otherwise the whole port is useless.

I just wonder what exactly are the dependencies of this port? I understand that you need MPICH, but which version of MPICH whould you pick? Does it make any sense to link build against any mpich for a different compiler other then gcc5? I guess it does not. So this would make mpich-gcc5 or mpich-devel-gcc5, maybe mpich-*gcc6 the set of possible mpich dependencies.

It might make sense for testing purposes, like compiling the source Fortran file with gcc5 and use intelmpi as transport layer (which is compiled for Intel). Anyway, the most common and useful case is the one with mpich compiled for gcc5. Gcc5 is currently the only compiler using opencoarrays, without it opencoarrays is useless.

[petrrr]

Gcc5 is currently the only compiler using opencoarrays, without it opencoarrays is useless.

What about gcc6? doesn't it support opencoarrys as well? Or is it that the current coarrays lib does not support gcc6?

[fanfarillo.gcc@…]

Replying to petr@…:

Gcc5 is currently the only compiler using opencoarrays, without it opencoarrays is useless.

What about gcc6? doesn't it support opencoarrys as well? Or is it that the current coarrays lib does not support gcc6?

gcc6 is even better. It contains some patches related with coarrays.

[rouson (Damian Rouson)]

Replying to fanfarillo.gcc@…:

Replying to petr@…:

It might make sense for testing purposes, like compiling the source Fortran file with gcc5 and use intelmpi as transport layer (which is compiled for Intel). Anyway, the most common and useful case is the one with mpich compiled for gcc5. Gcc5 is currently the only compiler using opencoarrays, without it opencoarrays is useless.

Not useless. Just so everyone knows, we are working on a version of OpenCoarrays that will work with any Fortran compiler and there has already been some user interest in this option on the OpenCoarrays mailing list. The syntax users get with gcc is nicer for many use cases, but there are even some use cases where the syntax of the user code will be identical whether the user is compiling with gcc or another compiler. In fact, I just published a journal article that studies an application for which the source would be identical whether using OpenCoarrays with gcc or another compiler and just today I found out about another application for which the same statement is true. There will be a strong preference for gcc for the foreseeable future, but I expect other compilers to become a reasonable option soon.

Damian

[petrrr]

Replying to damian@…:

Not useless. Just so everyone knows, we are working on a version of OpenCoarrays that will work with any Fortran compiler and there has already been some user interest in this option on the OpenCoarrays mailing list. The syntax users get with gcc is nicer for many use cases, but there are even some use cases where the syntax of the user code will be identical whether the user is compiling with gcc or another compiler. In fact, I just published a journal article that studies an application for which the source would be identical whether using OpenCoarrays with gcc or another compiler and just today I found out about another application for which the same statement is true. There will be a strong preference for gcc for the foreseeable future, but I expect other compilers to become a reasonable option soon.

Okay, so it is just an development library as any other. So what are the exact requirements to compile this piece of software?

[larryv (Lawrence Velázquez)]

Ryan, have you worked on this any further? I can take over, if not.

[ryandesign (Ryan Carsten Schmidt)]

Replying to damian@…:

The port claims to download version 1.0 of the software, but the opencoarrays web site does not mention such a version. It just refers users to the github page to download. The github page doesn't have any tags. If this is really version 1.0, why isn't there a 1.0 tag in the repository? If the URL listed in the portfile's master_sites is an official download URL, why isn't it mentioned anywhere on the opencoarrays web site?

Alessandro has been planning to post a version 1.0. He is most likely going to be offline and away from work for the next couple of days, but I'll ask him to respond to these questions as soon as he returns. There is a chance he and I will talk tomorrow. For now, just pushed a commit to github that adds a "VERSION 1.0" argument to "project" statement in the top-level CMakeLists.txt file. This in combination with the subsequent and previously existing CMake write_basic_package_version_file command causes CMake to write package version information in a subdirectory of the "lib" installation directory. If there is more that we should, please explain. In particular, I'm not sure how to what is meant by the phrase "a 1.0 tag in the repository." Is this tag to appear in the README file or in a separate file or some other way?

I mean: a tag in your git repository: ​http://git-scm.com/book/en/v2/Git-Basics-Tagging

I tried adding a universal variant but it failed because the mpich dependency doesn't have a universal variant.

I have no clue what this means -- hopefully Alessandro will -- but if you can provide more details, please do. Please keep in mind that neither of us has any experience with writing a Portfile. Andy advice is welcome.

A universal build is one that contains more than one architecture. For example, a non-universal build might contain code built for x86_64 only, while a universal build might contain code built for both x86_64 and i386. MacPorts implements universal builds via the variant mechanism. If we can add a universal variant to opencoarrays that would be great, but we may first need to address the lack of a universal variant in the mpich ports.

Replying to fanfarillo.gcc@…:

The CMake support has been added after I proposed the Portfile.

Maybe we should investigate using it, since the project documentation says it is the recommended way. If so, the cmake 1.0 portgroup should be used.

Just to be clear, opencoarrays does not need gcc5 to build.

I understand that. But your build script is currently trying to build with "gcc" (which is not allowed for the reasons explained in the UsingTheRightCompiler wiki page) and "mpicc" (which does not exist, which causes the build to fail with a "file not found" error for "mpicc").

We would like to keep the installation process as easy as possible (asking for a choice sounds less easy to me). Let's just use mpich-gcc5.

When -devel ports are used correctly, installation when -devel ports are available is no more difficult than when they are not available, because MacPorts will automatically choose the non-devel version of a dependency unless the user has specifically requested the -devel version by installing it first.

Unfortunately, the mpich -devel ports were not designed correctly, so we have no choice but to use either the -devel version or the non-devel version, until #48086 is resolved. In light of that, I agree, let's use mpich-gcc5 for now.

The tarball currently available from macport was a candidate version now obsolete. The plan was to release OpenCoarrays 1.0 as soon as this port was accepted. We still plan to do that; I should be able to work on the port tomorrow, after that, if all the changes will be accepted we can post the tarball of OpenCoarrays 1.0 on opencoarrays.org.

The correct order of operations would be that you release a version of your software, then we add it to MacPorts, not the other way around.

Replying to fanfarillo.gcc@…:

gcc6 is even better. It contains some patches related with coarrays.

Then the opencoarrays port should offer gcc5 and gcc6 variants, with gcc5 being the default until gcc6 becomes stable.

Replying to larryv@…:

Ryan, have you worked on this any further? I can take over, if not.

I'll attached my work in progress and hand it over to you at this point, Larry.

[fanfarillo.gcc@…]

Replying to ryandesign@…:

I'll attached my work in progress and hand it over to you at this point, Larry.

Reading your message on the email I missed the attachment. I modified the tarball on the website and produced the new attached Portfile (I merged your changes too).

It works for me locally; did I miss anything?

Thanks for the suggestions.

[petrrr]

I would strongly advice you to avoid stealth updates at any point of the process. This is very bad practice as you put into circulation tarballs, which appear to have the same release version, but have different content or checksum. In particular, this creates issues with packaging systems (like MacPorts) and mirrors. So please always bump the version part of the name when changing anything.

In this concrete case, where the the packages is not yet released, it is probably indicated to go for a -devel port first, and provide the stable port only after release of the software. I usually test release candidates with the -devel port as well, because any rc postfix would require to bump the epoch later.

[ryandesign (Ryan Carsten Schmidt)]

Replying to fanfarillo.gcc@…:

Reading your message on the email I missed the attachment. I modified the tarball on the website and produced the new attached Portfile (I merged your changes too).

It works for me locally; did I miss anything?

I'll let Larry handle this port from here, but yes, your new portfile has several issues:

• You reintroduced tabs into the portfile; only spaces should be used.
• You re-added gcc5 as a library dependency, though it does not appear to be used.
• You changed mpich-gcc5 back to a library dependency, though it only appears to be needed at build time.
• You changed configure.cc from the default clang to mpicc-mpich-gcc5, but clang had worked for me. Why change it? If changing it is important, use the absolute path.
• You changed configure.cxx from the default clang++ to mpicxx-mpich-gcc5, but this project's Makefiles don't seem to use $(CXX) so this is irrelevant.
• You changed configure.cpp from the default clang -E to mpic++-mpich-gcc5, which is a symlink to mpicxx-mpich-gcc5 and is a conceptual mismatch. configure.cpp is meant to be the C preprocessor, not the C++ compiler, but again it's irrelevant because the Makefiles don't appear to use $(CPP).
• You used set to set options, which don't need set to set them. (set is only used for variables, not options.)
• You removed the MPICC build argument I had set, thereby reintroducing the problem that setting it had fixed: namely, that the Makefile tries to run mpicc which does not exist.

[fanfarillo.gcc@…]

Replying to ryandesign@…:

Replying to fanfarillo.gcc@…:

Reading your message on the email I missed the attachment. I modified the tarball on the website and produced the new attached Portfile (I merged your changes too).

It works for me locally; did I miss anything?

I'll let Larry handle this port from here, but yes, your new portfile has several issues:

• You reintroduced tabs into the portfile; only spaces should be used.
• You re-added gcc5 as a library dependency, though it does not appear to be used.
• You changed mpich-gcc5 back to a library dependency, though it only appears to be needed at build time.

Yes, Damian explained what we plan to do with compilers different from gcc5. Currently, if a user wants to use opencoarrays has to use gcc5 or gcc6. Declaring gcc5/gcc6 and mpich-gcc5 as dependencies allows the users to download all the stuff needed to run a coarray program. It is just an easy way to get all you need with a simple "sudo port install opencoarrays". As Damian said, I am pretty new of the Mac and MacPorts world so, if my reasoning violates rules about dependencies, I'll be glad to find a different way to do it.

• You changed configure.cc from the default clang to mpicc-mpich-gcc5, but clang had worked for me. Why change it? If changing it is important, use the absolute path.
• You changed configure.cxx from the default clang++ to mpicxx-mpich-gcc5, but this project's Makefiles don't seem to use $(CXX) so this is irrelevant.
• You changed configure.cpp from the default clang -E to mpic++-mpich-gcc5, which is a symlink to mpicxx-mpich-gcc5 and is a conceptual mismatch. configure.cpp is meant to be the C preprocessor, not the C++ compiler, but again it's irrelevant because the Makefiles don't appear to use $(CPP).

I just read the instruction on UsingTheRightCompiler and forced to use mpich for all the building process, I noticed the new Portfile when I was about to upload mine...

• You used set to set options, which don't need set to set them. (set is only used for variables, not options.)
• You removed the MPICC build argument I had set, thereby reintroducing the problem that setting it had fixed: namely, that the Makefile tries to run mpicc which does not exist.

I modified the Makefile in the library in order to manage this new configuration. The library is composed by two .c files, one needs mpicc and the other one doesn't. Currently I compile both files with mpicc-mpich-gcc5.

[larryv (Lawrence Velázquez)]

Here’s my portfile​. Please test it and let me know if it’s satisfactory. Note that the additional patch​ needs to go into a “files” directory underneath the OpenCoarrays directory.

• I’ve renamed it to “OpenCoarrays”.
• I’ve moved it to the science category. This is in line with related ports, like the MPICH and OpenMPI ones.
• It uses CMake and the cmake-1.0 portgroup. This obviates the need to manually pass compiler flags to the build phase.
• It currently fetches from GitHub using the github-1.0 portgroup. This is easily changed if you end up serving your distfiles from elsewhere.
• It uses the mpi-1.0 portgroup to handle the dirty business of selecting compilers and using all the correct executables. The port can be compiled using mpich-gcc5, mpich-devel-gcc5, mpich-gcc6, or mpich-devel-gcc6 by selecting the appropriate combination of variants, which you can view with port variants OpenCoarrays. (Once #48091 is closed, the OpenMPI variants should work as well.)

  % sudo port install OpenCoarrays                    # Use MPICH 3.1.4 + GCC 5
  % sudo port install OpenCoarrays +gcc6              # Use MPICH 3.1.4 + GCC 6
  % sudo port install OpenCoarrays +mpich_devel       # Use MPICH 3.2b3 + GCC 5
  % sudo port install OpenCoarrays +gcc6 +mpich_devel # Use MPICH 3.2b3 + GCC 6
  
  % port variants OpenCoarrays
  OpenCoarrays has the variants:
     debug: Enable debug binaries
  [+]gcc5: Build using the MacPorts gcc 5 compiler
       * conflicts with gcc6
     gcc6: Build using the MacPorts gcc 6 compiler
       * conflicts with gcc5
  [+]mpich: Build using the MPICH compiler
       * conflicts with mpich_devel openmpi openmpi_devel
     mpich_devel: Build using the MPICH-devel compiler
       * conflicts with mpich openmpi openmpi_devel
     openmpi: Build using the OpenMPI compiler
       * conflicts with mpich mpich_devel openmpi_devel
     openmpi_devel: Build using the OpenMPI-devel compiler
       * conflicts with mpich mpich_devel openmpi
  

• It patches src/tests/unit/simple/CMakeLists.txt to prevent it from using the nonexistent "mpicc".
• It enables the test phase, which you can run by executing sudo port test OpenCoarrays.
• It installs both the single and MPI libraries, as well as opencoarrays.mod. Note that we cannot install to ${prefix}/mod because that would violate our directory hierarchy; I’ve used ${prefix}/include instead, as some other Fortran ports do.

There are still a couple of wrinkles, but I don’t think they are blockers.

• The mpi-1.0 portgroup creates a library dependency on the selected MPI port, while OpenCoarrays really just needs a build dependency. The portgroup is far too convenient overall for this to be a dealbreaker, but ideally it would be improved to let client ports declare that they only need MPI for building.
• The MPI ports do not have universal variants; thus, mpi-1.0 does not allow client ports to have both MPI variants and a universal variant.

[fanfarillo.gcc@…]

Thanks for the new Portfile. Unfortunately, I've got some problems:

1. The checksums are incorrect; the following seems to fix the issue (my local Portfile points to the latest version on github):

checksums           tests-compiler.patch \
                    rmd160  a84fd458052aaa178c0f6c6cc47bd18a6027b3ab \
                    sha256  4394d381e62b5517f301ffff0b1b075221ed75021653f41d2b625f002f4ebc53 \
                    opencoarrays-8301325f8e0a2d4a8f45f32d5e45dbb81b332e6a.tar.gz \
                    rmd160  a84fd458052aaa178c0f6c6cc47bd18a6027b3ab \
                    sha256  4394d381e62b5517f301ffff0b1b075221ed75021653f41d2b625f002f4ebc53

2. when the script tries to apply the patch I get the following error:

--->  Applying patches to OpenCoarrays
Error: org.macports.patch for port OpenCoarrays returned: command execution failed

In the log file there are the following statement before the error:

:debug:patch Assembled command: 'cd "/opt/local/var/macports/build/_Users_alessandrofanfarillo_ports_science_opencoarrays/OpenCoarrays/work/opencoarrays-8301325f8e0a2d4a8f45f32d5e45dbb81bdbb81b332e6a" && /usr/bin/patch -p0'
:debug:patch Executing command line:  cd "/opt/local/var/macports/build/_Users_alessandrofanfarillo_ports_science_opencoarrays/OpenCoarrays/work/opencoarrays-8301325f8e0a2d4a8f45f32d5e45dd5e45dbb81b332e6a" && /usr/bin/patch -p0 < '/opt/local/var/macports/distfiles/OpenCoarrays/tests-compiler.patch'

If I go inside the directory /opt/local/var/macports/distfiles/OpenCoarrays/ I see only the tests-compiler.patch and the compressed directory of OpenCoarrays: opencoarrays-8301325f8e0a2d4a8f45f32d5e45dbb81b332e6a.tar.gz

Note that the latest version on the repository has the file tests-compiler.patch inside the files directory and in the main directory of OpenCoarrays.

Am I doing something wrong?

Replying to larryv@…:

Here’s my portfile​. Please test it and let me know if it’s satisfactory. Note that the additional patch​ needs to go into a “files” directory underneath the OpenCoarrays directory.

• I’ve renamed it to “OpenCoarrays”.
• I’ve moved it to the science category. This is in line with related ports, like the MPICH and OpenMPI ones.
• It uses CMake and the cmake-1.0 portgroup. This obviates the need to manually pass compiler flags to the build phase.
• It currently fetches from GitHub using the github-1.0 portgroup. This is easily changed if you end up serving your distfiles from elsewhere.
• It uses the mpi-1.0 portgroup to handle the dirty business of selecting compilers and using all the correct executables. The port can be compiled using mpich-gcc5, mpich-devel-gcc5, mpich-gcc6, or mpich-devel-gcc6 by selecting the appropriate combination of variants, which you can view with port variants OpenCoarrays. (Once #48091 is closed, the OpenMPI variants should work as well.)

  % sudo port install OpenCoarrays                    # Use MPICH 3.1.4 + GCC 5
  % sudo port install OpenCoarrays +gcc6              # Use MPICH 3.1.4 + GCC 6
  % sudo port install OpenCoarrays +mpich_devel       # Use MPICH 3.2b3 + GCC 5
  % sudo port install OpenCoarrays +gcc6 +mpich_devel # Use MPICH 3.2b3 + GCC 6
  
  % port variants OpenCoarrays
  OpenCoarrays has the variants:
     debug: Enable debug binaries
  [+]gcc5: Build using the MacPorts gcc 5 compiler
       * conflicts with gcc6
     gcc6: Build using the MacPorts gcc 6 compiler
       * conflicts with gcc5
  [+]mpich: Build using the MPICH compiler
       * conflicts with mpich_devel openmpi openmpi_devel
     mpich_devel: Build using the MPICH-devel compiler
       * conflicts with mpich openmpi openmpi_devel
     openmpi: Build using the OpenMPI compiler
       * conflicts with mpich mpich_devel openmpi_devel
     openmpi_devel: Build using the OpenMPI-devel compiler
       * conflicts with mpich mpich_devel openmpi
  

• It patches src/tests/unit/simple/CMakeLists.txt to prevent it from using the nonexistent "mpicc".
• It enables the test phase, which you can run by executing sudo port test OpenCoarrays.
• It installs both the single and MPI libraries, as well as opencoarrays.mod. Note that we cannot install to ${prefix}/mod because that would violate our directory hierarchy; I’ve used ${prefix}/include instead, as some other Fortran ports do.

There are still a couple of wrinkles, but I don’t think they are blockers.

• The mpi-1.0 portgroup creates a library dependency on the selected MPI port, while OpenCoarrays really just needs a build dependency. The portgroup is far too convenient overall for this to be a dealbreaker, but ideally it would be improved to let client ports declare that they only need MPI for building.
• The MPI ports do not have universal variants; thus, mpi-1.0 does not allow client ports to have both MPI variants and a universal variant.

[larryv (Lawrence Velázquez)]

Please attach the log. You should not be needing checksums for the patch; are you sure you placed it in $(port dir OpenCoarrays)/files?

[petrrr]

@fanfarillo.gcc: You need to place the patch file into the files directory of the port. No checksum needed because it is supplied not downloaded.

Otherwise, if you want to base the port on a more recent commit

• update the hash of in github.setup;
• update the checksums;
• remove the patchfiles line if it is not needed any more;

[fanfarillo.gcc@…]

I didn't get that OpenCoarrays directory referred to the portfile directory. Larry's portfile works fine for me.

Tomorrow Damian and I will talk and probably release OpenCoarrays-1.0. I'll post the new Portfile with the new hash and checksum as soon as we are done.

Thanks

Replying to petr@…:

@fanfarillo.gcc: You need to place the patch file into the files directory of the port. No checksum needed because it is supplied not downloaded.

Otherwise, if you want to base the port on a more recent commit

• update the hash of in github.setup;
• update the checksums;
• remove the patchfiles line if it is not needed any more;

[fanfarillo.gcc@…]

Dear all,

we have finally released OpenCoarrays 1.0.0. In attachment the new Portfile, it works for me. I also commented the tests lines.

Replying to petr@…:

@fanfarillo.gcc: You need to place the patch file into the files directory of the port. No checksum needed because it is supplied not downloaded.

Otherwise, if you want to base the port on a more recent commit

• update the hash of in github.setup;
• update the checksums;
• remove the patchfiles line if it is not needed any more;

[rouson (Damian Rouson)]

* PING

In addition to the 1.0.0 release, we have now posted an update release. It would be great to know if these will be included in Macports soon.

Damian

[rouson (Damian Rouson)]

PING *

Any response would be greatly appreciated to Alessandro's message from 3 weeks ago and mine from 6 days ago would be greatly appreciated.

Damian

[raimue (Rainer Müller)]

I reviewed the last attachment. This statement here looks strange:

# Required to run the test phase.
pre-configure {
    configure.args-append -DMPIEXEC=${prefix}/bin/${mpi.exec}
}

Why is it wrapped in a pre-configure block? I don't recognize its purpose.

What is the status of the commented tests and the tests-compiler.patch? Should this be added?

[fanfarillo.gcc@…]

Thanks for the review. All the commands related to the test phase, including the pre-configure, can be removed without any problem. The patch as well is not required to be included in the port.

Thanks

Replying to raimue@…:

I reviewed the last attachment. This statement here looks strange:

# Required to run the test phase.
pre-configure {
    configure.args-append -DMPIEXEC=${prefix}/bin/${mpi.exec}
}

Why is it wrapped in a pre-configure block? I don't recognize its purpose.

What is the status of the commented tests and the tests-compiler.patch? Should this be added?

[tenomoto (Takeshi Enomoto)]

You don't need to redefine version when github.setup is given. I updated to version 1.0.1. I still have problem. mpicc is used to compile syncimages2.c.o. Adding environment variable

build.env           ARMCI_CC=${configure.cc}

didn't help. Note mpicc is available only if port select mpi is used and should not rely on it.

[fanfarillo.gcc@…]

Replying to takeshi@…:

You don't need to redefine version when github.setup is given. I updated to version 1.0.1. I still have problem. mpicc is used to compile syncimages2.c.o. Adding environment variable

build.env           ARMCI_CC=${configure.cc}

didn't help. Note mpicc is available only if port select mpi is used and should not rely on it.

Hi,

can you apply the tests-compiler.patch? It should fix the problem. If it works, then it should also be included in the port (the tests are built even though not executed).

Thanks

[rouson (Damian Rouson)]

Could we please get a response to Alessandro's latest post from a week ago?

[mojca (Mojca Miklavec)]

Cc Me!

[anddam (Andrea D'Amore)]

Cc Me!

[tenomoto (Takeshi Enomoto)]

The patch is already incorporated in version 1.0.1. syncimages2.c is compiled with mpicc as in the log I uploaded.

[ 31%] Building C object src/tests/unit/simple/CMakeFiles/C_sync_images.dir/syncimages2.c.o
cd /opt/local/var/macports/build/_Users_takeshi_MacPorts_ports_science_opencoarrays/opencoarrays/work/build/src/tests/unit/simple && mpicc  -DPREFIX_NAME=_gfortran_caf_ -pipe -Os  -DNDEBUG -arch x86_64 -mmacosx-version-min=10.10 -I/opt/local/var/macports/build/_Users_takeshi_MacPorts_ports_science_opencoarrays/opencoarrays/work/opencoarrays-1.0.1/src    -o CMakeFiles/C_sync_images.dir/syncimages2.c.o   -c /opt/local/var/macports/build/_Users_takeshi_MacPorts_ports_science_opencoarrays/opencoarrays/work/opencoarrays-1.0.1/src/tests/unit/simple/syncimages2.c
/bin/sh: mpicc: command not found

[tenomoto (Takeshi Enomoto)]

Cc Me!

[fanfarillo.gcc@…]

I removed that file from the test suite. It will not be compiled anymore.

Replying to takeshi@…:

The patch is already incorporated in version 1.0.1. syncimages2.c is compiled with mpicc as in the log I uploaded.

[ 31%] Building C object src/tests/unit/simple/CMakeFiles/C_sync_images.dir/syncimages2.c.o
cd /opt/local/var/macports/build/_Users_takeshi_MacPorts_ports_science_opencoarrays/opencoarrays/work/build/src/tests/unit/simple && mpicc  -DPREFIX_NAME=_gfortran_caf_ -pipe -Os  -DNDEBUG -arch x86_64 -mmacosx-version-min=10.10 -I/opt/local/var/macports/build/_Users_takeshi_MacPorts_ports_science_opencoarrays/opencoarrays/work/opencoarrays-1.0.1/src    -o CMakeFiles/C_sync_images.dir/syncimages2.c.o   -c /opt/local/var/macports/build/_Users_takeshi_MacPorts_ports_science_opencoarrays/opencoarrays/work/opencoarrays-1.0.1/src/tests/unit/simple/syncimages2.c
/bin/sh: mpicc: command not found

[eborisch (Eric A. Borisch)]

I've fixed this up and committed it as r140717. Give it a whirl.

[eborisch (Eric A. Borisch)]

Cc Me!

[rouson (Damian Rouson)]

This ticket can be closed now. Thanks, everyone for all the hard work to get OpenCoarrays into MacPorts!

[barracuda156]

Replying to rouson:

This ticket can be closed now. Thanks, everyone for all the hard work to get OpenCoarrays into MacPorts!

And then when I opened an issue trying to update it and fix, this is the reply I get: ​https://github.com/sourceryinstitute/OpenCoarrays/issues/768#issuecomment-1242822834