perl5 @5.34 +perl5_36: cannot install perl modules

[NorthboundTrain (Joe Casadonte)]

I don't understand why 5.036 is a variant instead of a real release, but it makes it very difficult to install Perl modules via MacPorts. For example, I had File::Copy::Recursive installed under 5.34.1 (p5-file-copy-recursive), but after "installing" 5.036 it no longer finds it. And re-installing it after activating 5.036 does not fix the issue.

re: 5.036 -- I could not figure out how to get the 5.036 variant to install until I found a decade-old StackOverflow article that essentially had me do the following:

sudo /opt/local/bin/port deactivate perl5 && sudo /opt/local/bin/port install perl5 +perl5_36

After doing that, perl -v reports 5.36.0 -- maybe that's not the right way to get 5.036 installed/activated? Anyhow, I tried all of the following and none of it made a difference:

sudo /opt/local/bin/port selfupdate && sudo /opt/local/bin/port upgrade outdated
sudo /opt/local/bin/port install p5-file-copy-recursive
sudo /opt/local/bin/port install p5-file-copy-recursive +perl5_36

Here's some other info that might be helpful:

$ perl -MFile::Copy::Recursive -eexit
Can't locate File/Copy/Recursive.pm in @INC (you may need to install the File::Copy::Recursive module) (@INC contains: /opt/local/lib/perl5/site_perl/5.34/darwin-thread-multi-2level /opt/local/lib/perl5/site_perl/5.34 /opt/local/lib/perl5/site_perl/5.36/darwin-thread-multi-2level /opt/local/lib/perl5/site_perl/5.36 /opt/local/lib/perl5/vendor_perl/5.36/darwin-thread-multi-2level /opt/local/lib/perl5/vendor_perl/5.36 /opt/local/lib/perl5/5.36/darwin-thread-multi-2level /opt/local/lib/perl5/5.36).
BEGIN failed--compilation aborted.

$ pwd
/opt/local/lib/perl5/vendor_perl

$ find . -name Recursive.pm
./5.34/File/Copy/Recursive.pm

$ sudo ln -s 5.34 5.36

$ perl -MFile::Copy::Recursive -eexit
HiRes.c: loadable library and perl binaries are mismatched (got first handshake key 0xfb80080, needed 0xfc00080)

$ sudo rm 5.36

$

Any help is greatly appreciated -- thanks!

[ryandesign (Ryan Carsten Schmidt)]

The perl5 port is a stub port. It doesn't install perl, it just installs perl and perl5 symlinks that points to whatever perl you have installed. By default, at present, those symlinks point to the perl5.34 executable installed by the perl5.34 port. You can use a different variant of the perl5 port if you'd like those symlinks to point to a perl executable provided by a different perl port. For example, by installing perl5 with the +perl5_36 variant, you tell it to depend on the perl5.36 port instead and to provide perl and perl5 symlinks that point to the perl5.36 executable.

All of the p5-* module ports are stubs too that don't install anything, and they don't have variants. For example, p5-file-copy-recursive doesn't install anything. It just depends on p5.34-file-copy-recursive which installs File::Copy::Recursive for perl5.34. If you want to install File::Copy::Recursive for perl5.36, install p5.36-file-copy-recursive.

It's designed this way so that you can install several different versions of perl, and modules, and they don't conflict with one another.

[ryandesign (Ryan Carsten Schmidt)]

Ahem, I should have checked before I responded. There is no p5.36-file-copy-recursive at this time. None of the p5-* module ports have had p5.36-* subports added to them yet. The person who usually does such major version updates of perl modules has not contributed to MacPorts for awhile (not at all yet this year). Also, perl version 5.38.0 was released a couple weeks ago but there is no perl5.38 port and no p5.38-* modules yet.

[NorthboundTrain (Joe Casadonte)]

Thanks for the info! I guess that's why there isn't a port of 5.36.1 yet (just 5.36.0 is available). Tackling the 5.36.1 and 5.38.0 ports may be a bit much for a newbie like me, but maybe there's something I can do to help with the p5.36 module ports? If there's any documentation you can point me to, I'd be happy to give it a go.

[NorthboundTrain (Joe Casadonte)]

Are these the current instructions?

wiki:howto/cpan2port

[ryandesign (Ryan Carsten Schmidt)]

cpan2port is how you would create ports for new perl modules. That's not what's needed here. The ports for the perl modules already exist, they just don't list 5.36 as an available version.

For any change to MacPorts, it's our hope that a contributor would test the change on their own system first before contributing it. So to add perl 5.36 support to a port like p5-file-copy-recursive, you would edit the Portfile​, add 5.36 to the end of the perl5.branches line, verify it builds, passes any available tests, and generally works, and send a pull request with that change. However, p5-file-copy-recursive depends on five other p5 modules; they would each need perl 5.36 added to them first. They may each have other dependencies, and so on. The perl ecosystem has a tendency to be composed of a zillion small interdependent modules. To identify all of the Portfiles that p5-file-copy-recursive depends on, you could use a command like this:

port -q file rdepof:p5.34-file-copy-recursive | grep /p5-

I get a list of 24 files. You could add 5.36 to the perl5.branches line in all of them, verify they all build and work, and submit the batch of changes as a pull request. The maintainers of the ports will be notified. I found five distinct maintainers across that collection of 24 ports. The maintainers would then approve or reject the change, or the change can be merged after 72 hours if maintainers do not respond.

In the past, because we have nearly 2,000 perl module ports, and individually testing small batches is tedious and nobody volunteered to do that work, the entire batch of 2,000 perl module ports was updated to add a new perl version simultaneously without testing any of it first. If that's how we want to handle it again this time around we can but it causes a number of problems.

First, somebody has to write a script to add 5.36 to each perl module's perl5.branches (or find someone's previous script that did that) and commit it. The resulting modifications have to be inspected carefully before committing. We have in years past inadvertently committed syntax errors or other mistakes at this point because the script did not anticipate some slightly nonstandard Portfile construction. You should be able to look through the git commit history or Trac ticket history to find previous perl version updates.

Normally, any commit will automatically schedule builds of any ports that need to be built on the buildbot, for each builder. We currently have 17 builders which would be about 34,000 builds scheduled. The buildbot software is not designed to accommodate anywhere near that many pending builds and this grinds the buildmaster to a complete standstill of 100% CPU usage and unresponsiveness. It then falls to the buildbot administrator (me) to manually cancel all of these builds (just cancelling the builds can take hours) and load the list of desired builds into a separate script that schedules them in small batches over the course of the next several weeks, which is probably about how long it would take to attempt to build them all. It is therefore appreciated if such a mass update commit can be coordinated with me so that I can shut off the buildbot commit notifications beforehand.

And then there's the problem that since the builds were never tested, some of them may fail. New perl versions are usually highly compatible with older ones so this is not usually a problem, however there have been notable exceptions like perl 5.26 which removed . from @INC which broke quite a few modules which had to be identified and fixed manually. Since this might also be the first time the module has been built for any perl version since the last time a perl version was added to it, a build might fail for reasons unrelated to the perl version such as an incompatibility with a new version of macOS or Xcode.

And when a port is fixed, that does not automatically schedule new build attempts for any ports that depended on that formerly-broken port. Someone either has to remember to do that manually, or else we don't do builds (and thus don't provide archives) for some ports that we should be able to build.

Personally I favor the bit-at-a-time approach since it avoids those problems. It's what we use for python module ports and it seems to work there.

[ryandesign (Ryan Carsten Schmidt)]

Updating the perl5.36 port to 5.36.1 would be comparatively easy. The perl5 Portfile​ is set up in a more complicated way than most Portfiles since it has multiple subports, one for each perl version, but it has been designed to be easy to update. You can look through the history of the perl5 Portfile to see how previous updates were done. For example, here's where perl5.34 was updated from 5.34.0 to 5.34.1. Looks like it could be as simple as just modifying the single line that starts with 5.36. Then of course test the build and if it works, submit a pull request.

Adding a perl5.38 subport to it is more involved but you can check the history of the port for inspiration, like this commit where perl5.34 was added. Looks like many patches were added, though I expect most were just copied from perl5.32. You would want to carefully study the perl 5.38 release notes to see if there are any changes in this version that need to be addressed in some way in the rest of the Portfile.

[NorthboundTrain (Joe Casadonte)]

Ryan,

I think I'd like to attempt the 5.36.1 update and the 5.36 perl modules update. How would you prefer to do further communications on the subject? Thanks!

[ryandesign (Ryan Carsten Schmidt)]

Replying to NorthboundTrain:

I think I'd like to attempt the 5.36.1 update

This update has been submitted here: ​https://github.com/macports/macports-ports/pull/19678

and the 5.36 perl modules update. How would you prefer to do further communications on the subject? Thanks!

You can comment here but I'm not sure who will see your comments; it might not be many developers. You could get a wider audience on the macports-dev mailing list, for example if you have questions.

[ryandesign (Ryan Carsten Schmidt)]

Whether your plan is to update all 2,000 perl modules at once, or just do small batches at a time, probably best to communicate your plan to macports-dev ahead of time in case others have opinions about it.

It might also be worthwhile to add perl5.38 @5.38.0 first. That way, you can add 5.36 and 5.38 to the perl modules at the same time.