CPAN.pm/cpan2port needs gzip?

[chrstphrchvz (Christopher Chavez)]

I have Perl 5.28.2 installed, which includes CPAN.pm 2.20; updating CPAN.pm to 2.26 does not resolve this issue.

The gzip port is not declared as a dependency of perl5.28, but CPAN shell (cpan-5.28) erroneously finishes when /opt/local/bin/gzip isn't present. Distributions are never extracted (e.g. to .cpan/build), but CPAN shell proceeds as if modules build and are ready to install. Had CPAN shell worked in this example (as when gzip is present), then it would have found a Makefile.PL and compiled a few XS files:

$ cpan Tk::IDElayout
Reading '/Users/christopherchavez/.cpan/Metadata'
  Database was generated on Sat, 15 Jun 2019 17:17:02 GMT
Running install for module 'Tk::IDElayout'
CPAN: Digest::SHA loaded ok (v6.02)
CPAN: Compress::Zlib loaded ok (v2.086)
Checksum for /Users/christopherchavez/.cpan/sources/authors/id/C/CA/CAC/Tk-IDElayout-0.34.tar.gz ok
sh: /opt/local/bin/gzip: No such file or directory
CPAN: YAML loaded ok (v1.29)
CPAN: CPAN::Meta::Requirements loaded ok (v2.140)
CPAN: CPAN::Meta loaded ok (v2.150010)
Package seems to come without Makefile.PL.
  (The test -f "/Users/christopherchavez/.cpan/build/CAC-1/Makefile.PL" returned false.)
  Writing one on our own (setting NAME to TkIDElayout)
Configuring C/CA/CAC/Tk-IDElayout-0.34.tar.gz with Makefile.PL
Generating a Unix-style Makefile
Writing Makefile for TkIDElayout
Writing MYMETA.yml and MYMETA.json
  CAC/Tk-IDElayout-0.34.tar.gz
  /opt/local/bin/perl5.28 Makefile.PL -- OK
Running make for C/CA/CAC/Tk-IDElayout-0.34.tar.gz
CPAN: Module::CoreList loaded ok (v5.20190419)
  CAC/Tk-IDElayout-0.34.tar.gz
  /usr/bin/make -- OK
Running make test
No tests defined for TkIDElayout extension.
  CAC/Tk-IDElayout-0.34.tar.gz
  /usr/bin/make test -- OK
Running make install
Password:

As a result, running cpan2port (20190228) without gzip present fails, because the distribution was never extracted:

$ cpan2port -v -t Tk::IDElayout
Reading '/Users/christopherchavez/.cpan/Metadata'
  Database was generated on Sat, 15 Jun 2019 17:17:02 GMT
Tk::IDElayout is not perl porter
Tk::IDElayout (no description available)
Tk::IDElayout will show deps
Running get for module 'Tk::IDElayout'
Checksum for /Users/christopherchavez/.cpan/sources/authors/id/C/CA/CAC/Tk-IDElayout-0.34.tar.gz ok
sh: /opt/local/bin/gzip: No such file or directory
trying to find dist in /Users/christopherchavez/.cpan/build/Tk-IDElayout-0.34*
trying to find dist in 1*
(((can't find dist at /opt/local/bin/cpan2port line 127.
))) at /opt/local/bin/cpan2port line 353.
	main::foreach_pkg(CODE(0x7fc08f004248)) called at /opt/local/bin/cpan2port line 378

I found a ticket that another MacPorts user opened for CPAN which seems related: ​https://rt.cpan.org/Public/Bug/Display.html?id=128063. I'm not sure their resolution of making cpan-5.xx use /usr/bin/gzip (if /opt/local/bin/gzip is missing) is appropriate, since the intention might be for ports to only use MacPorts' gzip.

[chrstphrchvz (Christopher Chavez)]

One question I have is whether CPAN.pm (and therefore Perl 5 when installed with core modules) actually requires/assumes gzip to be present. Linux distributions seem to have gzip mandatorily installed; I would think that is not the case for BSD or other UNIX.

[mojca (Mojca Miklavec)]

I will look into this, but if you are interested in cpan2port, you are invited to also test the new upt support and provide feedback.

[chrstphrchvz (Christopher Chavez)]

One question I have is whether CPAN.pm (and therefore Perl 5 when installed with core modules) actually requires/assumes gzip to be present.

This seems to be the case since CPAN.pm accepts a path to gzip when configuring. It just seems to have a separate issue of not failing early when the gzip it is told to use is missing.

the intention might be for ports to only use MacPorts' gzip

Another question: are ports are allowed to use /usr/bin/gzip, i.e. does the usual policy of using software outside MacPorts not apply to /usr/bin/gzip?

[mojca (Mojca Miklavec)]

One question: where does the path to gzip come from when running cpan?

By quickly inspecting the CPAN code (I did not try to execute it yet) it seems to me as if cpan creates a config file and writes location to the required executables the first time when you run it. This is problematic in case that you have gzip from MacPorts installed when you run cpan for the first time, but then uninstall gzip later.

Another question: are ports are allowed to use /usr/bin/gzip, i.e. does the usual policy of using software outside MacPorts not apply to /usr/bin/gzip?

Yes, I would say that ports should be allowed to use /usr/bin/gzip (unless maybe on older systems some functionality is missing or so). The problem comes from libraries like libpng where we try to avoid linking against ancient libraries and then debug forever why something doesn't work as expected. Or when the executable is so old that it's not usable. But occasionally running some minor executable from /usr is fine and reduces the number of dependencies (which isn't a bad thing). Note that gzip in particular is very important, as it would be a chicken-and-egg problem if we were not allowed to use it. (How would we upgrade / extract gzip port itself?)

We need to figure out how cpan decides which executable to use. If we can convince it to always consistently use /usr/bin/gzip, that would be somewhat preferable to declaring a dependency on the gzip port. Any help greatly appreciated.

This piece of code for searching for executables like gzip is also nice (I'm not exactly sure what it does, but ...):

        for $progname (@external_progs) {
            next if $matcher && $progname !~ /$matcher/;
            if ($^O eq 'MacOS') {
                $CPAN::Config->{$progname} = 'not_here';
                next;
            }

[mojca (Mojca Miklavec)]

One further point to clarify: we can of course add a runtime dependency on gzip, but since the percentage of users running cpan might be small, and perl itself works without /opt/local/bin/gzip (and even cpan does to some extent), and if there is a way to convince cpan to work correctly without declaring that dependency and use /usr/bin/gzip, I would prefer it if we would convince the cpan script to consistently use /usr/bin/gzip.

If we fail to find a solution ... well, then we can still add a dependency on gzip.

Note that the same thing will probably just as well fail if you install your own gzip somewhere (so that it takes precedence over the version installed by either MacPorts or the OS itself; that could include installing it via HomeBrew in case they provide a formula for gzip), and then uninstall your own gzip.

On Linux this is generally not a problem since you only get one single gzip at one single location, even if you upgrade your OS. I would be surprised if it was allowed to uninstall gzip from a Linux system.