LaTeXML: update to 0.8.1 and fix Perl branches

[brucemiller (bruce miller)]

I've updated LaTeXML to version 0.8.1, upstream, and want to update the MacPorts Portfile. I've attached the Portfile, and diff, by modifying the Portfile that is current on MacPorts. (I don't have commit access)

*However*, there is a question as to whether the handling of Perl versions is correct; it attempts to use whatever Perl is currently installed on the user's system (I think). Please take a look.

[brucemiller (bruce miller)]

diff -u Portfile-0.8.0 Portfile > Portfile.diff

[mojca (Mojca Miklavec)]

(I was thinking about splitting this into two parts: the module and the "executable", but I'm not sure if that makes sense or not.)

I was playing with the Portfile a bit and the thing I wasn't aware of was that the order of these two commands matters:

perl5.create_variants   ${perl5.branches}
perl5.setup             ${name} ${version}

else the setup picks the default perl version, no matter what the variant is (perl5.create_variants has to select perl5.major early enough).

Can you please test the attachment?

[mojca (Mojca Miklavec)]

update LaTeXML to 1.8.1 and try to fix perl variants

[brucemiller (bruce miller)]

Unfortunately, I rarely get access to a Mac to do development on, and am not in a position to do any testing soon. I develop on Linux, but have users on Macs. Ideally, I'd stumble across a MacPorts maintainer that was interested in actually using LaTeXML, and would be willing to take on the maintainer status for LaTeXML without feeling taken advantage of. Otherwise, I'm just here depending on the kindness of strangers! :>

I'd hoped that the update would be a simple matter of updating the version & checksums, but it looks like there's a deeper problem with the strategy for perl versions; and maybe some controversy? It sounds like Ryan isn't too fond of the idea of simultaneously supporting several versions. Should we just revert it to the straightforward 5.16 only?

Without understanding the details of the perl port group, however, what you said about the order of commands makes sense.

BTW: I don't think there's much use in separating out the executables from the modules; the modules are really just the implementation of the executables (although they could be used independently).

[mojca (Mojca Miklavec)]

Replying to bruce.miller@…:

Unfortunately, I rarely get access to a Mac to do development on, and am not in a position to do any testing soon. I develop on Linux, but have users on Macs.

Maybe at least one of the users (or someone CCed here) would be willing to do some basic testing. Else we can also commit the change and wait for the first user to start complaining ;)

Ideally, I'd stumble across a MacPorts maintainer that was interested in actually using LaTeXML, and would be willing to take on the maintainer status for LaTeXML without feeling taken advantage of.

(I usually don't even need XML output, but my favourite flavour of TeX already supports HTML/XML/MathML output out of the box ;)

It is very valuable to have the original developer as a maintainer or co-maintainer though. Sometimes (for some projects) it is very difficult to get the patches upstream.

I'd hoped that the update would be a simple matter of updating the version & checksums

It usually is. But you explicitly wrote that some user had a problem and I can verify that the "old" port doesn't behave properly if the version of Perl changes.

but it looks like there's a deeper problem with the strategy for perl versions; and maybe some controversy?

Yes. There's a big question of whether we need to support multiple versions of Perl or not. Both on the MacPorts scale as well as on the port scale.

It sounds like Ryan isn't too fond of the idea of simultaneously supporting several versions. Should we just revert it to the straightforward 5.16 only?

I don't know. There are lots of reasons to keep support for multiple versions and lots of reasons to remove it. It's more a matter of a personal opinion/taste, even though we should probably decide about the guidelines on the global MacPorts scale.

(In any case it is not wrong to keep support for multiple versions. But the way it is/was set up now hardly made any sense because the selection didn't even work.)

Without understanding the details of the perl port group, however, what you said about the order of commands makes sense.

BTW: I don't think there's much use in separating out the executables from the modules; the modules are really just the implementation of the executables (although they could be used independently).

OK.

[pixilla (Bradley Giesbrecht)]

Replying to mojca@…:

perl5.create_variants   ${perl5.branches}
perl5.setup             ${name} ${version}

Using both perl5.create_variants and perl5.setup is untested. The perl5.create_variants proc is intended to be used instead of perl5.setup when perl variants are desired instead of the perl subports perl5.setup provides.

Try this:

perl5.require_variant   yes 
perl5.branches          5.16 5.18 5.20 
perl5.create_variants   ${perl5.branches} 

Note: perl5.conflict_variants defaults to yes so omitted above.

[mojca (Mojca Miklavec)]

Did you actually test this or are you just speculating?

The problem is that this is actually a typical Perl module, so it needs almost the same handling as the rest of the Perl modules.

perl5.create_variants alone wasn't enough because it would expect the configure && make && make install workflow. And the current setup (without create_variants) doesn't create any variants at all.

I agree that this might need a closer look (and possibly some modifications of the PortGroup), but leaving one of those options out didn't work for me in quick attempts.

[pixilla (Bradley Giesbrecht)]

Replying to mojca@…:

Did you actually test this or are you just speculating?

I was unfamiliar enough with perl5.setup that I choose as a first step to add the proc perl5.create_variants as a standalone proc.

The problem is that this is actually a typical Perl module, so it needs almost the same handling as the rest of the Perl modules.

perl5.create_variants alone wasn't enough because it would expect the configure && make && make install workflow. And the current setup (without create_variants) doesn't create any variants at all.

I agree that this might need a closer look (and possibly some modifications of the PortGroup), but leaving one of those options out didn't work for me in quick attempts.

At some point I imagined perl5.setup would be extend to create subports _OR_ variants based on a Portfile variable, defaulting to current behavior in the absence of the variable. I currently am not familiar enough with perl5.setup to provide test cases for it's current functionality so I left it alone for now.

[brucemiller (bruce miller)]

Unfortunately, I couldn't rouse any guinea pigs on my end of things.

To try to get unwedged, I went looking for other likely ports of perl modules that have executables; the first one I looked at perl-Perl-Critic seemingly uses the same strategy to avoid committing to a particular version of perl. Other than not having variants for TeX installations, the only significant differences I notice in that Portfile from the one I originally propsed are that

• it omits perl5.link_binary_suffix
• it adds perl5.use_module_build just after depends_lib-append

I would assume that use_module_build is used to assert that the Perl Makfile.PL uses the Module::Build mechanism instead of the old-style ExtUtils::MakeMaker. I'm not sure what the link_binary_suffix was for; I must have copied it from somewhere(?)

So, I wonder, does perl-Perl-Critic actually install under different perl version?

[mojca (Mojca Miklavec)]

Do you mean the port perl/p5-perl-critic? Yes, it does.

> port contents p5.20-perl-critic
Port p5.20-perl-critic contains:
  /opt/local/bin/perlcritic-5.20
  /opt/local/lib/perl5/vendor_perl/5.20/Perl/Critic.pm
  /opt/local/lib/perl5/vendor_perl/5.20/Perl/Critic/Annotation.pm
  [...] /opt/local/lib/perl5/vendor_perl/5.20/Perl/Critic/...
  /opt/local/lib/perl5/vendor_perl/5.20/Test/Perl/Critic/Policy.pm
  /opt/local/lib/perl5/vendor_perl/5.20/darwin-thread-multi-2level/auto/Perl/Critic/.packlist
  /opt/local/libexec/perl5.20/perlcritic
  /opt/local/share/perl5.20/man/man1/perlcritic.1pm
  /opt/local/share/perl5.20/man/man3/Perl::Critic.3pm
  [...] opt/local/share/perl5.20/man/man3/...

[mojca (Mojca Miklavec)]

But of course one then ends up with bin/perlcritic-5.20 instead of bin/perlcritic. That could be fixed with some tricks, for example by creating a "blank" port perlcritic doing nothing else but adding a dependency on p5.x-perl-critic and creating a symlink to the binary. Or any other trick.

[brucemiller (bruce miller)]

In my original email, I mentioned that a user couldn't install LaTeXML (the 0.8.0 version) because it failed to install a dependency p5.12-Archive-Zip. I've since talked with him, and it appears that his system, even after a "port selfupgrade", is somehow stuck on perl 5.12 --- it didn't upgrade the obsolete perl. So, I think his problem is more fundamental and has nothing to do with LaTeXML's port. I've suggested that he contact you folks for help.

The only other problem was what @mojca reported in trying to install the proposed portfile for 0.8.1 under perl 5.20. More on this in next comment.

[brucemiller (bruce miller)]

Replying to mojca@…:

But of course one then ends up with bin/perlcritic-5.20 instead of bin/perlcritic. That could be fixed with some tricks, for example by creating a "blank" port perlcritic doing nothing else but adding a dependency on p5.x-perl-critic and creating a symlink to the binary. Or any other trick.

I think that is what setting perl5.link_binary_suffix to empty is supposed to achieve.

So, the only differences in our Portfiles seem, to me, to be innocuous, unless they have some sneaky side-effects. I can't really see what my Portfile is doing differently than p5-perl-critic. I don't want to second-guess you, but are you sure the Portfile I've proposed for 0.8.1 doesn't install correctly under any supported perl? (ie. perl 5.16, 5.20)

[brucemiller (bruce miller)]

We seem stalled here. I can't see what differences there are between my proposed portfile and that of perlcritic. It seems to work in the normal usecases, and is essentially the same as what I used for version 0.8.0.

Is there a way to move forward? Thanks.

[dbevans (David B. Evans)]

Let me try and help. mojca currently has her head down, trying to finish her thesis.

[dbevans (David B. Evans)]

Replying to bruce.miller@…:

We seem stalled here. I can't see what differences there are between my proposed portfile and that of perlcritic. It seems to work in the normal usecases, and is essentially the same as what I used for version 0.8.0.

Is there a way to move forward? Thanks.

The (hidden) difference is that the perl5 PortGroup behaves differently depending on how you name your port. If the name is of the form p5-* then it assumes this is a perl module and it attempts to make parallel installable versions (subports) for each of the perl branches that you specify. This is the case with p5-perl-critic. In addition to the main port, it creates subports p5.16-perl-critic, p5.18-perl-critic and p5.20-perl-critic.

In your case, as I understand it from the comments, this is a perl application not a perl module. To work this way, the name needs to be declared prior to invoking the PortGroup (which it is in your port).

Once you've decided on being an application, then you need decide if you want to support more than one version of perl. This is where some controversy (difference of opinion?) arises.

If it really doesn't matter which version of perl is being used, then something like you have done would work. The only issue is the declaration of multiple perl branches. I think your port as it stands will use the perl selected by the perl5 port. This is perl5.16 unless something other than the default variant is used. Need to look at the perl5 PortGroup code to make sure.

On the other hand, if you want users to be able to decide which perl to use when running your application then mojca's patch would do it. It results in variants being generated for each of the listed branches, such as +perl5_16 +perl5_18 +perl5_18. These variants are generally mutually exclusive so there is only one version of the port installed at a time.

Personally, I think that if the intent is to just run the application and the selection of perl version is immaterial, configuring the port to use a single fixed version of perl is sufficient. This is typically the default perl version, currently perl5.16.

However, mojca prefers giving people the choice even if it really doesn't matter. Hence, her implementation using variants.

As port maintainer, you get to have it your way! Which way makes the most sense to you?

[brucemiller (bruce miller)]

Thanks for your comments; I certainly don't want to interfere with Mojca's thesis!

Indeed, this really should be thought of as an application rather than a perl module. And in principle, it should work with any non-obsolete version of Perl. (Of course, with newer versions, occasionally there's a new warning or such that calls for an update).

I'm slightly less fond of the "pick a single perl version" approach, as that caused a big headache in the past. There was a long gap between "official" releases of LaTeXML (although lots of github activity) and the macport was stuck on perl 12, which had gone obsolete. Of course, my intent is not to let such a long gap reoccur, but nevertheless.

If there was an automatic mechanism to update the LaTeXML port whenever the official default perl changed, I'd be fine with that, but that doesn't seem to be a possibility at present(?). Does perl itself get automatically updated whenever there is a change to the default version?

So my intention was similar to what Mojca was proposing in the sense of: Use whatever perl the user currently has installed, whether or not it's the current default perl. I'm not sure if that's exactly what Mojca's patch achieves, but her patch sounds close enough isn't it?

[mojca (Mojca Miklavec)]

You don't need to bother about my suggestion about perl variants. The status with perl ports is a bit of a mess anyway. I'm happily leave it to David to do whatever you want to do as long as it works.

[brucemiller (bruce miller)]

I guess I'm fine with anything that ends up with the executables being withOUT the perl version suffix (ie. just "latexml"). And to the extent possible, minimize the need for users to have to install a particular version of perl, and minimize the need to track macport's default perl version to update the Portfile.

[dbevans (David B. Evans)]

I believe the executable version suffixes are only used in the case of perl modules where multiple subports need to coexist without conflict.

I agree with mojca that the perl variants are not really needed here. Using a single version of perl alone is sufficient and reduces potential maintenance headaches with perl version related problems because the same perl version is always used. By convention, the "default perl" version is usually chosen but any version that is supported will do.

Note that this doesn't necessarily mean that the port uses whatever port the user has installed. It means that the port will install the perl version it wants REGARDLESS of the perl version(s) installed.

At any rate, if you don't mind, I'll take a crack at cleaning up the Portfile patch that you submitted as a concrete example of the application of these comments.

[brucemiller (bruce miller)]

Not only don't mind; I'd be thrilled! Thanks!

[dbevans (David B. Evans)]

Maintainer's proposed patch as revised by devans

[dbevans (David B. Evans)]

Attached is my version of your original patch.

Issues addressed:

• remove revision statement (will use default revision 0)
• limit perl branches to just 5.16
• move categories statement to after perl5.setup to overide its selection (perl)
• declare noarch (no architecture dependent files installed -- suppresses universal variant)
• remove {${perl5.major} ne ""'} conditional from dependencies (only used when building modules to distinguish main port from subports)
• mark variants as conflicting with each other since it doesn't seem likely they can co-exist
• move variants' exec test into pre-configure since it can fail if executed before dependencies are installed.

Left to do:

Add a license statement. Without an explicit statement, license is set to unknown which suppresses distribution of prebuilt archives.

If you can indicate the license under which this is distributed and are OK with the changes I have made, I can commit this for you.

[brucemiller (bruce miller)]

I'm not sure what set of licenses are recognized; I'm a U.S.govt employee, so we usually claim "public domain". That's not always a well-defined concept, so CC0 could be used, instead.

Otherwise, it looks great. Thanks for the tune-up! :>

[larryv (Lawrence Velázquez)]

Replying to devans@…:

Left to do:

Add a license statement. Without an explicit statement, license is set to unknown which suppresses distribution of prebuilt archives.

If you can indicate the license under which this is distributed and are OK with the changes I have made, I can commit this for you.

The website claims public domain.

​http://dlmf.nist.gov/LaTeXML/notices.html

[dbevans (David B. Evans)]

Committed in r137055, license public-domain added. Thanks for the update. Sorry it took so long to get things ironed out.

[mojca (Mojca Miklavec)]

Minor thing: I would suggest to change this for the texlive variant:

configure.args-append   TEXMF=[exec kpsewhich --expand-var='\$TEXMFLOCAL'] 

This is fine for mactex (even though I would then use /usr/local/texlive/texmf-local), but there's no reason why /opt/local/share/texmf wouldn't be used. It actually makes a lot more sense than TEXMFLOCAL and it's more predictable (I could imagine users changing the TEXMFLOCAL variable, leading to inconsistent installation). The tree texmf-local is generally meant for files added manually by the user.

If one uses

PortGroup texlive 1.0

then ${texlive_texmfports} could be used.

(To clarify: if a user installs latexml manually or to some external distribution, then it makes sense to use TEXMFLOCAL. But if it's installed by a package manager, it should be installed to the "default" tree.)

[brucemiller (bruce miller)]

Replying to larryv@…:

The website claims public domain.

​http://dlmf.nist.gov/LaTeXML/notices.html

Ah, yes: Thanks for reminding me! I'm resynchronizing the notices page with the LICENSE file to clarify the Public Domain, CC0 issue.

[brucemiller (bruce miller)]

Replying to mojca@…:

If one uses

PortGroup texlive 1.0

then ${texlive_texmfports} could be used.

There's no hidden dependency introduced or problem having 2 PortGroups?

(To clarify: if a user installs latexml manually or to some external distribution, then it makes sense to use TEXMFLOCAL. But if it's installed by a package manager, it should be installed to the "default" tree.)

I agree completely; I just didn't know how to do it!

Does the above patch do what you're suggesting? (I'm not in a position to test)

[mojca (Mojca Miklavec)]

It looks OK to me, I'm about to test it. Now that I look into it, I would make one further change:

     post-activate {
-        system "mktexlsr"
+        texlive.mktexlsr
     }

And I have an additional question: why do you need a dependency on the complete texlive installation? Isn't there some subset of TeX Live that would be sufficient to make LaTeXML functional?

[mojca (Mojca Miklavec)]

Additionally I'm a bit worried about the following:

    post-activate {
        system /usr/texbin/mktexlsr
    }

Isn't executing this command as a macports user a tiny bit problematic?

[mojca (Mojca Miklavec)]

When testing the installation, here's what I get, most likely during destroot phase (I'm not saying that this is a real problem):

mktexlsr: /opt/local/etc/texmf: directory not writable. Skipping...
mktexlsr: /opt/local/share/texmf: directory not writable. Skipping...
mktexlsr: /opt/local/share/texmf-local: directory not writable. Skipping...
mktexlsr: /opt/local/share/texmf-texlive: directory not writable. Skipping...
mktexlsr: /opt/local/var/db/texmf: directory not writable. Skipping...

Does latexml call mktexlsr on its own during make install?

[mojca (Mojca Miklavec)]

I committed r137152. I should have increased the revision number, but somewhat deliberately didn't as the file location of a variant doesn't really matter.

I suggest opening a new ticket addressing possible improvements related to TeX Live.

[mojca (Mojca Miklavec)]

See #47972

[larryv (Lawrence Velázquez)]

Replying to mojca@…:

I committed r137152. I should have increased the revision number, but somewhat deliberately didn't as the file location of a variant doesn't really matter.

It doesn’t matter whether the change only affects a variant. If any files are added, changed, or moved, the revision should be increased. (Unless the importance of the change is truly dwarfed by the inconvenience of a rebuild. I wouldn’t revbump atlas to fix a documentation typo.)

[mojca (Mojca Miklavec)]

Replying to larryv@…:

It doesn’t matter whether the change only affects a variant. If any files are added, changed, or moved, the revision should be increased. (Unless the importance of the change is truly dwarfed by the inconvenience of a rebuild. I wouldn’t revbump atlas to fix a documentation typo.)

I know. This change might not be as extreme as a typo in atlas, but it still has zero influence on functionality. Plus, I was hoping that a quick action on #47972 would soon lead to a new increased revision anyway.

[mojca (Mojca Miklavec)]

But of course feel free to increase the revision if you consider it important enough. (I would rather see the other ticket fixed some time soon.)

[brucemiller (bruce miller)]

Replying to mojca@…:

When testing the installation, here's what I get, most likely during destroot phase (I'm not saying that this is a real problem):

mktexlsr: /opt/local/etc/texmf: directory not writable. Skipping...
mktexlsr: /opt/local/share/texmf: directory not writable. Skipping...
mktexlsr: /opt/local/share/texmf-local: directory not writable. Skipping...
mktexlsr: /opt/local/share/texmf-texlive: directory not writable. Skipping...
mktexlsr: /opt/local/var/db/texmf: directory not writable. Skipping...

Does latexml call mktexlsr on its own during make install?

Hmm, yes actually it does. I'm fuzzy now on the history of why it's also called in post-activate (similarly in '%post' of the rpm spec file). Perhaps because of not knowing the path to mktexlsr? Or permissions in the different phases? Which one might be the best place? (could disable either one)

Which variant (texlive or Mactex) caused these messages?

I'll respond to #47972...