Exclude unwanted files in ${prefix}/share/locale on activation (localepurge)

[RJVB (René Bertin)]

As suggested elsewhere I propose a new PortGroup which might inspire an extension of "base".

The aim is to allow users who don't like or cannot afford to waste disk space (on things they don't use) to reduce the number of translations installed by ports. A good example is port:VLC : this one consists for over 50% of translations, most of which will never be used by the (vast) majority of users. That may be "only" around 37Mb, which is why I've hesitated a bit to make this proposal. Still, leaving only the en_GB, fr and nl translations that overhead shrinks to about 1.2Mb .

The principle of this PortGroup is simple: if adds a variant (tentatively called langselect) which adds a post-destroot block that prunes translations from ${destroot}${prefix}/share/locale, leaving only translations the user is interested in. This happens only when the variant is active, of course, but also requires the user to list the translation(s) s/he is interested in. Those desired translations are read from a list assignment to a variable calledkeep_languages in ${prefix}/etc/macports/locales.tcl. A locales.conf file would be nicer but I didn't want to spend too much time figuring out how to write a parser for that kind of file.

Currently the code only removes translations from the share/locale directory, but one could extend the principle to prune unwanted .lproj directories too.

[raimue (Rainer Müller)]

This is a feature called localepurge on other distributions such as Debian.

The functionality should not be done in post-destroot, but instead at the time of activation. The archives should contain all the files, since they need to be reproducible. Since they are compressed, that would not waste much disk space anyway.

Also, this should work for all ports and not be in a PortGroup at all.

[RJVB (René Bertin)]

The reproducibility argument doesn't hold AFAIC, not with my proposed implementation which is activated using a non-default variant. Also, I'm not sure what good a reproducibility principle is if you allow for things to be removed between the unpacking and the moving-into-place or something with the same effect. You'd still end up with host/user specific installs. I think that if I had to debug an issue related to this kind of feature, I'd prefer to be able to request the archive and check what's in there rather than infer what goes on during a post-activate stage.

For the rest I do agree, it's something that could go into "base". I don't like calling it "purge" though: semantically that means you have to indicate what languages you do not want while typically you have a much better defined idea of the ones you do want.

[neverpanic (Clemens Lang)]

Yes, the reproducibility argument applies, because each user that activates the variant will get a different result, making the build not reproducible.

[RJVB (René Bertin)]

I still think that's taking things too far, being too rigid. The result is the same no matter at what point you purge the unwanted files, and 2 users with the same configuration file will get the same result.

Either way, wouldn't you get rev-upgrade errors about missing files if you remove things in the post-activate?

[raimue (Rainer Müller)]

Replying to rjvbertin@…:

I still think that's taking things too far, being too rigid. The result is the same no matter at what point you purge the unwanted files, and 2 users with the same configuration file will get the same result.

The archive would still be reproducible and reusable for everyone. I would not want to waste cycles to build from source just to get fewer files. That would even be counter-intuitive if I had to to install additional build dependencies.

Either way, wouldn't you get rev-upgrade errors about missing files if you remove things in the post-activate?

rev-upgrade does not check for missing translation files, it only checks for shared libraries. I did not suggest to remove files in post-activate, but not to move them into place during activate. These files would then also not be listed in the registry.

[ryandesign (Ryan Carsten Schmidt)]

Suppose a user wants to take advantage of this feature, but they have already installed a lot of ports. How will they do that? After having indicated their preferred locale, would they then have to deactivate and reactivate all affected ports, which would then prune the unwanted locales? How would the user determine which ports are affected? Or would the user have to deactivate and reactivate all ports?

Suppose a user has taken advantage of this feature, and now wants to stop using it. How do they get back the other locales? Same questions again. Do they have to deactivate and reactivate all ports?

I am not wild about this proposal because it is contrary to the principle we've used in MacPorts so far, which is that each port provides a complete set of files and features, even if not every user will use every file or feature. For example, we don't declare variants for every possible configure option; we make a best guess as to the features that would be helpful for most users, then provide variants for esoteric features that would require lots of extra disk space or dependencies. I worry that you will expand your request to add features for removing other aspects of ports that you personally don't need.

There are existing Mac apps, outside of MacPorts, that help you reclaim disk space by deleting unwanted translations, architectures, etc. You could probably use those apps to prune the files installed by MacPorts. I'm not convinced we need to duplicate this feature in MacPorts.

[RJVB (René Bertin)]

Replying to ryandesign@…:

Suppose a user wants to take advantage of this feature, but they have already installed a lot of ports. How will they do that?

With a PortGroup-based approach only ports including the PortGroup would be concerned, but more generally speaking, the use of a variant to activate or deactivate pruning you'd have to reinstall the port(s) on which you'd like to apply it. Just like with any other port that you'd want to install with a different set of variants.

There might be an alternative implementation where the pruning takes place in the pre-activate; if that's feasible you would be able to change the language selection without having to uninstall and reinstall; a simple deactivate and reactivate would suffice.

Of course there is also the approach like the one taken by the KDE4 ports: provide translation (sub)ports. With KDE4 that means you install the translations into a given set of languages of *all* translated upstream projects. With KF5 it would probably be possible to make a semi-automatic separation of the main ports and a fixed set of translations for each of those ports. That does lead to a considerable explosion of the number of ports.

For example, we don't declare variants for every possible configure option; we make a best guess as to the features that would be helpful for most users, then provide variants for esoteric features that would require lots of extra disk space or dependencies.

The difference here that we're talking about 1 single variant (or installation option if it were to be integrated into "base") which then allows the user to make a personal selection.

I worry that you will expand your request to add features for removing other aspects of ports that you personally don't need.

No, not this particular request. And you could have understood by now that I only make requests or raise suggestions about things that I do myself when I think they're in the general interest. There's a lot that I keep to my own personal branch...

You could probably use those apps to prune the files installed by MacPorts. I'm not convinced we need to duplicate this feature in MacPorts.

It wouldn't be hard to write something that throws out anything from ${prefix}/share/locale that you'll never use, and such a script could easily be provided as part of a port with useful utilities. But in that case "base" should probably have a feature to prevent the sh*tload of warnings you get about missing translation files each time rev-upgrade is run.

[raimue (Rainer Müller)]

Considering how small these locale files are on my installation (< 300 MiB), I do not think it is worth to invest time into such a feature. The saved disk space would be minimal.