Autoremove macports/software not to waste space with archived duplicates

[esbugz]

After installing some ports I realized there is this huge duplicate folder containing .tbz2 archives, which from the Guide:

software Contains the compressed archives of installed software. If a port is activated its files are extracted to the ${prefix} folders from the compressed files here. port uninstall <port> <version>+<variant> would remove it from here, but then this port can not be used anymore.

So this offers no rationale why this duplication exists, nor a practical convenient solution for cleaning it. I'd like to delete all of them, but still use the ports

This old SO post ​https://apple.stackexchange.com/questions/64074/macports-software-clean-up-opt-local-var-macports-software is similar: no rationale, no practical convenient solution

Homebrew has a command that allows you to delete the zips and it also runs the command automatically

I'd like to have something similar in MacPorts:

• ideally a config that would delete all the duplicates right away after a successful install
• a command to delete all the archives in the software folder manually
• maybe a default of running such a command every X days or something?

P.S. same thing should be done to distfiles

[kencu (Ken)]

I presume you know about:

sudo port -v reclaim

that cleans up your system of expired software you have upgraded, partial builds, and other cruft. It will automatically run every two weeks if you agree to that.

But it won't remove all the archives from software you have installed, stored in the software folder. To be honest, I thought this was a requirement of how macports worked -- it is used to reinstall software you already have installed, as lots of folks have had to build those and wouldn't be happy to go through the whole rebuild process.

It would be easy enough to test that. If MacPorts can work properly with the contents of that folder moved or deleted, and it will just rebuild or redownload what is needed, then emptying that software folder might something that could be added to the port reclaim mechanism.

But I am not certain MacPorts will work right if that folder is deleted -- I can see how it might malfunction -- so it would need some testing to see.

Homebrew in some ways has it much easier, a consequence of restricting your options and flexibility - it does not support you building your own software, for example - so it can make some assumptions that MacPorts cannot make.

[kencu (Ken)]

By the way, almost everything I have seen on stack overflow about MacPorts is ancient, out-of-date, and mostly useless information that is more misleading than helpful.

I would take anything you see there about MacPorts with a big lump (not grain) of NaCl.

[esbugz]

What's the up-to-date source? The official Guide ​https://guide.macports.org/#project.tickets provides me with this very useful information on how to pick one of the many outdated archive formats

Binary archive files are placed in ${prefix}/var/macports/software/. The archive file type is set in macports.conf using the portarchivetype key.
The default format is tbz2; other options are: tar, tbz, tbz2, tgz, tlz, txz, xar, zip, cpgz, and cpio.

but not how to clean the backups for the mostly unneeded functionality of multiple port versions (which I think is anyway better solved via symlinks that would link to either version1 or version2 of the port instead of having each port have a mandatory backup)

[esbugz]

Replying to kencu:

I presume you know about:

sudo port -v reclaim

Yes, though I'd prefer this to also autorun on a schedule rather than having to run it myself. Basically, I don't see much value in these backups, I almost never need them and when I do, a redownload is much easier since it's ore likely the package has been updated by then anyway, so the backup is stale

But it won't remove all the archives from software you have installed, stored in the software folder.

Yeah, that's the biggest one — I need to clean the versions that I actually use, not just the unused ones

To be honest, I thought this was a requirement of how macports worked -- it is used to reinstall software you already have installed, as lots of folks have had to build those and wouldn't be happy to go through the whole rebuild process.

Sure, but now the main use case is just download prebuilt stuff, and even for the build artifacts that should be optional as in my experience this is also very rarely needed to justify having it by default (but if someone wants to keep them, that's also fine)

It would be easy enough to test that. If MacPorts can work properly with the contents of that folder moved or deleted, and it will just rebuild or redownload what is needed, then emptying that software folder might something that could be added to the port reclaim mechanism.

I've already deleted the folder so already testing it, seems fine :) And the SO implied it's fine, so it should be fine ;) (but you're right, they are pretty old, so shouldn't rely on it much)

But I am not certain MacPorts will work right if that folder is deleted -- I can see how it might malfunction -- so it would need some testing to see.

Why would it? Anyway, the proper way would be to fix the cause of the malfunctioning, the whole design of mandatory backup isn't great

Homebrew in some ways has it much easier, a consequence of restricting your options and flexibility - it does not support you building your own software, for example

But it does, I've built a few formulas myself from the git HEAD versions when I needed a quick fix before it lands in the main formula. Yes, Homebrew removed the many options previously available in their main repository (very bad decision), but not the ability to build

• so it can make some assumptions that MacPorts cannot make.

Which ones? Even when I was building the latest HEAD versions, I never needed a backup since I'd just never uninstall it to require a full rebuild (and hence needing a backup).

[kencu (Ken)]

The official policy of homebrew is that they will not support users building their own software. This has been stated hundreds of times when people post up issues.

[kencu (Ken)]

To really test what happens when you delete the "software" folder, you'd need to do some digging... like try to deactivate and reactivate a port, and try to rebuild something with dependencies, especially toolchain components, and try it in a non-default prefix where the prebuilt binaries can't be used, and try to rebuild something where the default stdlib has been changed to "libstdc++" from "libc++" for example.

These are all the kinds of assumptions that homebrew never has to make, because they don't really support any of this.

[ryandesign (Ryan Carsten Schmidt)]

You're new here—you filed your first MacPorts ticket (and joined GitHub) 3 days ago. MacPorts has existed for 20 years and so there is a long history to explain why MacPorts works the way it does, in many cases with good reason, and for many of the suggestions you've casually made in this and other tickets, there would be substantial effort to make the changes you suggest, with significant consequences. I recommend instead of filing more tickets for further substantial changes, you write to the macports-dev mailing list to discuss your suggestions. Perhaps there you will get a bigger audience that can explain, for each of your suggestions, why things are the way they are.

[esbugz]

Replying to kencu:

The official policy of homebrew is that they will not support users building their own software. This has been stated hundreds of times when people post up issues.

Regardless of the "official policy", Homebrew does allow building from source, so this in itself is not a reason for the difference in behavior

Replying to kencu:

To really test what happens when you delete the "software" folder, you'd need to do some digging... like try to deactivate and reactivate a port

Currently deactivating deletes the port, though the guide is a bit confusing as to what's supposed to happen

it deactivates it, i.e., it stashes the files belonging to the older version away in a tarball.

This suggests that the files are not deleted, but move to an archive

Deactivating a port only removes the files from their activated locations (usually under ${prefix})—the deactivated port's image is not disturbed.

This says they're removed

So you could fix this by zipping the port on deactivation and putting it back to /software if it's empty, or you could skip autodelete to flagged ports (or automatically to ports with multiple versions), but the cleaner solution would be similar to what Homebrew does, so activating/deactivating would mean symlinking to the port's install folder rather than moving the files while leaving a backed up duplicate version behind

try it in a non-default prefix where the prebuilt packages can't be used and try to rebuild something where the default stdlib has been changed to "libstdc++" from "libc++" for example.

That defeats the core benefit of MacPorts, but then it's also fine just disabling autoremove functionality in such cases and let the user opt in

These are all the kinds of assumptions that homebrew never has to make, because they don't really support any of this.

Sure, they strongly advise against it and will close your issues, but them some people still installed to their home folder, for example, with the same huge downside - you'd have to rebuild everything from source instead of using the prebuilt packages from their cache.

But even then, none of this assumptions affect the most convenient use case of installing with the default prefix and relying on prebuilt packages, so none of them block removing the waste

[kencu (Ken)]

ok, your understanding of what happens is pretty incomplete.

I’m going to close this ticket as this is not the forum for a multiple-point running debate, and you can take this to macports-dev as the admin suggested.

[esbugz]

Replying to ryandesign:

MacPorts has existed for 20 years and so there is a long history to explain why MacPorts works the way it does, in many cases with good reason

And in many cases without a good reason; or with reasons that were only good 20 years ago. So, how is this of any help in discussing specific issues?

instead of filing more tickets for further substantial changes, you write to the macports-dev mailing list to discuss your suggestions.

But the mailing is a much worse substitute even to this outdated forum since:

• you have a per-issue-page anyone can track (once they find the tiny cc button ;))
• you see the whole discussion thread instead of having to click "next/previous" message to follow
• you have access to the WikiFormatting markup

Perhaps there you will get a bigger audience that can explain, for each of your suggestions, why things are the way they are.

And how would that help solve the issue?

[esbugz]

Replying to kencu:

ok, your understanding of what happens is pretty incomplete.

Ha-ha.

I’m going to close this ticket as this is not the forum for a multiple-point running debate

Then don't debate? How is it relevant to whether the feature request is valid?

, and you can take this to macports-dev as the admin suggested.

No, thanks, I've wasted enough time as it is, if you're so unresponsive to such obvious issues, why would I engage in an even less convenient avenue that a dedicated forum?