Stray .TMP files on distfiles mirrors

[ryandesign (Ryan Carsten Schmidt)]

It seems that sometimes the distfiles mirror gets interrupted while downloading a file, and then a ​.TMP file gets left behind. This is a bit untidy and it would be nice if the .TMP files could be cleaned up automatically so that they don't appear when browsing the distfiles servers' directory listings.

Perhaps a cron task could be installed to periodically clean it up. For example, it could run this command:

find /path/to/distfiles -type f -name '*.TMP' \! -newermt '1 hour ago' -delete

This would find and delete files in /path/to/distfiles whose names end with ".TMP" and that were last modified more than an hour ago.

If you just want to see what files would be deleted without deleting them, remove "-delete" from the command:

find /path/to/distfiles -type f -name '*.TMP' \! -newermt '1 hour ago'

The "1 hour" figure is picked a bit at random, but you want some nonzero value in there; you wouldn't want to clean up .TMP files that are currently being downloaded.

See also #44789.

[ryandesign (Ryan Carsten Schmidt)]

Could this script be implemented? It would be nice to clean up the distfiles directories, and it should be just as simple as the one-liner already provided in the ticket description.

[larryv (Lawrence Velázquez)]

Cc Me!

[mojca (Mojca Miklavec)]

One further suggestion:

• I believe that rsyncd can be configured not no serve certain files
• Apache can be tricked into hiding certain files as well

So even if those files do show up, they still won't be visible (and ideally not even transferred to other mirrors). Running a cleanup is still a good idea.

[ryandesign (Ryan Carsten Schmidt)]

Yes, rsyncd and httpd can be configured in the way you suggest. But I don't see any need to do so.

The new distfiles mirroring script, which I want to deploy soon, will not create .TMP files.

Once that script is deployed, I will remove the existing .TMP files.

[ryandesign (Ryan Carsten Schmidt)]

I've done a one-time removal of .TMP files from the distfiles server. Once mirroring is set up again (#53347) we should make sure that we don't preserve .TMP files.