Opened 4 years ago

#60632 new enhancement

Reduce disk writes by extracting distfiles directly from fileserver

Reported by: ryandesign (Ryan Carsten Schmidt) Owned by: admin@…
Priority: Normal Milestone:
Component: buildbot/mpbb Version:
Keywords: Cc:
Port:

Description

I want to reduce disk writes on the buildbot worker machines (#60628) and one idea is to avoid downloading distfiles to the buildworker's local disk.

Currently the buildbot workers download distfiles from a local fileserver over http when needed and they get written to the local disk as usual. Instead, we might be able to mount the distfiles collection read-only from the local fileserver via afp or smb and mpbb could symlink the files into /opt/local/var/macports/distfiles. Ideally this should be resilient against situations where some or all of the distfiles are not on the fileserver or the fileserver is down; in these cases, the worker should be allowed to download the distfiles via http as usual. There could be a launchd plist that establishes the connection to the fileserver at system startup and some method for reconnecting automatically in the event that the fileserver has to be restarted for example.

Or instead of putting this functionality into mpbb we could try to add it as a feature to MacPorts base so that the user can specify in macports.conf possible alternate locations where distfiles can be found. This is probably trickier since variables like distpath are exposed to and used by portfiles and are expected to contain a single path value.

Change History (0)

Note: See TracTickets for help on using tickets.