jmpp's proposal to add logging support to the MacPorts infrastructure

This proposal aims to add full logging support to the MacPorts infrastructure, so that a record of all of the activities of a port build run and its results is output upon request to a file writtten in a well known format, such as html, xml or yaml, and kept around for later usage, like user <--> port maintainer feedback or posting of the log to a web based form for automated processing of the build information, among many other potential uses.

The primary goal of the proposal is to provide all kinds of accountability while bulding ports in an automated fashion, when there's no one around to keep an eye open and readily figure out "what went wrong" when an error is encountered. With the code providing this new feature in place, the port(1) command line utility would initialize a log file whenever a non-informational action (a "target action") is requested for a port; that is to say, only actions acting on the port toward building & installing it would append to (or initialize) the log file, whereas informational actions querying the Portfile and/or ports index to provide information to users would not. Logs would be kept on a per port basis most probably in a hierarchy under the ${prefix}/var/macports directory, each accumulating all actions for the corresponding port as they are executed (whether in regular, serial operation fashion or if called separately), unless the user explicitly prunes the log and thus forces MacPorts to initialize a new one. A new port(1) command line level action, "log", would take care of displaying an existing log for port <port> through a terminal pager, "port log <port>", whereas a new "clean" subaction (action-flag) would prune it, "port clean --log <port>", just as the existing "--all" would as well. Informational command line options -v & -d could inform the user where the log file is being written to upon initialization, and what log is being accessed at read time, among other verbose and debugging information.

The log format itself could be a versioned work in progress we would refine as we work toward polishing the logging feature, governing it by a public specification that defines what metadata we output to the log and how. Said specification itself would be governed by us in turn, The MacPorts Project. On the other hand, eligible file formats for the log could be:

• one record per line plain text with pound sign delimted comments (the logging specification version being one of them);
• html, with Id's and/or classes delimiting each of the insallation stages;
• xml, a schema or dtd version defining the logging specification;
• yaml or others.

Well placed code abstractions could buy us some flexibility in separating our own log format specification (build information and metadata we output and how) from the file level format we choose (plain text, html, xml, etc), and in switching from one to another should the need ever arise. Two crucial factors in deciding what file level format we end up choosing, however, are the amount of metadata we decide to output to the log (e.g. MacPorts release, host platform, compiler version, requested port, requested action, built with root permissions? etc.) and the flexibility of the chosen file format to append data to an already open file.

A second part of the proposal would comprise the design of a web based application, written in widely adopted and web oriented scripting language like PHP or Ruby on Rails, that would accept a submitted log and process it in specific ways to provide in a clear and concise fashion as much information as possible about a particular build run to reviewers, openly disclosing all success and error status and, in case of the latter, their reported causes thereof. Said application would likely be hosted on some central and official location endorsed by The MacPorts Project, so as to provide the general user base with authoritative information on the state of our ports tree at any given time. A protocol governing who is allowed to submit logs to this central location would need to be developed, thus reducing the likelyhood of hosting and dispatching potentially fraudulent information. Regular users could still host their own logs and use them, for example, to communicate with maintainers and provide them with higher quality feedback. A command line level "submit" action-flag for the new "log" action would take care of submitting the log for port <port> to the passed value, e.g. "port log --submit maintainer <port>" or "port log --submit <central-location> <port>", where the "maintainer" parameter would expand to the maintainer of <port> and only authorized maintainers (probably holding a private digital signature) would be allowed to submit to the URL of the central location.

In its completed form, this proposal would pave part of the way to construct the software infrastructure that's needed to implement in a reliable fashion the massive build runs that are necessary to finally achieve the ultimate goal of distributing binary packages, since proper error reporting and handling is crucial in case something goes wrong while building in automated fashion a single port out of many in a large tree.

-jmpp

PS: Ticket tracking this request is #2020.