port install : in need of a progressbar

[pc@…]

Installing bigger Distributions, like texlive_texmf-full (>125MB), I was wondering if everything is allright after some minutes of nothingness. I know that other *nix-packet-managers have a progressbar (some dots or whatever) so that the user is able to track the progress, at least having an idea if everything is allright.

Since now I know I just have to wait, everything is fine for me. Thinking of other users - they would sometimes appreciate to have a progressbar when installing using port, I suppose. Thinking of gentoo-users sitting next to me, I would appreciate that anyway ;)

[pc@…]

Cc Me!

[raimue (Rainer Müller)]

See also #15167.

[milosh@…]

As a pis-aller, you can use the -d option. You will have curl's progressbar while fetching and lots of output while building reassuring you that things are moving.

Note: for texlive_texmf-full, the long phases should be feching and activating (macports is quite slow at activating thousands of files).

[pc@…]

Thx milosh, of course a good idea :)

[p.ixiemotion@…]

Cc Me!

[gguthe@…]

cc me too please

[ryandesign (Ryan Carsten Schmidt)]

We could provide a progress bar of the various MacPorts phases, but different phases take vastly different amounts of time, and it varies by port, so there would be no way to make such a progress bar have accurate proportions.

Within the fetch phase, we could hook into curl's progress meter, if the port fetches standard distfiles. If the port fetches from a VCS, we would have to see if the particular VCS used offers progress information; many don't.

Within the build phase, for most ports, a proper progress bar that shows you how far along the build process you are is impossible because most build systems do not provide that information. cmake is the one exception I know of, which does print percent completed information in its output which could be turned into a progress bar. For all other build systems, all we could do is an indeterminate progress bar which shows that something is happening but would give no indication of how far along the process is.

cmake might print progress information for the destroot phase as well.

For all other phases, I don't think we have any progress information available.

[gguthe@…]

Cc Me!

[cooljeanius (Eric Gallager)]

Cc Me!

[cooljeanius (Eric Gallager)]

r117044 added support for a progress bar for downloads, and r117046 added support for a progress bar for rev-upgrade... it is not quite the full install process yet, but it is getting there...

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #51489.

[neverpanic (Clemens Lang)]

Just so this is documented somewhere… I've had the idea in my head for a while that we could use the number of lines generated on stdout as an indicator of build progress. Since warnings go to stderr, this number of lines should be roughly the same on all installations for a given port at a given version. A webservice could be used to map a hash of (portname, version, variants, OS) to the median number of lines submitted for this tuple. We could encourage users to opt-in to provide this information if base only used the webservice if users also agree to submit this info. The hash would be used to prevent turning this into statistics, since we have other approaches for that.

This approach would work for the configure, build and destroot phases of pretty much all ports, regardless of their build system.

Progress for the operations involving (de-)activation and archiving could be handled per file because we know (or can easily determine) the number of files in a port.

[RJVB (René Bertin)]

That could work, as long as it doesn't happen too frequently that parallel builds start merging lines lllliiiikkkkeeee ssssoooo.

[neverpanic (Clemens Lang)]

Even then, the number of lines breaks (i.e. \ns) stays the same, just the order changes.

[mojca (Mojca Miklavec)]

One problem of line counting is running port build repeatedly (for example after fixing problems, fixing conflicting ports etc.).

[RJVB (René Bertin)]

Yes, and the number of lines generated may also depend on the compiler being used. I'd say that's actually a more severe issue because more likely to affect users in regular use. I think we developers wouldn't really mind if the progress display is wrong during incremental builds if we know why ... and we might even be doing a verbose build anyway in the situations you invoke.

But as I said before, it may not be worth the trouble to try and provide a generic progress reporting mechanism. Or maybe just an "indefinite" display that simply shows something is going on.

Side-thought: open `port logfile foo` opens the log in Console.app; would it be possible to bind that command to a keyboard shortcut in "base"?

[neverpanic (Clemens Lang)]

We could re-use the download progress bars in the configure, build and destroot steps. A first iteration could probably just move a progress indicator (not a progress bar!) when a new line was printed by the build system.

Further iterations could parse CMake output when CMake is used as a build system, for example.

[ryandesign (Ryan Carsten Schmidt)]

Replying to neverpanic:

Further iterations could parse CMake output when CMake is used as a build system, for example.

I would suggest not hardcoding information about cmake's progress output in base, but instead make a way in base for a port or portgroup to specify maybe a regexp to be applied to output lines to find progress information. The cmake portgroup can then set the correct values for cmake. We also need to consider that some builds may have multiple parts. For example, a port that uses cmake and the muniversal portgroup will go from 0% to 100% progress more than once (once for each arch).

[RJVB (René Bertin)]

A first iteration could probably just move a progress indicator (not a progress bar!) when a new line was printed by the build system.

That should be easy, IIRC all output already goes through single Tcl function. It probably should also be a bit more elaborate than updating the indicator on every new line (instead, update every N lines, where N is determined from some sliding average over the number of lines printed per second?).

See my comments above. The way to go is probably via a hook/callback approach where different buildsystems can provide progress information. CMake is probably not the only such system that can provide the required information.

Muniversal builds are an annoyance in this matter, but only if you think you have to hide the fact that such a build is going on. Assuming that a display progress information is to be optional it's logical to assume that users who are interested in this are also not uninterested in knowing what part is being built. (I already patch my local muniversal PG so it tells me that.) The alternative is a simple *approximation*: scale by the number of architectures. If the returned progress information is a (currentFile,numberOfFiles) pair, multiply numberOfFiles by the number of architectures and you're set. When the returned information is already a percentage, divide it by that number and treat it as if on a circular scale (progress from ((100/N -i)% to j% would become (100/N +j)%).

[neverpanic (Clemens Lang)]

Just to make this clear, I think we should first provide progress indicators. Once that's done, we can add a new ticket to figure out how to extract progress information from different build systems.

[landonf (Landon Fuller)]

In da3524e8a3006a9b9a2370b66fa91bb554dd768b/macports-base (master):

Basic target progress bar support (​https://github.com/macports/macports-base/pull/150)

• Add callback event support for monitoring the progress of SystemCmd.
• Use SystemCmd events to implement basic progress bar support for a subset of targets (including build).

Closes: #15939