set ${prefix} in portindex

[RJVB (René Bertin)]

It would be nice if the portindex script set ${prefix}, so that code that is conditional on the existence of specific files also works during port indexing.

[mf2k (Frank Schima)]

Note that a "request" ticket ticket is only for requesting a new port.

[ryandesign (Ryan Carsten Schmidt)]

Can you give an example?

[RJVB (René Bertin)]

One example would be with port families like python, perl or Qt5 which all share (are served by) a common PortGroup. If that PortGroup or the Portfiles need to do something that depends on what the user has installed it is a lot easier (and probably faster) to check for the presence of an installed file that to query the registry. (file exists is a Tcl primitive and doesn't need catching.)

One application would be port:qt5 vs. my pending port:qt5-kde . These have partial PortGroup sharing, but the main payload sits in dedicated PG files which are loaded as a function of what is installed.

It would be trivial to work around $prefix not being declared in/by portindex, but not elegant. Is there a solid reason why $prefix and possibly other useful and often-used variables aren't set when portindex is run?

[ryandesign (Ryan Carsten Schmidt)]

I don't know why prefix isn't declared by/in portindex, but the fact that we haven't needed it for the past decade suggests you should do what you're doing a different way, or shouldn't do it.

For example, the portindex should generate identically on every system (of the same OS version and cxx_stdlib setting); it should not generate differently depending on what ports might be installed or what files might be present.

[RJVB (René Bertin)]

I was waiting for that kind of remark ...

The point here is not being able to generate a prefix-dependent port index (to the extent that dependency and conflict information isn't stored there). The point is just avoiding an abort because of a variable that's not not declared but that's set to a bogus value.

I realise I made that error myself: what's happening is that prefix expands to ${prefix}. If the variable weren't set at all a large number of ports wouldn't index, of course.

This obliges me to do

if {[file exists ${prefix}/include/qt5/QtCore/QtCore] || ${os.major} == 10
        || ([catch {registry_active "qt5-kde"}] == 0 || [catch {registry_active "qt5-kde-devel"}] == 0) } {

instead of

if {[file exists ${prefix}/include/qt5/QtCore/QtCore] || ${os.major} == 10} {

Methinks that if portindex can set prefix to \$\{prefix\} it could also just set it to the actual value.

[ryandesign (Ryan Carsten Schmidt)]

If portindex is seeing that if statement, it must be in the main part of the portfile, not inside a phase. What are you doing in response to this if statement? If you're causing the port to, for example, declare different dependencies based on what other ports are already installed, then I would say we do not want to do that, because it would cause the index to be generated differently on different systems.

[RJVB (René Bertin)]

Hmmm, yes, but in this case that's unavoidable I fear, and already introduced a while ago independent of the issue I brought up here.

If the user choses to install port:qt5x-qtbase (or qt5-kde) and later needs to add, say, QtWebEngine, he'd need to install qt5x-qtwebengine (or qt5-kde-qtwebengine) and the matching dependencies of that port. So, for instance, qt5x-qtlocation and not some other qt5y-qtlocation -- and when he installed qt5-kde most Qt components are actually provided by that main port.

This may be novel behaviour, but as I said I don't see how we can do this any other way (that doesn't increase complexity enormously at least). We'd probably be facing the same thing for ports written in Python or Perl if those didn't install to version-specific locations. At the same time those ports can use any of the installed Python, Perl (or LLVM or ...) ports while only 1 version of Qt5 can be installed and active at a given time (and Qt guarantees backwards ABI compatibility).

The final dependency computation has to run the Portfiles anyways because of the build variant selection, no? IOW, build dependencies are host/user dependent even if they're not always different from the default?!

What kind of probem do you think a host-dependent PortIndex could cause? They're already OS version dependent and there can be any number of PortIndex files, so I'd assume they're already tested for robustness in this domain, AND they have to be generated locally anyway.

[ryandesign (Ryan Carsten Schmidt)]

portindexes are generated on the server, once per OS version/arch combination, and, if the user receives their ports tree via rsync, is distributed to the user when they sudo port sync or sudo port selfupdate. The portindex that is generated must therefore be suitable for all users of that OS version/arch combination, and cannot vary based on what might be already installed on the system.

[RJVB (René Bertin)]

So how does that finagle out for build variants (those which modify the dependency list)?

[neverpanic (Clemens Lang)]

The portindex does not contain information regarding dependencies of variants.

Btw, I accidentally changed the value of $prefix from its current value ${prefix} during portindex back in the day. It went unnoticed for quite a while and then we discovered that it was actually causing problems and we were relying on the value being ${prefix}, so I don't think we can change this here. See [9a23db2fde81660ca42425ae0647a4f347571f45/macports-base].

I'd also argue that we shouldn't change it if there is a simple alternative available, as there is in this case (just use registry_active as you already do).

In addition to that, I completely agree with Ryan that Portfiles must not change behavior depending on what the user has installed on their system other than changing default variants, but that seems to be a lesson you seem to refuse to learn, so I won't waste my time on repeating it any further.

[RJVB (René Bertin)]

Replying to neverpanic:

The portindex does not contain information regarding dependencies of variants.

Exactly my point. There is thus already a potential discrepancy between the dependencies for a to-be-installed port as provided by the portindex and as actually required.

I'm not going to waste anything on this anymore either.

[neverpanic (Clemens Lang)]

Replying to RJVB:

Replying to neverpanic:

The portindex does not contain information regarding dependencies of variants.

Exactly my point. There is thus already a potential discrepancy between the dependencies for a to-be-installed port as provided by the portindex and as actually required.

The contents of the portindex are used for informational and search-operator purposes only, where this is a valid tradeoff between speed and correctness. So far, in all of our ports, the dependency info in the portindex was correct for default variants. Your code changes that.

I'm not going to waste anything on this anymore either.

Closing wontfix then.

[RJVB (René Bertin)]

Replying to neverpanic:

The contents of the portindex are used for informational and search-operator purposes only, where this is a valid tradeoff between speed and correctness. So far, in all of our ports, the dependency info in the portindex was correct for default variants. Your code changes that.

Final remark: my code *could* change that for default variants, but that's not the goal and I'm associating a variant with the depspec change in my own ports. The approach has been discussed on the ML a while back, and seemed acceptable.