RFE: allow Portfile to upgrade dependents

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

There are times when a port undergoes such a drastic change that
all its dependents need to be rebuilt.

One such example is when perl is upgraded.
As noted in #16830, all the installed modules need to rebuild so they
are installed in the correct directory.

Attached is a proposed solution.
It allows a port to set a poison_version.
During an upgrade, if the installed version <= poison_version, then all dependents
are forced to rebuild.

[jmroot (Joshua Root)]

I don't think this will behave correctly when multiple ports are being upgraded. Won't all ports upgraded subsequent to one that trips the poison_version check have their dependents force-rebuilt as well?

[jmroot (Joshua Root)]

Ah no, never mind. I forgot the options are local to the upgrade proc.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Upon further consideration, I think the inequality should be strict:
force upgrade if installed_version < poison_version.
That way, revision increases don't force rebuilds.

Uploading new macports.tcl.diff.

[blb@…]

Is macports.tcl.diff against an old version, it's 230 lines off against trunk?

So the usage on this would be (using perl5), when it gets version set to 5.10.0 (or whatever at the time it happens), then poison_version would also be set to that, correct? And for future 5.10.x updates, it'd be left at the same 5.10.0 as originally set so slower upgraders coming from 5.8.x would still have the right thing happen? Being left at 5.10.0 until a (hypothetical) 5.12.0 is released, then updating to 5.12.0.

Then it becomes like epoch, once added it can never be removed.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Replying to blb@…:

Is macports.tcl.diff against an old version, it's 230 lines off against trunk?

I did use an old version (the current release), but based on what I've looked at so far,
nothing new in the trunk is a problem.

So the usage on this would be (using perl5), when it gets version set to 5.10.0 (or whatever at the time it happens), then poison_version would also be set to that, correct?

Correct.

And for future 5.10.x updates, it'd be left at the same 5.10.0 as originally set so slower upgraders coming from 5.8.x would still have the right thing happen?

Yes.

Being left at 5.10.0 until a (hypothetical) 5.12.0 is released, then updating to 5.12.0.

Yes.

Then it becomes like epoch, once added it can never be removed.

If the port maintainer wants to accommodate those slow to upgrade, then yes, it would have to remain.
Removing it, however, should pose no problem to new installations or upgrades at or beyond the poison_version.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Attached (force_upgrade.diff) is a diff file against the trunk.
I also added a poison_revision.

[raimue (Rainer Müller)]

Cc Me!

[febeling@…]

Cc Me!

[febeling@…]

Another example of a failure due to the same cause:

​http://mail-archives.apache.org/mod_mbox/couchdb-dev/200812.mbox/%3c49454ADA.5050908@utoronto.ca%3e

This is quite a general problem, it seems, even if it does hit apparently not too often in practice. But every time a library rolls over a version number which some port chose to link against this should strike.

Wouldn't it be cleaner to rebuild all dependants of a port every time a port upgrades.

I see how this might make small changes to common libraries prohibitively expensive. Is there some good general practice for cases like this maybe? FreeBSD and Gentoo should have the same problem, what do they do?

[febeling@…]

Another example where this problem hits is when ruby is being upgraded. Native extensions live in a path like this: /opt/local/lib/ruby/1.8/i686-darwin9. This used to include also minor and teeny version, but they are removed during build for both ruby (1.8.7) and ruby19 (1.9.1) now. This was devised as a means to make this problem just strike much less often. But it's only a workaround in my opinion. site_ruby and vendor_ruby locations also each contain arch subdirectories, so this applies for gems and other third-party libraries as well. And in the event of a major version rollover we still get LoadErrors in the ruby case, for which it can be quite tough to understand that you have to reinstall all your ruby libraries with native extensions.

This just to give more context to this issue.

[febeling@…]

Any package manager that relies on the download-and-build pattern has the problem that it can't really tell if they break dynamically linked binaries/libs when they upgrade libraries (because of new path or new lib name). Unless they rebuild all dependents every time, which is way too expensive, that is. Gentoo is one example of an OS with this style of packages. This is how they solve the problem:

​http://www.gentoo.org/doc/en/gentoolkit.xml#doc_chap4

Basically, there is a tool (revdep-rebuild) which scans through binaries and checks if dynamic linking still works everywhere. If not, you can rebuild only those packages which are marked, so that they link against existing libraries again.

[jmroot (Joshua Root)]

This year's GSoC should give us a solution to this problem that requires less manual intervention.