MacPorts reuses previous OS portindex after OS upgrade if no portindex is available via rsync

[captainproton1971 (Captain Proton)]

After completing migration steps ​https://trac.macports.org/wiki/Migration, my MacPorts installation has been mostly reconstituted. However, trying to update ports isn't working... just scans for linking errors.

√ [my_username] ~ % port outdated                                                                                                                                    7:25:59
The following installed ports are outdated:
cyrus-sasl2                    2.1.27_3 < 2.1.27_4       
zstd                           1.5.0_0 < 1.5.0_1         
√ [my_username] ~ % sudo port -v upgrade outdated                                                                                                                    7:26:05
--->  Scanning binaries for linking errors
--->  No broken files found.
--->  No broken ports found.
√ [my_username] ~ % sudo port upgrade -f zstd                                                                                                                        7:26:19
--->  Scanning binaries for linking errors
--->  No broken files found.
--->  No broken ports found.
√ [my_username] ~ % sudo port upgrade -f cyrus-sasl2                                                                                                                 7:26:31
--->  Scanning binaries for linking errors
--->  No broken files found.
--->  No broken ports found.
√ [my_username] ~ % port outdated                                                                                                                                    7:26:48
The following installed ports are outdated:
cyrus-sasl2                    2.1.27_3 < 2.1.27_4       
zstd                           1.5.0_0 < 1.5.0_1         

I also tried uninstalling+reinstalling and port -nR upgrade --force as suggested on ​http://guide.macports.org/#project.tickets. The latter led to many hours rebuilding, but left zstd and cyrus-sasl2 outdated.

[ryandesign (Ryan Carsten Schmidt)]

Show the output of port installed cyrus-sasl2 zstd. If you somehow already have the updated versions installed, but they are not the active versions, MacPorts will assume you did that intentionally and will not activate the newer versions automatically. But you can activate them manually if you want (i.e. sudo port activate zstd @1.5.0_1.

[captainproton1971 (Captain Proton)]

Thanks, Ryan. But I don't think that's the issue:

√ [my_username] ~ % port installed cyrus-sasl2 zstd                         17:21:32
The following ports are currently installed:
  cyrus-sasl2 @2.1.27_3+kerberos (active)
  zstd @1.5.0_0 (active)
√ [my_username] ~ % port outdated                                           17:22:03
The following installed ports are outdated:
cyrus-sasl2                    2.1.27_3 < 2.1.27_4       
zstd                           1.5.0_0 < 1.5.0_1         
√ [my_username] ~ % sudo port upgrade outdated                              17:25:36
Password:
--->  Scanning binaries for linking errors
--->  No broken files found.                             
--->  No broken ports found.
√ [my_username] ~ % port outdated                                           17:26:08
The following installed ports are outdated:
cyrus-sasl2                    2.1.27_3 < 2.1.27_4       
zstd                           1.5.0_0 < 1.5.0_1         
√ [my_username] ~ % port installed cyrus-sasl2 zstd                         17:27:28
The following ports are currently installed:
  cyrus-sasl2 @2.1.27_3+kerberos (active)
  zstd @1.5.0_0 (active)

[ryandesign (Ryan Carsten Schmidt)]

Agreed. Then I don't know what's wrong.

[captainproton1971 (Captain Proton)]

I'm thinking this is an issue with the individual ports, not MacPorts base. I had assumed that given the two unrelated ports were all that needed updating is that somehow MacPorts wasn't updating correctly. However, , some other ports (p5.28-net-ssleay, p5.30-net-ssleay, and p5.32-net-ssleay) were later updated just fine.

I've tried forced uninstall + reinstall with specified version (see below) and still had the ports showing as outdated.

  √ [my_username] ~ % sudo port uninstall -f cyrus-sasl2 zstd && sudo port install cyrus-sasl2@2.1.27_4 +kerberos zstd@1.5.0_1 && port outdated                                                18:59:24
--->  Unable to uninstall cyrus-sasl2 @2.1.27_3+kerberos, the following ports depend on it:
--->  	subversion @1.14.1_0+tools
Warning: Uninstall forced.  Proceeding despite dependencies.
--->  Deactivating cyrus-sasl2 @2.1.27_3+kerberos
--->  Cleaning cyrus-sasl2
--->  Uninstalling cyrus-sasl2 @2.1.27_3+kerberos
--->  Cleaning cyrus-sasl2
--->  Unable to uninstall zstd @1.5.0_0, the following ports depend on it:
--->  	curl @7.79.1_0+ssl
--->  	rsync @3.2.3_0
--->  	tiff @4.3.0_0
Warning: Uninstall forced.  Proceeding despite dependencies.
--->  Deactivating zstd @1.5.0_0
--->  Cleaning zstd
--->  Uninstalling zstd @1.5.0_0
--->  Cleaning zstd
--->  Computing dependencies for cyrus-sasl2
--->  Fetching archive for cyrus-sasl2
--->  Attempting to fetch cyrus-sasl2-2.1.27_3+kerberos.darwin_21.x86_64.tbz2 from https://packages.macports.org/cyrus-sasl2
--->  Attempting to fetch cyrus-sasl2-2.1.27_3+kerberos.darwin_21.x86_64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/cyrus-sasl2
--->  Attempting to fetch cyrus-sasl2-2.1.27_3+kerberos.darwin_21.x86_64.tbz2 from https://ema.uk.packages.macports.org/cyrus-sasl2
--->  Fetching distfiles for cyrus-sasl2
--->  Verifying checksums for cyrus-sasl2
--->  Extracting cyrus-sasl2
--->  Applying patches to cyrus-sasl2
--->  Configuring cyrus-sasl2
--->  Building cyrus-sasl2                               
--->  Staging cyrus-sasl2 into destroot                  
--->  Installing cyrus-sasl2 @2.1.27_3+kerberos          
--->  Activating cyrus-sasl2 @2.1.27_3+kerberos
--->  Cleaning cyrus-sasl2
--->  Computing dependencies for zstd
--->  Fetching archive for zstd
--->  Attempting to fetch zstd-1.5.0_0.darwin_21.x86_64.tbz2 from https://packages.macports.org/zstd
--->  Attempting to fetch zstd-1.5.0_0.darwin_21.x86_64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/zstd
--->  Attempting to fetch zstd-1.5.0_0.darwin_21.x86_64.tbz2 from https://ema.uk.packages.macports.org/zstd
--->  Fetching distfiles for zstd
--->  Verifying checksums for zstd
--->  Extracting zstd
--->  Applying patches to zstd
--->  Configuring zstd
--->  Building zstd
--->  Staging zstd into destroot                         
--->  Installing zstd @1.5.0_0                           
--->  Activating zstd @1.5.0_0
--->  Cleaning zstd
--->  Scanning binaries for linking errors
--->  No broken files found.
--->  No broken ports found.
The following installed ports are outdated:
cyrus-sasl2                    2.1.27_3 < 2.1.27_4       
zstd                           1.5.0_0 < 1.5.0_1         

Thanks for the suggestions, I think this is probably related to the individual ports.

[jmroot (Joshua Root)]

See if port -v installed shows anything different or unusual for the affected ports.

[jmroot (Joshua Root)]

Has duplicate #63703. As I said there, I guess you're probably using a locally generated PortIndex that was originally generated on a different OS version than you're now running on? That would interact badly with changing the revision on a per-platform basis as these two ports do. It's at least only a cosmetic issue; the ports are not upgraded because they are already the latest version for your current platform.

[i0ntempest]

This can be fixed by nuking PortIndex* and ports* at /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ and letting port regenerate them for you by using selfupdate. It’s going to take a long time.
I didn’t think of this as I thought portindex was always rsynced from the servers…

[jmroot (Joshua Root)]

If you're syncing with rsync, the PortIndex will be synced as well, as long as one for your platform exists on the server. But there isn't one there for darwin 21 yet.

[captainproton1971 (Captain Proton)]

Thank you for the info on this. I'm not sure what is meant by a 'locally generated PortIndex'. I had a MacPorts installation on my laptop that was running Big Sur (PortIndex was presumably rsync'ed), followed the steps on the Migration page, and I'm now experiencing the reported issue. This hasn't happened in previous during migrations arising from OS updates in the past... did I just get lucky then?

"the ports are not upgraded because they are already the latest version for your current platform." Great, thanks.

Is there a way to remediate this in the short term (I'm assuming it will fix itself once a Darwin 12 PortIndex exists on the server)? Can steps be added to the Migration instructions to avoid this in the future?

Thanks!

[FaradayLight (Faraday Light)]

It might be of little help, but I updated these two packages successfully on macOS Big Sur 11.6 arm64:

% port installed | grep cyrus
  cyrus-sasl2 @2.1.27_3+kerberos
  cyrus-sasl2 @2.1.27_4+kerberos (active)

% port installed | grep zstd 
  zstd @1.5.0_0
  zstd @1.5.0_1 (active)

[ryandesign (Ryan Carsten Schmidt)]

This issue is apparently limited to those users upgrading from Big Sur to Monterey, because these two ports have an increased revision only on Big Sur. I did this because the rebuild that this revision increase caused was exclusively intended to fix a Big Sur-specific problem and I didn't want to force users of other systems to rebuild the ports for no reason. The problem will resolve itself whenever in the future the revision or version of these ports is increased for any other reason thereby removing the inconsistency.

Darwin 21 portindexes should exist on the servers now. Does the problem still persist after sudo port sync? My expectation is that it does not.

It probably would be a good idea for MacPorts base to be able to know if the portindex it has is appropriate for the current OS arch and version, and if not, throw it away and make a new one. It sounds like what it does now is use and update the existing portindex on the computer which is for the user's previous OS version. This is wrong not just for the specific issue mentioned in this ticket but for any number of other reasons. Until such a change is made in base, advising users to do so manually in the migration instructions isn't a bad idea.

[captainproton1971 (Captain Proton)]

Thank you so much for the explanation. Indeed, upgrading my ports this morning has fixed the problem for me.

[ednl (Ewoud Dronkert)]

This problem does persist for me with librsvg 2.40.20_4 < 2.52.3_0 on Monterey arm64 after sudo port -d sync which used ftp.fau.de. Is that mirror not yet up to date, perhaps? Compare:

$ port -v installed | grep librsvg
  librsvg @2.40.20_4 (active) requested_variants='' platform='darwin 21' archs='arm64' date='2021-11-04T14:12:02+0100'

$ port list librsvg
librsvg                        @2.52.3         graphics/librsvg

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #64076.

[ryandesign (Ryan Carsten Schmidt)]

The problem was fixed in cyrus-sasl2 in [5848d16670ddcf04e06163a1800e00a9b5606c50/macports-ports] when it was bumped to the same revision for all OS versions.

I will fix the problem in zstd by revbumping it on Monterey.

Replying to ednl:

This problem does persist for me with librsvg 2.40.20_4 < 2.52.3_0 on Monterey arm64 after sudo port -d sync which used ftp.fau.de. Is that mirror not yet up to date, perhaps? Compare:

ftp.fau.de is the public master from which all the other mirrors get their updates.

$ port -v installed | grep librsvg
  librsvg @2.40.20_4 (active) requested_variants='' platform='darwin 21' archs='arm64' date='2021-11-04T14:12:02+0100'

$ port list librsvg
librsvg                        @2.52.3         graphics/librsvg

$ port outdated
The following installed ports are outdated:
librsvg                        2.40.20_4 < 2.52.3_0      
$ sudo port upgrade outdated
--->  Scanning binaries for linking errors
--->  No broken files found.
--->  No broken ports found.

Edit: and today two more:

gcc-devel                      12-20211028_0 < 12-20211031_0
libgcc-devel                   12-20211028_0 < 12-20211031_0

librsvg, gcc-devel, and libgcc-devel use build_arch to determine which port version to use, which is not permitted. Ports must use os.arch for such a determination. I can fix this in a moment.

[ryandesign (Ryan Carsten Schmidt)]

In 8f255fd58f959cffb67a94f286af21f67023143e/macports-ports (master):

zstd: Revbump on Monterey and later

Fixes PortIndex issues for users who upgraded from Big Sur to Monterey
and do not receive our server-generated PortIndex.

See: #63685

[ryandesign (Ryan Carsten Schmidt)]

In 3891aebaf6bb192124d35f507653b78731a86cb5/macports-ports (master):

gcc-devel: Use os.arch instead of build_arch

Distinguish the OS architecture using os.arch instead of build_arch
because unique PortIndexes are only created on the basis of os.arch not
build_arch.

See: #63685

[ryandesign (Ryan Carsten Schmidt)]

Chris, I can't come up with a quick fix for librsvg because it wants to offer different versions for i386 vs x86_64 and there are not separate PortIndexes at that level of granularity. (There is only one PortIndex for any Intel system for each OS version.) Also, as you already know the existing code does not work for the universal variant, so it needs a rewrite for that anyway.

[ryandesign (Ryan Carsten Schmidt)]

My half-serious suggestion for librsvg is to downgrade to the last non-rust version for all users and never update it again, given the insanity we've had to go through to accommodate its rust version so far.