ports tree on rsync is stale

[barracuda156]

Synchronizing local ports tree from rsync://rsync.macports.org/macports/release/tarballs/ports.tar

Willkommen auf dem RSYNC-server auf ftp.fau.de.
Nicht all unsere Mirror sind per rsync verfuegbar.

Welcome to the RSYNC daemon on ftp.fau.de.
Not all of our mirrors are available through rsync.


receiving file list ... done
./
ports.tar
ports.tar.rmd160

sent 72890 bytes  received 20164 bytes  5317.37 bytes/sec
total size is 108035584  speedup is 1161.00

Willkommen auf dem RSYNC-server auf ftp.fau.de.
Nicht all unsere Mirror sind per rsync verfuegbar.

Welcome to the RSYNC daemon on ftp.fau.de.
Not all of our mirrors are available through rsync.


receiving file list ... rsync: change_dir "/release/tarballs/PortIndex_darwin_10_powerpc" (in macports) failed: No such file or directory (2)
done

sent 50 bytes  received 9 bytes  23.60 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-40/rsync/main.c(1400) [receiver=2.6.9]
Command failed: /usr/bin/rsync -rtzvl --delete-after --include=/PortIndex.rmd160 --include=/PortIndex --exclude=* rsync://rsync.macports.org/macports/release/tarballs/PortIndex_darwin_10_powerpc/ /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs
Exit code: 23
Creating port index in /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ports
Adding port sysutils/di

Total number of ports parsed:	1 
Ports successfully parsed:	1 
Ports failed:			0 
Up-to-date ports skipped:	33467

Same problem on 10.6 PPC, so it is not a specific machine issue. Day before yesterday all worked fine.

[barracuda156]

Hmm, does not work on Tiger either, it seems.

[ryandesign (Ryan Carsten Schmidt)]

There never has been and never will be a file PortIndex_darwin_10_powerpc on the server because Apple never released that OS for that processor.

Failure to get a PortIndex from the server should not be a problem. MacPorts should generate or update the PortIndex locally.

What error do you get on Tiger?

[barracuda156]

Replying to ryandesign:

There never has been and never will be a file PortIndex_darwin_10_powerpc on the server because Apple never released that OS for that processor.

Failure to get a PortIndex from the server should not be a problem. MacPorts should generate or update the PortIndex locally.

What error do you get on Tiger?

Yes, of course there is no 10.6 ppc index. What problem I face now is that ports do not get actually synced at all from Macports servers. Data is downloaded, sync does not happen: old ports are still there. Same on Tiger.

There is no specific error, they just do not sync with remote ones. Local ports are synced, as usual.

I.e., like above:

sent 72890 bytes  received 20164 bytes  5317.37 bytes/sec
total size is 108035584  speedup is 1161.00

But then:

Creating port index in /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ports
Adding port sysutils/di

Total number of ports parsed:	1 
Ports successfully parsed:	1 
Ports failed:			0 
Up-to-date ports skipped:	33467

There are obviously much more ports updated in reality, but they do not get updated.

Nothing was changed in my local config, and same problem happens on three machines (10.6.8 Rosetta, 10.6 PPC, 10.4.11).

[catap (Kirill A. Korinsky)]

Seems that issue with used mirror: ​http://ftp.fau.de

It says that the last sync was at 2023-04-25 19:20 and my some reason it marks that red.

[catap (Kirill A. Korinsky)]

Yeah, something is wrong: ​http://ftp.fau.de/cgi-bin/show-mirror-status.cgi?mirror=macports

[catap (Kirill A. Korinsky)]

I've checked mirrors:

• rsync://nue.de.rsync.macports.org/macports/release/tarballs/ports.tar
• rsync://fra.de.rsync.macports.org/macports/release/tarballs/ports.tar
• rsync://ykf.ca.rsync.macports.org/mprelease/tarballs/ports.tar
• rsync://cph.dk.rsync.macports.org/macports/release/tarballs/ports.tar

all of them contains the same file:

-rw-r--r--    108,035,072 2023/04/25 14:27:09 ports.tar

=> seems something is broken at source.

[ryandesign (Ryan Carsten Schmidt)]

We've had multiple occasions in the past couple days when the off-site Apple Silicon builder has unexpectedly lost the connection to the master, and I've also noticed some network issues on my machine on the same network. It's possible that FAU is experiencing similar unexpected network disconnections when trying to rsync from the main server. (All other mirrors sync from FAU.) Or perhaps all of the traffic from FAU is what was causing other services to disconnect.

We have been building ~2000 R ports on each builder, in small batches, since about the time this issue started. It's not unusual for the builders to be busy for days at a time, e.g. when several clang ports are updated, so I'm not sure why this would be a problem now. Maybe the difference is that with clang, we spend several hours building each port during which time an rsync run could complete without any files having changed on the server, whereas now we're building many small ports and between the time an rsync run starts and the time it finishes new files have appeared on the packages server. I'm not sure how rsync handles that situation. Maybe it retries automatically to get the new files. There might be a limit to the number of retries. Or maybe it just returns an error, and maybe FAU is running a script that retries a number of times if it gets that specific error.

Most buildbot builds have been paused due to a tornado watch but that seems to be done now. I'll take the opportunity to update and reboot the router before starting builds again.

A sync was happening now, and it transferred many R packages before failing with rsync: [sender] writer error: Broken pipe (32). I'll check what happens on the next sync.

[ryandesign (Ryan Carsten Schmidt)]

The next couple syncs ended with the same Broken pipe error after transferring some random number of files. Network traffic in Activity Monitor dropped to 0 for some time before the error message appeared in the log.

The next sync after updating and restarting the router is still going but has already gotten a lot farther than before.