10.5 ppc buildbot: multiple SSL/TLS fetch failures, implement work around?

[dbevans (David B. Evans)]

Not unexpectedly, a number of ports are failing to fetch on the 10.5 ppc buildbot due to known SSL/TLS issues discussed in #46539. See also #46630, #44615, #46361.

In the current instance (py-cryptography @1.5.2)

--->  Attempting to fetch cryptography-1.5.2.tar.gz from https://files.pythonhosted.org/packages/source/c/cryptography

DEBUG: Fetching distfile failed: SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

I understand that this is a duplicate of the previous tickets, but I'm suggesting that this may be an opportunity to implement and test a possible work around. As it is, this defect significantly reduces the benefit of the buildbot both for testing and for generating binary archives.

[dbevans (David B. Evans)]

See also ​the relevant 10.5 ppc build log.

[jmroot (Joshua Root)]

The workaround has been to fetch from our distfiles mirror instead. I believe Ryan was hoping to set up the mirroring as a buildbot job so it could happen right after each commit and before any builds.

[dbevans (David B. Evans)]

Replying to jmr@…:

The workaround has been to fetch from our distfiles mirror instead. I believe Ryan was hoping to set up the mirroring as a buildbot job so it could happen right after each commit and before any builds.

Through this week it looked like few, if any, of the distfiles mirrors were being updated at all. However, as of today, it looks like they are being updated once again. Although the root problem remains, many ports that were failing to fetch at all are now fetching through the mirrors. Doing more detailed testing now on a leopard ppc machine that I now have remote access to.

[ryandesign (Ryan Carsten Schmidt)]

Replying to devans@…:

Through this week it looked like few, if any, of the distfiles mirrors were being updated at all. However, as of today, it looks like they are being updated once again.

The script that mirrors all ports' distfiles currently runs twice a week. This will be changed to mirror relevant ports' distfiles immediately after a commit.

Although the root problem remains, many ports that were failing to fetch at all are now fetching through the mirrors. Doing more detailed testing now on a leopard ppc machine that I now have remote access to.

I'm not sure any additional testing is required. We know Leopard's curl/openssl can't understand modern SSL certificates. We will address the problem by mirroring the distfiles to our non-SSL server before attempting to build on the buildbot.

[dbevans (David B. Evans)]

Replying to ryandesign@…:

Replying to devans@…:

Through this week it looked like few, if any, of the distfiles mirrors were being updated at all. However, as of today, it looks like they are being updated once again.

The script that mirrors all ports' distfiles currently runs twice a week. This will be changed to mirror relevant ports' distfiles immediately after a commit.

Although the root problem remains, many ports that were failing to fetch at all are now fetching through the mirrors. Doing more detailed testing now on a leopard ppc machine that I now have remote access to.

I'm not sure any additional testing is required. We know Leopard's curl/openssl can't understand modern SSL certificates. We will address the problem by mirroring the distfiles to our n

The testing is for my own purposes so as to understand how things are working. GNOME is publishing their 3.22.1 release this week and I didn't want to bog the buildbot down by making a lot of commits that will just fail on fetch. That's pretty much everything right now since gtk3 won't build on ppc right now. Thanks for the update. I'll try and moderate my commits until I hear that your mirror-on-commit fix is working. I know there's a lot to do behind the scenes these days. Let me know if there's something that I can do to help.

[ken-cunningham-webuse]

please see 51516#comment:19.

rather simple workaround appears to work well, tested on 10.5 and 10.6. bootstrap with standard install, install newer curl, then re-install macports in place from source referencing the updated curl in ${prefix}.

only hiccup might be when curl is updated -- but even then, it's taken through to the destroot stage with the existing curl, and then the new curl is installed all in a burst so I can't see any issue there.

unless the user accidentally uninstalls curl, or course -- and then there is a mess....

[ryandesign (Ryan Carsten Schmidt)]

This problem will be solved by resolving #53347.