GitHub fetches fail under OSX 10.5 (and presumably 10.4)

[fhgwright (Fred Wright)]

While testing the new Portfile for osxfuse, I ran into a problem where my machines running 10.5 were unable to fetch the distfiles from GitHub due to an ostensible certificate problem. However, it's not actually a certificate problem - it's apparently an OpenSSL code version problem. There are two parts to this problem:

1) Under 10.5, the OSX-provided OpenSSL library (version 0.97l) is too old for the corresponding curl to work with GitHub. The OpenSSL 0.9.8y supplied by 10.6 server works, though I don't know if standard 10.6 is different. I suspect that the watershed is between 0.9.7 and 0.9.8, possibly related to "secure renegotiation" support.

2) Even if the curl and openssl ports are installed, MacPorts doesn't use the newer curl internally.

This isn't a terribly serious problem, since:

1) It can be worked around by adding "fetch.ignore_sslcert=yes" to the install or fetch command. This is actually quite safe, since the checksum verification makes MitM attacks on the content pretty pointless, anyway.

2) "Published" ports normally have the distfiles available from the MacPorts servers, so the GitHub access is unnecessary.

If there isn't a reasonable fix for this, it should probably at least be documented, to keep others from tearing their hair out while chasing phantom certificate issues. Due to #2, it's mainly a port developer issue, not an end-user issue.

P.S.: Perhaps someone with a vanilla Snow Leopard "client" install could post the result of "/usr/bin/openssl version".

[ryandesign (Ryan Carsten Schmidt)]

I am not aware of a way we could fix this. I thought the problem was that Apple is not updating the SSL certificates for Leopard anymore, and it's not a job that we can really take over from Apple.

[danielluke (Daniel J. Luke)]

Replying to ryandesign@…:

I am not aware of a way we could fix this. I thought the problem was that Apple is not updating the SSL certificates for Leopard anymore, and it's not a job that we can really take over from Apple.

Sounds like there is also a problem with compatibility between modern ssl configs and the really-old openssl that shipped with 10.5.

I would be in favor of explicitly dropping support for OS X releases that are no longer being updated by Apple.

[ned-deily (Ned Deily)]

There are other issues with using the system OpenSSL 0.9.7 libs in OS X 10.5 and 10.4 besides certificates being updated. In particular, 0.9.7 does not support newer certificates signed with SHA256, which are becoming more common. Updating the root certificate bundle won't help for these. We (python-dev) ran into this when the certificate for python.org web pages and downloads was upgraded to SHA256. We ended up having to supply a private copy of current OpenSSL libs for our 10.5+ Python installers. It appears that github.com is also using a SHA256 certificate. 10.6 (and later) versions of OS X also ship with OpenSSL 0.9.8 libraries that do work with these certs. To work around it in MacPorts, you'd probably also need to supply a private version of curl and libssl/libcrypto and a minimum set of root CAs.

[fhgwright (Fred Wright)]

I hardly think that this is a serious enough problem to justify dropping support for all PowerMacs, which is what dropping 10.4 and 10.5 would do. Even the documentation-only fix wouldn't be too bad.

I believe the only reason HTTPS is being used for GitHub at all is because it refuses to speak plain HTTP. So merely operating as "fetch.ignore_sslcert=yes" in this case would be fine (given that the separate checksum verification makes the certificate verification unnecessary).

For a "proper" fix, the pre-10.6 installers could bundle a working version of curl, which might as well statically link against libssl and libcrypto in this context, so that only one file is needed. This would have to either go someplace other than /opt/local/bin/, or have a different name (to avoid conflicting with the curl port). Then there'd just need to be some sort of "my-curl" variable, pointing either at the bundled version or the system version.

AFAICT, the system root CAs are fine, since I can access GitHub from the old versions of Safari and Firefox on 10.5 just fine (except for GitHub's warnings about the old browser versions :-)). If you try HTTP, it just redirects to HTTPS.

Another question is whether it would be acceptable to use /opt/local/bin/curl when available. Presumably, MacPorts tries to avoid using tools from its own ports in general to avoid getting burned by the non-atomic upgrade process. But that may not be an issue for a tool that's only used in the fetch phase. If it did that, then the only missing piece would be a 10.4/10.5 fetch dependency on curl, for any port using HTTPS fetches. For safety, it might use the "ported" curl only for HTTPS (or only when the system curl fails).

[larryv (Lawrence Velázquez)]

Replying to fw@…:

I hardly think that this is a serious enough problem to justify dropping support for all PowerMacs, which is what dropping 10.4 and 10.5 would do.

We already don’t support 10.4 through 10.7. We try not to actively break things on those systems if we can help it, and some of us go to great lengths to keep ports working, but it’s all best-effort and not at all guaranteed.

Another question is whether it would be acceptable to use /opt/local/bin/curl when available.

MacPorts does not use the curl(1) executable. It links to the system libcurl and calls into it directly.

[ned-deily (Ned Deily)]

"AFAICT, the system root CAs are fine, since I can access GitHub from the old versions of Safari and Firefox on 10.5 just fine"

FWIW, on 10.5 (and earlier), the system OpenSSL and curl do not use the same source for root CAs as either Safari or Firefox. By default, curl looks for a certificate bundle file at /usr/share/curl/curl-ca-bundle.crt, which you can manually update but, as noted above, that still won't help for newer SHA256 certs. The system OpenSSL looks for root CAs in /System/Library/OpenSSL. Starting in 10.6, if no root CA match is found in /System/Library/OpenSSL, the system OpenSSL will consult the system trust store of certificates via TEA (see ​https://hynek.me/articles/apple-openssl-verification-surprises/ for details).

[ryandesign (Ryan Carsten Schmidt)]

Now that MacPorts has the ability to bundle libraries with base, and does so for tcl already, I am in favor of including our own up-to-date copy of libcurl, if that will help with anything.

[danielluke (Daniel J. Luke)]

Replying to ryandesign@…:

Now that MacPorts has the ability to bundle libraries with base, and does so for tcl already, I am in favor of including our own up-to-date copy of libcurl, if that will help with anything.

It sounds like it would be libcurl + openssl that would be required. Since Apple no longer releases security updates for PPC machines, people probably shouldn't be running them anymore - and we shouldn't be encouraging it. The Power Mac G5 was discontinued August 7, 2006.

While I'm happy for people to keep old machines running/working (I do it myself with some old hardware) - the people with that hardware really need to be responsible for anything that MacPorts needs to do to help (and any necessary changes shouldn't negatively impact people running actual supported systems). This isn't about 'dropping support' since MacPorts officially only supports the current + previous Mac OS X release (we probably shouldn't release easy-to-install packages for older OS). Essentially - older Mac OS X releases should be treated like our 'other platform' support.

[neverpanic (Clemens Lang)]

With outdated roots being a growing problem I don't see how a private copy of libcurl would help here.

[fhgwright (Fred Wright)]

Replying to larryv@…:

Replying to fw@…:

I hardly think that this is a serious enough problem to justify dropping support for all PowerMacs, which is what dropping 10.4 and 10.5 would do.

We already don’t support 10.4 through 10.7. We try not to actively break things on those systems if we can help it, and some of us go to great lengths to keep ports working, but it’s all best-effort and not at all guaranteed.

Umm, from the homepage:

"Installers for legacy platforms Lion, Snow Leopard, Leopard and Tiger are also available."

So I guess it depends on what you mean by "supported". :-)

Another question is whether it would be acceptable to use /opt/local/bin/curl when available.

MacPorts does not use the curl(1) executable. It links to the system libcurl and calls into it directly.

OK, different file, same concept.

Actually, based on what I see in macports.conf, MacPorts does use its own tools when available (unless the default binpath setting is changed). However, that only applies to executables, not libraries; hence the libcurl/libssl issue. That seems somewhat inconsistent.

I've verified that I can make the fetch work under 10.5 by prefixing the command with "DYLD_LIBRARY_PATH=/opt/local/lib" (with the curl and openssl ports installed, of course).

Replying to nad@…:

"AFAICT, the system root CAs are fine, since I can access GitHub from the old versions of Safari and Firefox on 10.5 just fine"

FWIW, on 10.5 (and earlier), the system OpenSSL and curl do not use the same source for root CAs as either Safari or Firefox. By default, the system curl looks for a certificate bundle file at /usr/share/curl/curl-ca-bundle.crt, which you can manually update but, as noted above, that still won't help for newer SHA256 certs. The system OpenSSL looks for root CAs in /System/Library/OpenSSL. Starting in 10.6, if no root CA match is found in /System/Library/OpenSSL, the system OpenSSL will consult the system trust store of certificates via TEA (see ​https://hynek.me/articles/apple-openssl-verification-surprises/ for details).

Here, /System/Library/OpenSSL seems to be empty except for some scripts under misc/ (in both 10.5 and 10.9). So either Safari and Firefox are using curl-ca-bundle.crt, or the browsers have their own root CAs elsewhere.

Though if the only use of SSL in MacPorts tself is for fetching content that's independently checksum-verified, then it might be entirely reasonable for "fetch.ignore_sslcert=yes" to be the default, at least under 10.5 and earlier.

Replying to cal@…:

With outdated roots being a growing problem I don't see how a private copy of libcurl would help here.

But outdated roots isn't the issue, and if cert validation is considered unimportant (see my last comment), then it wouldn't matter if it were.

[ned-deily (Ned Deily)]

"Here, /System/Library/OpenSSL seems to be empty except for some scripts under misc/ (in both 10.5 and 10.9). So either Safari and Firefox are using curl-ca-bundle.crt, or the browsers have their own root CAs elsewhere."

Neither Safari nor Firefox use the Apple-supplied versions of OpenSSL, which is basically only there for the benefit of some open source software shipped with OS X, like Python; note that OpenSSL has officially been deprecated by Apple as of 10.7. AFAIK, Safari uses certificates from the system trust store which is maintained in system and user keychains and uses the OS X native crypto services. The ​Cryptographic Services Guide has more info on how this all works; it has nothing to do with OpenSSL or, in 10.5, curl. And, AFAIK, Firefox provides its own root CAs and certificate management, independent of the system trust store. So, yes, as shipped, /System/Library/OpenSSL has no root certs. It has always (at least, since 10.5) been up to the user to manually supply and manage these for the benefit of any user- or (the few) system-supplied apps (like the system Python) that link with the system OpenSSL libs. As noted above, though, starting in 10.6, the system-supplied OpenSSL 0.9.8 does have a fallback hook into the system trust store (via TEA) which may or may not be what the application wants.

But, to summarize, I believe it is the case that if MacPorts base needs to securely download (TLS) on 10.5 systems from all sources, it will need to supply its own versions of libssl, libcrypto, and libcurl.

[larryv (Lawrence Velázquez)]

Replying to dluke@…:

Since Apple no longer releases security updates for PPC machines, people probably shouldn't be running them anymore - and we shouldn't be encouraging it. The Power Mac G5 was discontinued August 7, 2006.

+1000000, especially if continued “support” requires us to become some sort of ad hoc security vendor.

[raimue (Rainer Müller)]

Duplicate of #51516.