[apple-pki-bundle] : extend to cover all certificates from "System Roots"

[RJVB (René Bertin)]

Cf. ​https://apple.stackexchange.com/questions/422332/how-do-i-update-my-root-certificates-on-an-older-version-of-mac-os-e-g-el-capi/429778#429778

Lots of sites have been broken recently on Macs running older versions of the OS because of an expired certificate for which no replacement is included in this port at the moment (LetsEncrypt ISRG). Apparently Apple do ship it in one of their OS updates because you can get it by transferring the contents of the System Roots certificate store from an up-to-date system to your out-of-date system.

This port already has a precedent for the argument that it does-but-shouldn't include non-Apple certificates, given its name, by having a default variant which includes GeoTrust and Digicert certificates.

Why not keep it up-to-date by including all certificates that Apple ship in that System Roots store?

BTW: note how several answers in the StackExchange discussion above point out the uncertainty inherent in downloading certificates, apparently even in downloading from the issuer's site. If that's not being overly paranoid, just how does this port do better than that? Has the maintainer verified each cert. checksum against a copy exported from the Keychain on his up-to-date system, and has this been double-checked by at least 1 independent authoritative MacPorts maintainer?

As an alternative, cannot the System Roots store be obtained directly from the latest Apple updater that contains it?

[essandess (Steve Smith)]

has a precedent for the argument that it does-but-shouldn't include non-Apple certificates, given its name

Apple PKI isn’t self-contained. Those CAs sign Apple certs.

See: ​https://github.com/macports/macports-ports/blob/344fb99500b93daf4890f7031f95601c23506a06/net/apple-pki-bundle/Portfile#L134

There is a variant to include additional CAs, but in no circumstances do I believe the port should include expired CAs.

[RJVB (René Bertin)]

The meaning of that commented-out Tcl expression and the line above isn't really clear; I had taken it for a maintainer-convenience function to help in upgrading.

There is a variant to include additional CAs, but in no circumstances do I believe the port should include expired CAs.

In no circumstances did I suggest that either, assuming that Apple don't allow expired certificates in their most up-to-date "System Root" store?!

[essandess (Steve Smith)]

The linked bash code just shows the trust chain for Apple PKI certs.

ISRG Root X1 is included in Mozilla’s bundle, therefore it’s already in port curl-ca-bundle. Does this address what you’re looking for?

[cooljeanius (Eric Gallager)]

This is the same idea as this discussion from the mailing lists, correct? ​https://lists.macports.org/pipermail/macports-users/2021-October/050410.html

[RJVB (René Bertin)]

Replying to essandess:

ISRG Root X1 is included in Mozilla’s bundle, therefore it’s already in port curl-ca-bundle. Does this address what you’re looking for?

Not really, because that those certificates typically aren't available outside of MacPorts (nor probably to ports that ship native Mac applications). Which reminds me that this port should probably have a notes description of the steps required to use/apply the installed bundle. I see no evidence in the Portfile that the certificates are actually being added to any of the OS's certificate stores.

Replying to cooljeanius:

This is the same idea as this discussion from the mailing lists, correct? ​https://lists.macports.org/pipermail/macports-users/2021-October/050410.html

Yeah, it is. I didn't want to file a ticket at first.

[mascguy (Christopher Nielsen)]

Are there any legal issues with including the various required certs? If not, that would simplify everyone's life immensely!

[essandess (Steve Smith)]

Replying to RJVB:

Replying to essandess:

ISRG Root X1 is included in Mozilla’s bundle, therefore it’s already in port curl-ca-bundle. Does this address what you’re looking for?

Not really, because that those certificates typically aren't available outside of MacPorts

??? These certs are simply downloaded from ​https://curl.se/docs/caextract.html.

I see no evidence in the Portfile that the certificates are actually being added to any of the OS's certificate stores.

If there’s a circumstance for which a port installs CAs in the System Keychain, I can’t imagine what that would be. This sounds like a Bad Idea. Users/Admins should manage their PKI.

[mascguy (Christopher Nielsen)]

Replying to essandess:

Replying to RJVB:

I see no evidence in the Portfile that the certificates are actually being added to any of the OS's certificate stores.

If there’s a circumstance for which a port installs CAs in the System Keychain, I can’t imagine what that would be. This sounds like a Bad Idea. Users/Admins should manage their PKI.

Perhaps the port could include an ultra-simple shell script to effect the changes? We'd want the script to backup the keychain first, and tell the user where said backup is. But otherwise, this would simply everyone's life, without forcibly making changes.

We'd also want to include a port note, mentioning the helper script. Along with a quick blurb on how to use it.

How does that sound?

[essandess (Steve Smith)]

Replying to mascguy:

Replying to essandess:

Replying to RJVB:

I see no evidence in the Portfile that the certificates are actually being added to any of the OS's certificate stores.

If there’s a circumstance for which a port installs CAs in the System Keychain, I can’t imagine what that would be. This sounds like a Bad Idea. Users/Admins should manage their PKI.

Perhaps the port could include an ultra-simple shell script to effect the changes? We'd want the script to backup the keychain first, and tell the user where said backup is. But otherwise, this would simply everyone's life, without forcibly making changes.

We'd also want to include a port note, mentioning the helper script. Along with a quick blurb on how to use it.

How does that sound?

It’s not clear from this thread or the email thread the problem that is being addressed, or whether adding these certs to the keychain would actually fix it. There are more fundamental issues on old systems, like TLS1 being deprecated.

If there were a confirmed, working solution for PKI on unsupported OS’s, then that should be separate port that uses apple-pki-bundle in depends_lib.

[RJVB (René Bertin)]

The context here is "not-so-old" systems like 10.11 for which up-to-date browsers are still being provided, and where the lack of updates to the System Roots causes connection errors to sites that use other or renewed certificate authority certificates. I can confirm explicitly that adding the missing or updated certificates to the System store does indeed restore connectivity to the affected sites.

When I wrote "outside of MacPorts" I referred to applications not installed through MacPorts, NOT to download locations. But as indicated any application that uses the system's certificate stores would be affected - I presume that would include Qt and GTk apps using the Security framework. The deprecation of TLS1 is an orthogonal problem, regardless of whether or not it's more fundamental.

Having random ports that try to install certificates at the system level wouldn't be a very good idea, though evidently they could only do that through the intervention of a local administrator who already has the power to apply system updates or mess with the central certificate store(s). Except the System Roots store directly, FWIW, because that one can only be modified by the system (although I presume anyone with sudo powers could replace the corresponding file).

[essandess (Steve Smith)]

I can confirm explicitly that adding the missing or updated certificates to the System store does indeed restore connectivity to the affected sites.

Great! That’s simple. What about providing a bash one-liner in the notes or a script that adds CAs to the System keychain, per mascguy’s suggestion?

The less simple thing is that there appear to be different Apple PKI bundles: the one at ​https://www.apple.com/certificateauthority/, the macOS System Roots, and the ones on iOS.

Which one did you install to get macOS 10.11 working again?

Also, port apple-pki-bundle must be updated to include all of System Roots, so I’ll issue a PR for that too.

[RJVB (René Bertin)]

Replying to essandess:

Great! That’s simple.

Indeed. Me too I was pleasantly surprised, after all the hair-pulling to understand what was going on, why an up-to-date Chrome would no longer connect to lots of sites that Firefox had no issue with, etc.

What about providing a bash one-liner in the notes or a script that adds CAs to the System keychain, per mascguy’s suggestion?

Either is fine with me! There's also the option of doing it in the post-activate is a variant is set.

The less simple thing is that there appear to be different Apple PKI bundles: the one at ​https://www.apple.com/certificateauthority/, the macOS System Roots, and the ones on iOS.

Which one did you install to get macOS 10.11 working again?

The 10.11 system was done remotely with me instructing my favorite "Jane User" via IM so we kept it to just the ISRG certificate. Updating just that one solves the connection issues with most common sites. On my own 10.9 system I also installed the .pem file from the current port:apple-pki-bundle. Stupidly I didn't check if any were absent or out-of-date first, but I did notice (on Linux) that you need them to connect to certain Apple sites.

If in any way possible I'd ship the collection in an up-to-date System Roots. If the iOS equivalent has different certificates there must a reason why Apple make that so (and why that is not a problem) - I don't see what we would need them for ... and I could imagine that Apple might object to making them easily available on Mac.

Also, port apple-pki-bundle must be updated to include all of System Roots, so I’ll issue a PR for that too.

[cooljeanius (Eric Gallager)]

I feel like the certsync port is relevant to this discussion

[RJVB (René Bertin)]

Potentially, if it were made co-installable with port:curl-ca-bundle and assuming we're indeed discussing x509 certificates here. Oh, and it would still have to export only the certificates in System Roots.

But then what ... the apple-pki-bundle maintainer would use the utility to create a redistributable .pem file?

[essandess (Steve Smith)]

Please see ​https://github.com/macports/macports-ports/pull/12757

[essandess (Steve Smith)]

In 3060be4c7c4fb0249ac85ab51eeaa6f4a99cf96b/macports-ports (master):

apple-pki-bundle: Add macOS System Roots, utility scripts, and notes

• See: #63740