Replace rmd160 use in MacPorts with something else

[ryandesign (Ryan Carsten Schmidt)]

The default checksum types used by MacPorts in portfiles are sha256, rmd160, and size. rmd160 is also used for each archive on the packages server.

OpenSSL 3 has designated ripemd160 (rmd160) as a legacy algorithm.

​https://www.openssl.org/docs/man3.0/man7/EVP_MD-RIPEMD160.html

I guess that means we should phase out the use of rmd160 in MacPorts, replacing it with something more modern.

[pmetzger (Perry E. Metzger)]

So I'm not sure there's a security reason to use two algorithms at once; SHA256 is enough for our purposes. We could just deprecate using two checksums at once.

Alternatively, if we decide we really need to use two, I'd recommend using SHA-3 which is Keccak based and uses a quite different construction than SHA-2, and is a national standard. Using a different construction makes it less likely that both SHA-2 and SHA-3 would have security issues at once. If tastes run against SHA-3, I'd suggest BLAKE2 or BLAKE3, which are based on very heavily studied primitives.

In no case should a hash as short as 128 bits be used; birthday attacks on such hashes are feasible.

However, again, my own recommendation would be to just drop RMD160 and not replace it with anything.

[ryandesign (Ryan Carsten Schmidt)]

We use two algorithms so that a compromise of one algorithm does not compromise the integrity of the files.

[pmetzger (Perry E. Metzger)]

As a side note:

$ find . -name Portfile -print | xargs egrep -w '(sha1|md5)' | wc -l
    3954

[cjones051073 (Chris Jones)]

Replying to ryandesign:

We use two algorithms so that a compromise of one algorithm does not compromise the integrity of the files.

but then, we only use rmd160 to validate the binary tarballs, no ?

[pmetzger (Perry E. Metzger)]

Replying to ryandesign:

We use two algorithms so that a compromise of one algorithm does not compromise the integrity of the files.

I think the probability of a high quality exploit that occurs without prior warning against any of the modern hash algorithms is quite low. That said, SHA-3 or BLAKE2/BLAKE3 are good options as I mentioned. I'd personally pick SHA3.

We should also systematically get rid of reliance on MD5 (people with inexpensive machines can fake that at this point) and SHA1 (people with expensive machines can fake that at this point.)

[ryandesign (Ryan Carsten Schmidt)]

As a side note:

$ find . -name Portfile -print | xargs egrep -w '(sha1|md5)' | wc -l
    3954

Yes, of course.

Many ports are old and have not been touched. There are probably still many that list only an md5 sum. They should be changed to list two newer sums and size.

For many years, our default set of checksum types was md5, sha1 and sha256. Even though md5 and sha1 are considered insecure, the fact that both are used to secure a single file means that the file's integrity is still assured.

[ryandesign (Ryan Carsten Schmidt)]

Replying to cjones051073:

Replying to ryandesign:

We use two algorithms so that a compromise of one algorithm does not compromise the integrity of the files.

but then, we only use rmd160 to validate the binary tarballs, no ?

Yes, but the rmd160 used for the binary archives is not merely a checksum; it is also somehow validating a signature with our public key. I have not attempted to understand exactly how that works. If it is a problem that we only use one algorithm there, we could use more than one.

[ryandesign (Ryan Carsten Schmidt)]

Replying to pmetzger:

I think the probability of a high quality exploit that occurs without prior warning against any of the modern hash algorithms is quite low.

It's obviously not about prior warning. It's about the fact that ports often do not get touched for years, so we want security in case an algorithm is discovered to be insecure and the portfile is then not updated for years after that.

We should also systematically get rid of reliance on MD5 (people with inexpensive machines can fake that at this point) and SHA1 (people with expensive machines can fake that at this point.)

Not when two different checksum types protect one file.

[cjones051073 (Chris Jones)]

What constraints are there on what crypto algorithms we could consider using ? i.e. how does base generate them ? I presume via whatever SSL library the OS ships with, in which case are we at all limited here ?

[pmetzger (Perry E. Metzger)]

BTW, just to mention: Apple Silicon has acceleration for SHA-3. BLAKE variants aren't (to my knowledge) accelerated by any architecture, but it is very very fast. Typical unaccelerated SHA-3 implementations are around 12 cycles/byte, BLAKE-3 is about half a cycle per byte (yes, 24x faster.) Maybe (arguably) given our support for older hardware BLAKE-3 might be friendlier. It is also a 256 bit hash function, FWIW.

[pmetzger (Perry E. Metzger)]

Replying to ryandesign:

Many ports are old and have not been touched. There are probably still many that list only an md5 sum. They should be changed to list two newer sums and size.

I could write a script to do that if I had access to the distfiles cache (or someone else could).

[mouse07410 (Mouse)]

When hash is used for non-cryptographic purposes (just to produce a unique identifier for a package), it does not really matter whether it's cryptographically broken or not. Thus, I wouldn't bend over backwards to eradicate all the usage of MD5, SHA1, RIPEMD160, etc.

. . . rmd160 used for the binary archives is not merely a checksum; it is also somehow validating a signature with our public key

This, ideally, should be replaced. Anything SHA-2 would be good, or SHA-3. I personally like SHA-3, and some candidates (like Blake2). Didn't pay attention to Blake3 (and probably won't, enough Post-Quantum things on the plate to occupy my time)...

Given how port validation and signature verification are done, I don't think we need to worry about performance of the hash.

In short: if you can move to SHA-2 or (better yet) SHA3, it would be great from several points of view. Especially if Perry can write a script to speed up the update.

[neverpanic (Clemens Lang)]

The best alternative for the .rmd160 signatures we use for packages is probably signify(1).

See ​https://www.openbsd.org/papers/bsdcan-signify.html and ​https://man.openbsd.org/signify.1.

​https://github.com/macports/macports-base/pull/184 would already add a copy of signify to MacPorts base. This PR doesn't change the archive signature mechanism to use signify, but it was my intention to eventually do this as well. When doing this, we should also switch from one globally configured key to a key per source, so that other sources can provide their own key with their own binaries.

[ryandesign (Ryan Carsten Schmidt)]

Replying to neverpanic:

the .rmd160 signatures we use for packages

Any change to this will require changing how MacPorts base fetches and verifies archives, since it currently expects to be able to fetch a file whose name is the archive name with ".rmd160" appended (and for it to contain data in an rmd160-based format). This would be a good opportunity to implement a suggestion I mentioned previously, where instead of a ".rmd160" file, MacPorts fetches an information file, whose name could be the archive name minus the archive/compression format suffix plus a suffix like ".info.json" (e.g. "zlib-1.2.11_0.darwin_21.arm64.info.json"). This file would contain fields that specify the archive/compression suffix (e.g. "tbz2") and the signature and its format. This would give us the ability to modify, over time, on an archive by archive basis, what archive/compression and signature format we use.

[ryandesign (Ryan Carsten Schmidt)]

Replying to pmetzger:

I could write a script to do that if I had access to the distfiles cache (or someone else could).

Everyone has access to it at ​http://distfiles.macports.org and rsync://rsync.macports.org/macports/distfiles/. But you don't need the full set of distfiles, only those distfiles mentioned in portfiles that don't use the right checksum types. Once you've identified which portfiles those are, you can use sudo port fetch to fetch the distfiles, sudo port checksum to verify that the deficient checksums in the portfile match, and then replace the checksums with modern ones.