libressl cannot be installed when it depends on a clang port because it conflicts with openssl

[ryandesign (Ryan Carsten Schmidt)]

libressl blacklists Xcode clang < 1001, so with Xcode versions earlier than 10.2, MacPorts will select a MacPorts clang compiler. But that won't work because MacPorts clang compilers depend on cmake which depends on curl which depends on openssl which conflicts with libressl.

​https://build.macports.org/builders/ports-10.8_x86_64-builder/builds/168292/steps/install-port/logs/stdio

This would be resolved by uncoupling libressl and openssl as long suggested in #54744.

[artkiver (グレェ)]

Thanks for bringing this to my attention!

It looks similar to #68811 which had somehow slipped from my awareness until earlier this week.

I had locally experimented with modifying the net/wget Portfile to use the path:lib/libssl.dylib:libressl instead of the OpenSSL PortGroup and it seemed to function OK, but I never submitted that as a PR.

I am guessing, curl in my use case is a little different since that appears to be dependent upon which variant is installed?

e.g. from the net/curl Portfile:

    variant ssl conflicts darwinssl gnutls mbedtls wolfssl description {Allow se
cure connections using OpenSSL} {
        depends_lib-append      path:lib/libssl.dylib:openssl \
                                path:share/curl/curl-ca-bundle.crt:curl-ca-bundle
        configure.args-delete   --without-ssl
        configure.args-replace  --without-openssl --with-openssl
        configure.args-append   --with-ca-bundle=${prefix}/share/curl/curl-ca-bundle.crt
    }

I admit, I am a little bit over my head with some of this stuff. I had been thinking of doing a TLS library audit of Portfiles and making assiduous use of the path:lib/libssl.dylib:openssl methodology wherever possible, but I haven't even managed to get a local build of mpbb working correctly yet. I did recently upgrade the SSD on one of my systems so I should at least have some more storage space with which to work, but my resources (not just computationally, but also financially and in terms of having peace and quiet and space to myself) are extremely constrained, which is why I tend to leave openmaintainer in place on any MacPort where I am also a comaintainer, so I welcome any suggestions, help or PRs as well!

Not to dismiss this in the slightest, I wish I were able to rectify it more quickly.

As a first guess: do you think that ameliorating the curl dependencies to be a bit more LibreSSL friendly/preferential might improve the situation? I do have some laptops with older versions of OS X but most of them are in storage and not too easy for me to get to for testing personally at the moment.

[ryandesign (Ryan Carsten Schmidt)]

The dependencies of curl are already as friendly toward libressl as they are in any port, according to our current methodology. Per #54744, I believe our current methodology is flawed and we should stop pretending that libressl is a simple drop-in replacement for openssl. openssl and libressl should install to different locations and we should have a separate libressl variant in ports that support libressl. Far from being something that can be fixed quickly, this will require coordination among the over 800 ports that depend on openssl. That would probably be impossible, so a solution that allows the migration to the new strategy to be implemented gradually on a port-by-port basis would be required. #54744 has been open a long time because nobody has yet wanted to put in the effort to do this.

The problem I'm reporting here primarily affects our buildbot environment because it always install ports with default variants. A user who chooses to install libressl might be able to make it work for example by forcibly deactivating openssl right before activating libressl and then rebuilding everything that had used openssl (which rev-upgrade would do for them automatically), or by installing curl with a TLS variant other than +ssl. Fixing it for the buildbot is still a worthy goal as that will allow binary archives to be produced, as is fixing it so users don't have to do weird deactivation/activation steps or pick a non-default curl variant.

[artkiver (グレェ)]

I think I might be seeing this issue rear its ugly head here as well:

​https://github.com/macports/macports-ports/pull/23716

Albeit, I am sort of wondering if this Trac issue is assigned correctly, since I haven't touched any of the buildbot code and the last time I updated Got and LibreSSL didn't run into any issues in GitHub Action's CI either (which is again a bit outside of my purvey).

Candidly, I have tried, twice, to set up mpbb locally and haven't managed to succeed in either attempt so far; and most certainly the tests I ran locally with got-portable0.98.2 were OK, so it's kind of difficult for me to recreate the GitHub Actions CI environment to see how I might be able to rectify it otherwise.

In that instance, I could force port:libretls instead of using the dylib approach which had been functional previously I guess?

I also tried to crib from the approach I used in the rpki-client Portfile, which attempts to be a little bit smarter with looking into what TLS library MacPorts has installed already and if using OpenSSL, to also install libretls to provide libtls (which is where the GitHub Action CI appears to be breaking with the got 0.98.2 PR I submitted) but when I tried that approach locally, for reasons that are a bit beyond me, MacPorts was still trying to install OpenSSL as a dependency, even when I was explicitly using the +libressl variant; which isn't how that should be functioning, at all.

[ryandesign (Ryan Carsten Schmidt)]

Replying to artkiver:

I think I might be seeing this issue rear its ugly head here as well:

​https://github.com/macports/macports-ports/pull/23716

I don't believe that's related. That's about libretls; this ticket is about libressl.

Albeit, I am sort of wondering if this Trac issue is assigned correctly, since I haven't touched any of the buildbot code

It's assigned to the libressl maintainers because it occurs due to a design decision of the libressl port: pretending that it is a drop-in replacement for openssl. I'm not suggesting that you should modify buildbot or GitHub Actions or mpbb to work around this or set up a local environment for either of those; I'm suggesting that the design of the libressl port should be changed per #54744.

[artkiver (グレェ)]

Respectfully, this doesn't track.

I don't know where you got the impression that anyone insinuated that LibreSSL is a "drop in replacement" for OpenSSL. LibreSSL forked from OpenSSL, a decade ago (specifically, April 22nd, 2014) and has taken a different trajectory over the ensuing years, even if it was initially intended to be relatively compatible (which is a given, since it was a fork) the LibreSSL and OpenSSL codebases have, and continue to diverge. So, while maybe in 2014, it was relatively easy to substitute OpenSSL with LibreSSL, doing so now might be a bit more challenging?

The good news is: many projects, like macOS, have long since done the heavy lifting of having migrated to LibreSSL.

MacPorts, is a different beast though and I am guessing there are reasons jeremyhu left 54744 unaddressed for years before he went AWOL?

Moreover, your suggestion here: ​https://github.com/macports/macports-ports/pull/23716#issuecomment-2078328088 suggesting that one can simply add

port:libretls

As a dependency, doesn't really function.

e.g. if done to the Got Portfile, one will encounter this sort of error when attempting to install it:

 got # port -v install        
--->  Computing dependencies for got...
Error: Can't install openssl because conflicting ports are active: libressl
Error: Follow https://guide.macports.org/#project.tickets if you believe there
is a bug.
Error: Processing of port got failed

If you examine the Portfile for devel/libretls you'll see:

depends_lib         port:openssl

Indeed, libtls is already supplied by libressl, as mentioned in the port description, from looking at e.g. ​https://ports.macports.org/port/libressl/

You'll see (apologies for the strange formatting, italics added for emphasis):

LibreSSL SSL/TLS cryptography library

LibreSSL is composed of four parts: The openssl(1) utility, which provides tools for managing keys, certificates, etc. libcrypto: a library of cryptography fundamentals libssl: a TLS library, backwards-compatible with OpenSSL

libtls: a new TLS library, designed to make it easier to write foolproof applications

This port tracks the stable releases, for development releases please use libressl-devel.

In other words, the entire point of libretls /devel/libretls is to provide libtls functionality, to OpenSSL. LibreSSL, already has libtls as a given.

But, for the sake of experimentation, I went ahead and modified the /devel/libretls Portfile manually to have

depends_lib         path:lib/libssl.dylib:openssl \

Instead of:

depends_lib         port:openssl

Attempting to install that locally, fails with the following:

Error: Failed to activate libretls: Image error: /opt/local/include/tls.h is being used by the active libressl port.  Please deactivate this port first, or use 'port -f activate libretls' to force the activation.
    while executing
"throw registry::image-error $msg"
    ("foreach" body line 47)
    invoked from within
"foreach file $imagefiles {
                set srcfile "${extracted_dir}${file}"

                # To be able to install links, we test if we can lst..."
    invoked from within
"registry::write {
            foreach file $imagefiles {
                set srcfile "${extracted_dir}${file}"

                # To be able to instal..."
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_libretls/libretls/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there
is a bug.
Error: Processing of port libretls failed

Candidly, it would probably save everyone some headaches if devel/libretls also had conflicts statements with libressl and libressl-devel.

Having written as much, there's an OpenSSL PortGroup too, which some Portfiles reference instead of any of the previously mentioned methodologies for linking a particular TLS library. So, while I don't disagree that it's quite a bit of effort to migrate your estimate of 800 some odd MacPorts to something that has better affinity with libressl, which seems possible in theory, that doesn't change the reality that most definitely, something changed within MacPorts and the GitHub Actions CI as well since some of these MacPorts (e.g. libressl, got) were last updated that is resulting in more breakage, whereas previously

depends_lib         path:lib/libssl.dylib:openssl \

Was seemingly sufficient to facilitate user choice in which TLS library was in use without throwing errors or causing undue breakage that is more recently observable.

I could certainly benefit for more help, but retreading old suggestions and old tickets that predate any of the time with when I began to contribute to MacPorts doesn't seem particularly helpful; especially given that ​https://trac.macports.org/ticket/54744 makes reference to boringssl which near as I can discern, does not exist within MacPorts at all, and if it did previously, I have no idea when it was deprecated.

Replying to ryandesign:

Replying to artkiver:

I think I might be seeing this issue rear its ugly head here as well:

​https://github.com/macports/macports-ports/pull/23716

I don't believe that's related. That's about libretls; this ticket is about libressl.

Albeit, I am sort of wondering if this Trac issue is assigned correctly, since I haven't touched any of the buildbot code

It's assigned to the libressl maintainers because it occurs due to a design decision of the libressl port: pretending that it is a drop-in replacement for openssl. I'm not suggesting that you should modify buildbot or GitHub Actions or mpbb to work around this or set up a local environment for either of those; I'm suggesting that the design of the libressl port should be changed per #54744.