curl @8.5.0+darwinssl: Port install reports that variant clashes with gnutls and exits

[FaradayLight (Faraday Light)]

Performing an install of curl with following variants after an upgrade to Xcode 15.2:

sudo port install curl +darwinssl +http2 +openldap

The following errors are returned:

Error: curl: Variant darwinssl conflicts with gnutls
Error: Unable to open port: Error evaluating variants

I do not recall experiencing this conflict during prior upgrades.

[FaradayLight (Faraday Light)]

Current installed version of gnutls is 3.7.10_0.

[ryandesign (Ryan Carsten Schmidt)]

The darwinssl variant does conflict with the gnutls variant. curl has many different options for ssl backends, including darwinssl, gnutls, and others, but only one can be used at a time.

The http2 and openldap variants you requested are standalone and don't impose any other variant requirements.

So based on what you told me, this error should not be happening. Is there something you haven't told me? For example, have you put +gnutls or +http3 into your variants.conf file? (The http3 variant does depend on the gnutls variant.)

If you can't work out why the gnutls variant is being requested, attach the main.log file; maybe it explains why.

[FaradayLight (Faraday Light)]

Thank you.

I reinstall a set of packages from a list after each upgrade. The list contains gnutls and and is followed later in the list by curl, but I manually install curl again to select the variants mentioned above. This is the first time I have encountered the issue.

These are the relevant entries from the list generated by "port installed" created prior to the update detailed above:

  gnutls @3.7.10_0 (active) requested_variants='' platform='darwin 22' archs='arm64' date='2023-08-11T18:28:57+0100'

  curl @8.2.1_0+darwinssl+http2+openldap requested_variants='+darwinssl+http2+openldap' platform='darwin 22' archs='arm64' date='2023-07-28T12:46:02+0100'
  curl @8.3.0_0+darwinssl+http2+openldap (active) requested_variants='+darwinssl+http2+openldap' platform='darwin 22' archs='arm64' date='2023-09-17T10:33:27+0100'

I have not amended 'variants.conf' - it matches the current variants.conf.default.

I can remove the gnutls from the installation list and try a full re-install of the packages to see if they all build correctly. Would that be an optimal course of action?

[ryandesign (Ryan Carsten Schmidt)]

Having the gnutls port installed shouldn't make any difference. The problem is that something is causing curl's gnutls variant to become selected. Can you attach curl's main.log so we can check if it explains why that happened?

Or, if you don't care what TLS backend curl uses, allow curl to be installed with the gnutls variant instead of the darwinssl variant.

[FaradayLight (Faraday Light)]

No long is created for the error.

However, I posted the wrong copy of the command I was using. This is the install command that fails:

sudo port -v install curl +darwinssl +http2 +openldap +http3

If I remove the http3 variant the installation is successful.

[FaradayLight (Faraday Light)]

P.S. I have being using the darwinssl variant because perlbrew.pl will fail with not being able to validate site certificates if curl is built using the gnutls package.

[ryandesign (Ryan Carsten Schmidt)]

Ok great, then that explains the error. The http3 variant has required the gnutls variant ever since the http3 variant was added. So there's nothing to do here, unless you have reason to believe that http3 no longer requires gnutls.

[FaradayLight (Faraday Light)]

Thanks.

The only issue I see then is that the http3 variant will prevent the curl package from being installed if gnutls is already on the system.

I have no specific need for the http3 variant I was just going to try it out. I have been able to install the darwinssl variant of curl so that addresses my immediate needs and the purpose of the ticket.

[ryandesign (Ryan Carsten Schmidt)]

Replying to FaradayLight:

The only issue I see then is that the http3 variant will prevent the curl package from being installed if gnutls is already on the system.

Why would it do that?

[FaradayLight (Faraday Light)]

This was the issue I was observing when I raised this ticket.

[ryandesign (Ryan Carsten Schmidt)]

I still don't see how that's possible. The http3 variant requires the gnutls variant which requires the gnutls port.

[FaradayLight (Faraday Light)]

This is the current build:

MacPortsVersion:        2.9.1
XcodeVersion:           15.2
XcodeBuild:             15C500b
XcodeSDKVersion:        14.2
ProductName:		macOS
ProductVersion:		14.3
BuildVersion:		23D56
PlatformArchitecture:   arm64e

If I run the following (NOTE: gnutls is already installed at this stage from earlier in an installation list), curl will install:

sudo port install curl +darwinssl +http2 +openldap

This is what port then reports is installed:

  gnutls @3.7.10_0 (active)
  curl @8.6.0_1+darwinssl+http2+openldap (active)

However, if attempt the following:

sudo port install curl +darwinssl +http2 +openldap +http3

Then I will get the following:

Error: curl: Variant darwinssl conflicts with gnutls
Error: Unable to open port curl: Error evaluating variants

No main.log instance is created in this scenario.

This prevents the install of curl if both darwinssl and http3 variants are selected.

I have no specific need for the http3 variant I was just going to try it out. I have been able to install the darwinssl variant of curl so that addresses my immediate needs and the purpose of the ticket. However in the case where there is a requirement for both variants curl will not install. It is this latter point that I was referring to in my comment dated 2024-01-25.

In short: There is a conflict between to the two variants due to - as you mentioned - the http3 variant building against gnutls but not darwinssl which will prevent the curl package from installing if both are selected.

[ryandesign (Ryan Carsten Schmidt)]

Well, I already partly explained, but let me give some more detail about why the port is behaving correctly and why any changes to its behavior will require work from the curl developers.

curl has many TLS backends, available through many variants in the port (darwinssl, gnutls, mbedtls, ssl (openssl or libressl), and wolfssl), but you can only select one of them.

You can read all about ​HTTP/3 support in curl on the curl web site. It requires the use of one of four different QUIC libraries. Only one of those four implementations—the one that uses ngtcp2 and nghttp3—isn't considered experimental anymore so that's what the MacPorts curl port's http3 variant uses.

ngtcp2 requires a TLS library that has QUIC support. The possibilities listed on the curl web site are the quictls fork of openssl, gnutls, and wolfssl. At present, the ngtcp2 port uses gnutls unconditionally. Therefore, the curl port's http3 variant also has to use gnutls unconditionally, and thus prevent you from using any other TLS variant. It has nothing to do with whether or not the gnutls port is already installed when you install the curl port.

It would be possible to add variants to the ngtcp2 port so that you could choose between gnutls or wolfssl (or quictls, if we made a port for that). However, then the curl port's http3 variant would have to attempt to match the variant that was selected in the ngtcp2 port. MacPorts base doesn't have the capability for a port to declare a dependency on a variant of another port (see #126) so this is complicated and brittle and not something we usually want to do, so it's just as well that ngtcp2 doesn't offer this choice.

darwinssl, mbedtls, and openssl do not have QUIC support so they cannot be used by the ngtcp2 library, and thus you cannot select one of these as your curl TLS variant if you also want HTTP/3 support.

darwinssl currently uses the macOS SecureTransport framework. Apple has deprecated this framework and will not add new features to it, like TLS 1.3 or QUIC support. Apple wishes for developers to migrate to the Network framework. curl hasn't done that yet. If you want curl to support HTTP/3 with the darwinssl option, the prerequisite is that curl adds support for the Network framework. I didn't see an issue in their issue tracker specifically about that, although it has been mentioned in passing in other issues over there, for example here where ​the lead developer of curl said "I don't see this happening anytime soon". You can file an issue there if being able to use HTTP/3 with darwinssl is important to you.

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign:

darwinssl currently uses the macOS SecureTransport framework. Apple has deprecated this framework and will not add new features to it, like TLS 1.3 or QUIC support. Apple wishes for developers to migrate to the Network framework. curl hasn't done that yet. If you want curl to support HTTP/3 with the darwinssl option, the prerequisite is that curl adds support for the Network framework. I didn't see an issue in their issue tracker specifically about that, although it has been mentioned in passing in other issues over there, for example here where ​the lead developer of curl said "I don't see this happening anytime soon". You can file an issue there if being able to use HTTP/3 with darwinssl is important to you.

The issue came up again last month, and after I inquired, ​the curl developer reiterated that he has no plans to add support for the Network framework to curl (although I suspect that if someone were to contribute code that uses the Network framework it would be accepted). There was also a message today where ​the curl developer proposed removing support for libraries like SecureTransport that do not support TLS 1.3. So if you have some need for curl to use a macOS-provided TLS library that is not met by having curl use gnutls, please tell me or the curl developer everything you can about it.

There is ​an upcoming curl meeting which I plan to attend, and HTTP/3 and TLS are on the agenda, so if there is a reason why adding support for Network framework to curl is important to you that you would like me to advocate for at the meeting, please provide the necessary ammunition.

So far the only issue you mentioned is this one:

Replying to FaradayLight:

I have being using the darwinssl variant because perlbrew.pl will fail with not being able to validate site certificates if curl is built using the gnutls package.

Could you provide more details or a reproduction recipe? What error message do you get?