can't build curl -- curl-config differs due to PSL support?

[michael-j-oconnor]

I just upgraded from Yosemite to Sierra yesterday. I think I successfully followed the steps at wiki:Migration Things *seem* sane enough:

$ sudo port rev-upgrade
--->  Scanning binaries for linking errors
--->  No broken files found.

Today, I try doing port selfupdate/port upgrade outdated and hit a snag with curl.

I do "sudo port clean curl" and "sudo port install curl" and get the following:

--->  Configuring curl
Warning: reinplace s|-arch [a-z0-9_]+||g didn't change anything in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/curl-7.56.1-x86_64/libcurl.pc
Warning: reinplace s/ '(host_alias|--host)=[^']+'//g didn't change anything in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/curl-7.56.1-x86_64/curl-config
Warning: reinplace s|-arch [a-z0-9_]+||g didn't change anything in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/curl-7.56.1-i386/libcurl.pc
Warning: reinplace s/ '(host_alias|--host)=[^']+'//g didn't change anything in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/curl-7.56.1-i386/curl-config
--->  Building curl
--->  Staging curl into destroot
Error: Failed to destroot curl: curl-config differs in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-i386//opt/local/bin and /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-x86_64//opt/local/bin and cannot be merged
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.

Looking at what's different, I see:

$ diff /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-i386//opt/local/bin /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-x86_64//opt/local/bin
Binary files /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-i386//opt/local/bin/curl and /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-x86_64//opt/local/bin/curl differ
diff /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-i386//opt/local/bin/curl-config /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-x86_64//opt/local/bin/curl-config
86c86
<         for feature in SSL IPv6 UnixSockets libz AsynchDNS IDN NTLM NTLM_WB TLS-SRP HTTPS-proxy ""; do
---
>         for feature in SSL IPv6 UnixSockets libz AsynchDNS IDN PSL NTLM NTLM_WB TLS-SRP HTTPS-proxy ""; do
151c151
<           echo ${CURLLIBDIR}-lcurl -lidn2 -lssl -lcrypto -lssl -lcrypto -lz
---
>           echo ${CURLLIBDIR}-lcurl -lidn2 -lpsl -lssl -lcrypto -lssl -lcrypto -lz
159c159
<           echo ${exec_prefix}/lib/libcurl.a -L/opt/local/lib -Wl,-headerpad_max_install_names  -L/opt/local/lib -L/opt/local/lib -L/opt/local/lib -lidn2 -lssl -lcrypto -lssl -lcrypto -lz
---
>           echo ${exec_prefix}/lib/libcurl.a -L/opt/local/lib -Wl,-headerpad_max_install_names  -L/opt/local/lib -L/opt/local/lib -L/opt/local/lib -lidn2 -lpsl -lssl -lcrypto -lssl -lcrypto -lz

I'm not sure what's going on here. I do appear to have libpsl:

$ sudo port info libpsl
libpsl @0.19.1_1 (net)
Variants:             universal

Description:          A C library and utility to handle the Public Suffix List
Homepage:             https://github.com/rockdaboot/libpsl

Any ideas of what went wrong, or perhaps better yet, how to fix in a way that isn't "start from scratch"?

[mf2k (Frank Schima)]

In the future, please Cc the port maintainers (port info --maintainers curl).

[ryandesign (Ryan Carsten Schmidt)]

Could you attach the main.log file, please? And also the two config.log files.

[michael-j-oconnor]

/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/main.log

[michael-j-oconnor]

/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-i386opt/local/bin

[michael-j-oconnor]

/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-x86_64opt/local/bin

[ryandesign (Ryan Carsten Schmidt)]

Thanks but could you also attach the two config.log files?

Is the libpsl port installed with the universal variant?

[michael-j-oconnor]

/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/workcurl-7.56.1-i386/config.log

[michael-j-oconnor]

/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/workcurl-7.56.1-x86_64/config.log

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign:

Is the libpsl port installed with the universal variant?

The i386 config.log suggests it is not:

ld: warning: ignoring file /opt/local/lib/libpsl.dylib, file was built for x86_64 which is not the architecture being linked (i386): /opt/local/lib/libpsl.dylib

[ryandesign (Ryan Carsten Schmidt)]

I thought we just fixed this for curl, but I misremembered: we fixed it for wget.

[michael-j-oconnor]

It doesn't appear to be. I just installed the universal variant for now and it works. Thanks!

Having said that:

• Should curl have sorted that dependency out?
• I don't knowingly need "universal" variant or anything at this point. If there's an automated way to rebuild the world without +universal...

[ryandesign (Ryan Carsten Schmidt)]

As with wget, curl apparently uses libpsl if present. I didn't know that, which is why these ports didn't handle this situation. Now that I know, I will investigate the proper solution: either make curl depend on libpsl unconditionally as was done for wget; or prevent the use of libpsl; or add a variant so the user can choose.

If MacPorts installed curl universal without you asking for that, it must've been required at the time. It is of course possible that you subsequently uninstalled whatever it was that needed curl to be universal. But there is no built-in automatic way to reinstall universal ports nonuniversally. Some scripting would be required. I'm sure I've posted some part of such a script to the mailing list some years ago.

[michael-j-oconnor]

Thanks again.

I was kinda hoping something like "sudo port upgrade --enforce-variants installed -universal" followed by "sudo port uninstall inactive" would work, or perhaps a "-universal" in variants.conf. I was a little leery of running it because my install is working again. Long ago and far away, this started as a Leopard install, where universal binaries were more-needed.

[ryandesign (Ryan Carsten Schmidt)]

Replying to michael-j-oconnor:

I was kinda hoping something like "sudo port upgrade --enforce-variants installed -universal" followed by "sudo port uninstall inactive" would work, or perhaps a "-universal" in variants.conf. I was a little leery of running it because my install is working again. Long ago and far away, this started as a Leopard install, where universal binaries were more-needed.

Well if that works, it would remove the universal variant from all installed ports, even if for some ports it might still be required by some other port. For example, if you have the wine port installed, you need its library dependencies installed universal.

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #55431.

[rlhamil]

I had a similar (maybe not exactly the same) problem upgrading to curl @7.56.1_1+ssl+universal on Snow Leopard.

[rlhamil]

main.log building curl @7.56.1_1+ssl+universal on Snow Leopard

[ryandesign (Ryan Carsten Schmidt)]

Replying to rlhamil:

I had a similar (maybe not exactly the same) problem upgrading to curl @7.56.1_1+ssl+universal on Snow Leopard.

Yes, that is exactly the same problem.

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign:

As with wget, curl apparently uses libpsl if present. I didn't know that, which is why these ports didn't handle this situation. Now that I know, I will investigate the proper solution: either make curl depend on libpsl unconditionally as was done for wget; or prevent the use of libpsl; or add a variant so the user can choose.

Jeremy added the libpsl dependency in [8b1bbc9f8412c8fd1d71db2096113cca4a4e42bc/macports-ports]. I guess that's ok since it doesn't adversely affect curl's distributability, but it does increase curl's recursive dependency count from 27 to 64. :/

[ryandesign (Ryan Carsten Schmidt)]

In 9fbfc98ccf4c559caf29570d4873d05e10de7a76/macports-ports:

curl: Revbump following addition of libpsl dependency

Closes: #55409

[Schamschula (Marius Schamschula)]

Replying to ryandesign:

In 9fbfc98ccf4c559caf29570d4873d05e10de7a76/macports-ports:

curl: Revbump following addition of libpsl dependency

Closes: #55409

This breaks curl on Leopard and Snow Leopard, as libpsl can't presently be installed on these OS versions. See #55440

[rlhamil]

I had no trouble upgrading to curl @7.57.0_1+ssl+universal on Snow Leopard. Alas, the log file is gone now, so I can't say just where it downloaded libpsl from.

myeye:~ root# port installed libpsl
The following ports are currently installed:
  libpsl @0.19.1_0
  libpsl @0.19.1_1
  libpsl @0.19.1_1+universal (active)

[lyssdod (Alex R)]

I'm still hitting this with 2.6.4 and 7.73.0, despite having libpsl as universal:

$ sudo port installed libpsl
The following ports are currently installed:
  libpsl @0.21.1-20200817_0+universal (active)

so sudo port -d install curl +ssl+universal results in

DEBUG: system: /usr/bin/cmp -s "/tmp/muniversal.urTTHiA9/1-curl-config" "/tmp/muniversal.urTTHiA9/2-curl-config"
Command failed: /usr/bin/cmp -s "/tmp/muniversal.urTTHiA9/1-curl-config" "/tmp/muniversal.urTTHiA9/2-curl-config"
Exit code: 1
Error: Failed to destroot curl: curl-config differs in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-i386//opt/local/bin and /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-x86_64//opt/local/bin and cannot be merged
DEBUG: Error code: NONE
DEBUG: Backtrace: curl-config differs in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-i386//opt/local/bin and /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_net_curl/curl/work/destroot-x86_64//opt/local/bin and cannot be merged

[lyssdod (Alex R)]

sudo port -d install curl +ssl+universal

[ryandesign (Ryan Carsten Schmidt)]

Alex, this ticket was resolved three years ago, so the cause of the problem you are experiencing is different, even if the error message is the same. Please file a new ticket and attach the two copies of curl-config to the new ticket so that we can see how they differ.