Opened 20 months ago

Last modified 7 months ago

#67186 new defect

Set Host header when invoking curl — at Version 1

Reported by: ryandesign (Ryan Carsten Schmidt) Owned by:
Priority: Normal Milestone: MacPorts Future
Component: base Version: 2.8.1
Keywords: Cc:
Port:

Description (last modified by ryandesign (Ryan Carsten Schmidt))

curl versions from 7.37.0 (2014) up to but not including 7.82.0 (2022) fail to handle the situation where a web site's canonical hostname ends with a period. See https://github.com/curl/curl/issues/8290. Here is an example of the problem:

% /usr/bin/curl --version
curl 7.79.1 (x86_64-apple-darwin21.0) libcurl/7.79.1 (SecureTransport) LibreSSL/3.3.5 zlib/1.2.11 nghttp2/1.45.1
Release-Date: 2021-09-22
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL UnixSockets
% /usr/bin/curl -I https://pyropus.ca./
HTTP/1.1 301 Moved Permanently
Date: Mon, 03 Apr 2023 01:17:16 GMT
Server: Apache/2.4.56 (Debian)
Location: https://pyropus.ca./
Content-Type: text/html; charset=iso-8859-1

The memtester port is an example that fails to fetch distfiles because of this (if the secondary master_sites entry is removed from the Portfile and --no-mirrors is used when fetching). It also fails livecheck.

It was pointed out to me by the author of memtester that affected curl versions can be made to work by setting the Host header properly, as in:

% /usr/bin/curl -H 'Host: pyropus.ca.' -I https://pyropus.ca./
HTTP/1.1 200 OK
Date: Mon, 03 Apr 2023 01:18:35 GMT
Server: Apache/2.4.56 (Debian)
Accept-Ranges: bytes
Vary: Accept-Encoding
X-Frame-Options: sameorigin
Content-Type: text/html

MacPorts should use a regexp to extract the host out of the URL and set the Host header every time it invokes curl for any purpose to work around buggy curl versions.

Change History (1)

comment:1 Changed 20 months ago by ryandesign (Ryan Carsten Schmidt)

Description: modified (diff)
Note: See TracTickets for help on using tickets.