port 2.2.1: Error: org.macports.archivefetch for port libiconv returned: Failed to fetch signature for archive: The requested URL returned error: 400 Bad Request

[itslaksh@…]

Lakshmis-MacBook-Pro:~ lreguna$ sudo port -fp install gettext
Password:
--->  Computing dependencies for gettext
--->  Dependencies to be installed: libiconv
--->  Fetching archive for libiconv
--->  Attempting to fetch libiconv-1.14_0.darwin_13.x86_64.tbz2 from http://packages.macports.org/libiconv
--->  Attempting to fetch libiconv-1.14_0.darwin_13.x86_64.tbz2.rmd160 from http://packages.macports.org/libiconv
Error: org.macports.archivefetch for port libiconv returned: Failed to fetch signature for archive: The requested URL returned error: 400 Bad Request
Error: Failed to install libiconv
Please see the log file for port libiconv for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_textproc_libiconv/libiconv/main.log
Error: The following dependencies were not installed: libiconv
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets
Error: Processing of port gettext failed
--->  Scanning binaries for linking errors: 100.0%
--->  No broken files found.

[itslaksh@…]

log file

[itslaksh@…]

Cc Me!

[vincent.emanuele@…]

Cc Me!

[vincent.emanuele@…]

I just reviewed this ticket and I have the exact same problem. I've been researching this for 1.5 days with no success. Here is a summary of what I've noticed.

Setup:

• MBP Retina 2013 running OSX 10.9.1
• Xcode: Version 5.0.2 (5A3005) + Command-line tools
• Macports Version: Installed using binary via ​https://distfiles.macports.org/MacPorts/MacPorts-2.2.1-10.9-Mavericks.pkg
• Bash config: Completely clean/empty

Symptoms

• I'm experiencing exactly the same broken libiconv issue that itslaksh@… pointed out. My main.log file looks very similar.
• Other packages break too. Confirmed the same error message for:
  • All packages that depend on libiconv (there are a lot of them, and they fail at the libiconv install step)
  • openssl
• However, the following commands/ports install okay
  • sudo port selfupdate
  • sudo port install zlib
  • bzip2
  • libpng
  • freetype
• I've attached the output of running $ sudo port -d install libiconv as the attached file libiconv.debug to the ticket

Things I've tried

• Wipe everything and reinstall
  • Uninstalled Macports according to the directions: ​https://guide.macports.org/chunked/installing.macports.uninstalling.html
  • Uninstalled Xcode by moving /Applications/Xcode.app to the Trash bin
  • Installed Xcode (I've tried 2 ways, from developer.apple.com & from the App store with same result)
  • Accepted Xcode license, reinstalled command-line tools
  • Reinstalled Macports from binary (link given above)
  • Problem completely reproduced
• Look for related trac tickets in Macports (search for "bad request")
  • Tickets: 41849 (this ticket), 39949, 33715, 28590, and 15901
  • None of them seem particularly related
• Google: googled for macports "bad request"
  • Looked through several hits, was not able to see what actions I needed to take to resolve this
• Looked at the code
  • It looks like the problem occurs in this file: /opt/local/share/macports/Tcl/package1.0/portarchivefetch.tcl
    • Lines 242 - 245
  • I could not figure out the cause after putting some debug statements in that script

Please let me know if there's anything I can do to help you all diagnose and resolve this outstanding issue, as it's crippled my ability to use macports and I'd love to resolve this ASAP!

[vincent.emanuele@…]

Output of the command: $ sudo port -d install libiconv > libiconv.debug 2>&1

[nerdling (Jeremy Lavergne)]

From the log, you can piece together the URL that MacPorts is attempting: ​http://packages.macports.org/libiconv/libiconv-1.14_0.darwin_13.x86_64.tbz2.rmd160

When I ask curl for a HEAD request of that, I get 200 OK:

curl -I http://packages.macports.org/libiconv/libiconv-1.14_0.darwin_13.x86_64.tbz2.rmd160

If you run the curl command do you also get 200, or 400?

[neverpanic (Clemens Lang)]

To further debug this, can you run tcpdump while trying to download the file like this:

sudo /usr/sbin/tcpdump -w macports-tcpdump.pcap host packages.macports.org

then open a new shell and attempt to install libiconv. After it failed, press Ctrl+C to stop recording packages and attach the file macports-tcpdump.pcap.

Please also provide the output of otool -L /opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib.

Maybe MacPorts should also try the other mirrors for the signature file rather than failing when the first mirrors fails to serve it.

[vincent.emanuele@…]

Output of running command: $ sudo tcpdump -i en0 -w macports-tcpdump.pcap

[vincent.emanuele@…]

Output of command: $ otool -L /opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib

[mkae (Marko Käning)]

Cc Me!

[neverpanic (Clemens Lang)]

Can you provide the unmodified pcap with the host packages.macports.org filter as described above? I'd like to open it in Wireshark and make sure there's no binary junk or weird packages being sent and that's not really possible with the textual output you've attached. Using the filter should ensure there's no private data in the file either.

The otool output looks fine.

Are you currently in a country that might be affected by US export restrictions, such as Iran, etc.? Are you behind a (corporate) proxy server?

[vincent.emanuele@…]

Output of command: $ sudo tcpdump -w macports-tcpdump.pcap host packages.macports.org

[vincent.emanuele@…]

Thanks for your quick responses! I just attached the unmodified pcap file.

I am in the United States, and we've had a couple new hires within the past few weeks set up macports successfully on similar hardware. Since they don't have the same problem, it feels like it must be a local setting on my computer or a config issue?

[neverpanic (Clemens Lang)]

It's a legitimate bad request, because it's using HTTP/1.1 but doesn't send a Host header. I should have paid more attention to your filtered output, it also contains the same problem. The headers for the first GET request are:

GET /libiconv/libiconv-1.14_0.darwin_13.x86_64.tbz2 HTTP/1.1
User-Agent: MacPorts 2.2.1 libcurl/7.30.0
Host: packages.macports.org
Accept: */*

but the second GET doesn't send Accept or Host:

GET /libiconv/libiconv-1.14_0.darwin_13.x86_64.tbz2.rmd160 HTTP/1.1
User-Agent: MacPorts 2.2.1 libcurl/7.30.0

I wonder why it doesn't and whether we can do something to force curl to send the headers, but it's too late to investigate further now.

[vincent.emanuele@…]

According to ​http://stackoverflow.com/questions/356705/how-to-send-a-header-using-a-http-request-through-a-curl-call, it looks like we could force curl to put the Host and Accept in the header by adding:

curl --header "Host: packages.macports.org" --header "Accept: */*" <keep the rest of the curl call the same>

This may be a band-aid on the underlying problem, but I'll take it if it works.

Do you have any suggestions for work-arounds? I suppose manually downloading the package is an idea?

[neverpanic (Clemens Lang)]

MacPorts does not use the curl executable, but links against libcurl and wraps the C interface in a few different Tcl commands. I think the problem might be related to source:trunk/base/src/pextlib1.0/curl.c@117017:369-375#L365​, although the documentation says this should strip the existing header, and it would expect why it works the first time (where the exact same code is being executed).

I think this might actually be a bug in curl.

[neverpanic (Clemens Lang)]

Manually downloading might certainly work, yes. You can try that, but it's tedious, if it keeps failing with other packages. You could also try a different mirror by removing packages.macports.org from the file at $(port dir yubico-pam)/../../_resources/port1.0/fetch/archive_sites.tcl. If that helps, we can draft the necessary directives to put into macports.conf to make this change permanent (otherwise the next selfupdate will revert it).

[vincent.emanuele@…]

I removed packages.macports.org from the file you suggested. The install is working now.

$ sudo port install libiconv
--->  Fetching archive for libiconv
--->  Attempting to fetch libiconv-1.14_0.darwin_13.x86_64.tbz2 from http://mse.uk.packages.macports.org/sites/packages.macports.org/libiconv
--->  Attempting to fetch libiconv-1.14_0.darwin_13.x86_64.tbz2.rmd160 from http://mse.uk.packages.macports.org/sites/packages.macports.org/libiconv
--->  Installing libiconv @1.14_0
--->  Activating libiconv @1.14_0
--->  Cleaning libiconv
--->  Updating database of binaries: 100.0%
--->  Scanning binaries for linking errors: 100.0%
--->  No broken files found.
$ sudo port install openssl
--->  Computing dependencies for openssl
--->  Fetching archive for openssl
--->  Attempting to fetch openssl-1.0.1f_0.darwin_13.x86_64.tbz2 from http://mse.uk.packages.macports.org/sites/packages.macports.org/openssl
--->  Attempting to fetch openssl-1.0.1f_0.darwin_13.x86_64.tbz2.rmd160 from http://mse.uk.packages.macports.org/sites/packages.macports.org/openssl
--->  Installing openssl @1.0.1f_0
--->  Activating openssl @1.0.1f_0
--->  Cleaning openssl
--->  Updating database of binaries: 100.0%
--->  Scanning binaries for linking errors: 100.0%
--->  No broken files found.

So yes, please! What do I need to do to make sure this change is permanent? I really depend on MacPorts for all kinds of command-line goodness at work. Thanks very much for your response.

[neverpanic (Clemens Lang)]

Adding host_blacklist packages.macports.org to your macports.conf in /opt/local/etc/macports/ should make MacPorts ignore this server forever.

[raimue (Rainer Müller)]

I doubt this is still a problem after switching the hosting site for packages.macports.org.

[ryandesign (Ryan Carsten Schmidt)]

A similar problem was reported against a different server in #43689. We didn't get enough information to know if the cause was the same, and we still don't know what we could do about it. Both reports were on Mavericks, so maybe that indicates a bug in the Mavericks version of the curl library.