py27-appdirs: fetch failure

[noloader (Jeffrey Walton)]

My PowerMac has been having problems with python27 for months. sudo port clean did not help. Search did not return obvious matches for python27 package.

$ sudo port upgrade outdated
--->  Computing dependencies for kerberos5
--->  Fetching archive for kerberos5
--->  Attempting to fetch kerberos5-1.16_0.darwin_9.ppc.tbz2 from http://packages.macports.org/kerberos5
--->  Attempting to fetch kerberos5-1.16_0.darwin_9.ppc.tbz2.rmd160 from http://packages.macports.org/kerberos5
--->  Installing kerberos5 @1.16_0
--->  Cleaning kerberos5
--->  Computing dependencies for kerberos5
--->  Deactivating kerberos5 @1.15.2_0
--->  Cleaning kerberos5
--->  Activating kerberos5 @1.16_0
--->  Cleaning kerberos5
--->  Computing dependencies for libunwind-headers
--->  Fetching archive for libunwind-headers
--->  Attempting to fetch libunwind-headers-3.9.1_1.darwin_9.noarch.tbz2 from http://packages.macports.org/libunwind-headers
--->  Attempting to fetch libunwind-headers-3.9.1_1.darwin_9.noarch.tbz2 from http://ywg.ca.packages.macports.org/mirror/macports/packages/libunwind-headers
--->  Attempting to fetch libunwind-headers-3.9.1_1.darwin_9.noarch.tbz2 from http://lil.fr.packages.macports.org/libunwind-headers
--->  Fetching distfiles for libunwind-headers
--->  Attempting to fetch libunwind-3.9.1.src.tar.xz from http://distfiles.macports.org/llvm
--->  Verifying checksums for libunwind-headers
--->  Extracting libunwind-headers
--->  Applying patches to libunwind-headers
--->  Configuring libunwind-headers
--->  Building libunwind-headers
--->  Staging libunwind-headers into destroot
--->  Installing libunwind-headers @3.9.1_1
--->  Cleaning libunwind-headers
--->  Deactivating libunwind-headers @3.9.1_0
--->  Cleaning libunwind-headers
--->  Activating libunwind-headers @3.9.1_1
--->  Cleaning libunwind-headers
--->  Computing dependencies for py27-appdirs
--->  Fetching distfiles for py27-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from https://files.pythonhosted.org/packages/source/a/appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from https://pypi.python.org/packages/source/a/appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://distfiles.macports.org/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://ykf.ca.distfiles.macports.org/MacPorts/mpdistfiles/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://ywg.ca.distfiles.macports.org/mirror/macports/distfiles/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://lil.fr.distfiles.macports.org/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://sea.us.distfiles.macports.org/macports/distfiles/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://mse.uk.distfiles.macports.org/sites/distfiles.macports.org/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://fco.it.distfiles.macports.org/mirrors/macports-distfiles/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://nue.de.distfiles.macports.org/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://kmq.jp.distfiles.macports.org/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://pek.cn.distfiles.macports.org/macports/distfiles/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://aarnet.au.distfiles.macports.org/pub/macports/distfiles/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://cjj.kr.distfiles.macports.org/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://jnb.za.distfiles.macports.org/distfiles/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://jog.id.distfiles.macports.org/macports/distfiles/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://nou.nc.distfiles.macports.org/pub/macports/distfiles.macports.org/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://her.gr.distfiles.macports.org/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://osl.no.distfiles.macports.org/py-appdirs
Error: Failed to fetch py27-appdirs: Connection time-out after 30778 ms
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_python_py-appdirs/py27-appdirs/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.
--->  Cleaning kerberos5
--->  Cleaning libunwind-headers
--->  Uninstalling kerberos5 @1.15.2_0
--->  Cleaning kerberos5
--->  Uninstalling libunwind-headers @3.9.1_0
--->  Cleaning libunwind-headers

[ccorn]

Fetching has indeed been a problem for my apple-darwin9-powerpc too. To me, it looks like the MacPorts Tcl curl command does not use current CA bundles and/or does not handle redirects properly. There seems to have been an approach in use that tries HTTP instead of HTTPS on older Darwin versions, but that does not work well because most sites redirect to HTTPS nowadays. I suppose that the Tcl curl command needs an update that renders such kludges obsolete.

As a temporary workaround I have removed most mirror sites that the current fetch mechanism cannot handle with HTTP. For python eggs this would have left nothing, so I ended up with an own shell script that fetches the ports using sudo -u macports curl -L $URL. Of course this is an unsatisfactory state of affairs.

[ccorn]

Shell script that should mostly work like "port mirror". Needs gpg.

[noloader (Jeffrey Walton)]

To me, it looks like the MacPorts Tcl curl command does not use current CA bundles and/or does not handle redirects properly...

Thanks.

If it is the former (CA certs), then I can update them. Is there a specific location Tcl looks for them, or should I set an envar to tell Tcl where to look? Or something else?

Here is what I see, so I'm guessing I need to do more than updating the cacert file.

$ find /opt/local -name 'cacert*'
/opt/local/lib/perl5/vendor_perl/5.24/Mozilla/CA/cacert.pem

[noloader (Jeffrey Walton)]

Setting CURL_CA_BUNDLE (cURL) and SSL_CERT_DIR (OpenSSL) does not solve the problem. I can't find an environmental variable for Tcl.

 CURL_CA_BUNDLE=/Users/jwalton/.cacert/cacert.pem \
 SSL_CERT_DIR=/Users/jwalton/.cacert/cacert.pem \
 /opt/local/bin/port upgrade outdated

I am not sure what to try next.

[ccorn]

Does it work if you add fetch.ignore_sslcert=yes to the port command line?

My (superficial) impression is that this has to do with MacPorts implementation of the Tcl curl command. Then the file to look at might be ${prefix}/var/macports/sources/rsync.macports.org/macports/release/tarballs/base/src/pextlib1.0/curl.c. Or it might be the use of that Tcl command (curl fetch ...) elsewhere. In both cases this would be a base issue, not a port issue for py-appdirs. Python ports just aggravate the problem because none of their mirrors work well with the fetch configuration under Darwin 9 and earlier.

[noloader (Jeffrey Walton)]

Replying to ccorn:

Does it work if you add fetch.ignore_sslcert=yes to the port command line?

Thanks @ccorn. It looks like no joy with fetch.ignore_sslcert=yes.

# /opt/local/bin/port install py27-appdirs fetch.ignore_sslcert=yes
--->  Computing dependencies for py27-appdirs
--->  Fetching distfiles for py27-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from https://pypi.python.org/packages/source/a/appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from https://files.pythonhosted.org/packages/source/a/appdirs
...
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://her.gr.distfiles.macports.org/py-appdirs
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://osl.no.distfiles.macports.org/py-appdirs
Error: Failed to fetch py27-appdirs: Connection time-out after 30722 ms

[ryandesign (Ryan Carsten Schmidt)]

I have mirrored py-appdirs for you now; try again in an hour.

Duplicate of #51516 and #53347.

[noloader (Jeffrey Walton)]

Perfect, thank you very much. Things are working as expected on this old iron.

# /opt/local/bin/port install py27-appdirs
--->  Computing dependencies for py27-appdirs
--->  Fetching distfiles for py27-appdirs
...
--->  Attempting to fetch appdirs-1.4.3.tar.gz from http://distfiles.macports.org/py-appdirs
--->  Verifying checksums for py27-appdirs
--->  Extracting py27-appdirs
--->  Configuring py27-appdirs
--->  Building py27-appdirs
--->  Staging py27-appdirs into destroot
--->  Installing py27-appdirs @1.4.3_0
--->  Activating py27-appdirs @1.4.3_0
--->  Cleaning py27-appdirs
--->  Updating database of binaries
--->  Scanning binaries for linking errors
      [                                        ]  71.6 %
...

Awesome job. Thank you again.

Out of morbid curiosity, what is your mirror doing that no ​other mirrors are doing? It seems like the other mirrors should be following your lead. Or, half should do the old thing and half should do the new thing.