py-setuptools: setuptools-21.2.1.tar.gz not available

[RivetBenoit (Benoit Rivet)]

Upgrading py-setuptools fails with the following error :

--->  Fetching distfiles for py35-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from https://distfiles.macports.org/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://lil.fr.distfiles.macports.org/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from https://pypi.python.org/packages/source/s/setuptools/
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://mse.uk.distfiles.macports.org/sites/distfiles.macports.org/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from https://files.pythonhosted.org/packages/source/s/setuptools/
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://fco.it.distfiles.macports.org/mirrors/macports-distfiles/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://osl.no.distfiles.macports.org/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://nue.de.distfiles.macports.org/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://ykf.ca.distfiles.macports.org/MacPorts/mpdistfiles/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://sea.us.distfiles.macports.org/macports/distfiles/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://jnb.za.distfiles.macports.org/distfiles/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://cjj.kr.distfiles.macports.org/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://aarnet.au.distfiles.macports.org/pub/macports/mpdistfiles/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://nou.nc.distfiles.macports.org/pub/macports/distfiles.macports.org/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://jog.id.distfiles.macports.org/macports/distfiles/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://her.gr.distfiles.macports.org/py-setuptools
--->  Attempting to fetch setuptools-21.2.1.tar.gz from http://svn.macports.org/repository/macports/distfiles/py-setuptools
Error: org.macports.fetch for port py35-setuptools returned: fetch failed

Since py-setuptools was updated yesterday, the setuptools-21.2.1.tar.gz may have not made its way to all the macports mirrors.

Anyway, after downloading setuptools-21.2.1.tar.gz from ​https://www.versioneye.com/python/setuptools/21.2.1, everything installs fine. Should be easy to fix : it may be just of matter of waiting for setuptools-21.2.1.tar.gz to land on macports mirrors; or somehow the file was not put online and it's not too late to fix it.

[udbraumann]

Cc Me!

[udbraumann]

The problem you have is platform related and has nothing to do with py-setuptools. E.g. on my 10.5.8 box I had the same. It happens when the respective source is placed first on some https site and none of the http sites have already received a copy. As port relies on /usr/lib/libcurl.dylib which might provide somehow outdated encodings or ciphers (depending on your MacOSX version), not all https connections will work. In our case, the tarball setuptools-21.2.1.tar.gz first was only present under https://files.pythonhosted.org/packages/source/s/setuptools/, but port just was unable to fetch, probably since your libcurl.dylib (in fact the libcurl.4.dylib) was not built with the asn1 encoding. Or since the ca cert bundle in /usr/share/curl/curl-ca-bundle.crt libcurl.dylib (and also /usr/lib/curl) is using is outdated. Under these two circumstances port cannot download from websites using https.

The question might arise why MacPorts is not utilizing the port curl which both is modern concerning the ciphers and encodings plus provides a valid ca cert bundle.

I found a hack how to update libcurl.4.dylib on my 10.5.8 PPC box. I downloaded new sources from https://curl.haxx.se/download.html and unpacked the tarball (curl version 7.49.0) . I called the configure script with these parameters:

./configure --enable-shared --disable-static --enable-ldap --enable-ldaps --with-ca-bundle=/usr/share/curl/curl-ca-bundle.crt CC=gcc-mp-5 && make

Some switches I used might be a bit redundant here, and gcc-mp-4.5 or thelike might also do (I did not use Xcode's gcc-4.2 as I noticed some version mismatch between the openssl headers and libraries during configure). Note that I decided to directly refer to the system ca cert bundle, as I found in the full error log from previous py27-setuptools upgrade attempts that no ca cert problems had occurred, but just some asn1 related trouble.

So the rest of the story is short, I copied the resulting libcurl.4.dylib (from the lib/.libs folder) to /usr/lib (of course after making a backup copy from the previous system provided library file with the same name).

And voilà, now port automatically could grab the setuptools-21.2.1.tar.gz from https://files.pythonhosted.org/packages/source/s/setuptools/, while at that moment still none of the repositories using http were offering setuptools-21.2.1.tar.gz.

Admittedly, I am not 100% sure if my hack has no side effect at all, e.g. I hope there was no runtime dependency introduced between the MacPorts installation and the modified libcurl.4.dylib. At the moment I expect that similar https related trouble which has recently occurred with a number of other ports now no longer comes back on my box. As I also have a 10.6.8 box I btw. can tell that there was not such trouble as probably the system provided curl is newer or is just more complete.

[ryandesign (Ryan Carsten Schmidt)]

Replying to braumann@…:

The problem you have is platform related and has nothing to do with py-setuptools. E.g. on my 10.5.8 box I had the same. It happens when the respective source is placed first on some https site and none of the http sites have already received a copy. As port relies on /usr/lib/libcurl.dylib which might provide somehow outdated encodings or ciphers (depending on your MacOSX version), not all https connections will work. In our case, the tarball setuptools-21.2.1.tar.gz first was only present under https://files.pythonhosted.org/packages/source/s/setuptools/, but port just was unable to fetch, probably since your libcurl.dylib (in fact the libcurl.4.dylib) was not built with the asn1 encoding. Or since the ca cert bundle in /usr/share/curl/curl-ca-bundle.crt libcurl.dylib (and also /usr/lib/curl) is using is outdated. Under these two circumstances port cannot download from websites using https.

This analysis is correct. The only upstream download location uses https in a way that older systems' libcurl doesn't support, and the MacPorts mirrors have not yet mirrored the file. Mirroring can currently be delayed by as much as 4 days. See #50875.

The question might arise why MacPorts is not utilizing the port curl which both is modern concerning the ciphers and encodings plus provides a valid ca cert bundle.

MacPorts can't rely on a MacPorts-provided curl library existing. (It won't, for example, when you initially install MacPorts, since no ports have been installed yet at that point.) I've long thought that we should bundle a newer copy of libcurl with MacPorts to solve issues like this. I filed #51516 for this.

I found a hack how to update libcurl.4.dylib on my 10.5.8 PPC box. I downloaded new sources from https://curl.haxx.se/download.html and unpacked the tarball (curl version 7.49.0) . I called the configure script with these parameters:

./configure --enable-shared --disable-static --enable-ldap --enable-ldaps --with-ca-bundle=/usr/share/curl/curl-ca-bundle.crt CC=gcc-mp-5 && make

Some switches I used might be a bit redundant here, and gcc-mp-4.5 or thelike might also do (I did not use Xcode's gcc-4.2 as I noticed some version mismatch between the openssl headers and libraries during configure). Note that I decided to directly refer to the system ca cert bundle, as I found in the full error log from previous py27-setuptools upgrade attempts that no ca cert problems had occurred, but just some asn1 related trouble.

So the rest of the story is short, I copied the resulting libcurl.4.dylib (from the lib/.libs folder) to /usr/lib (of course after making a backup copy from the previous system provided library file with the same name).

And voilà, now port automatically could grab the setuptools-21.2.1.tar.gz from https://files.pythonhosted.org/packages/source/s/setuptools/, while at that moment still none of the repositories using http were offering setuptools-21.2.1.tar.gz.

Admittedly, I am not 100% sure if my hack has no side effect at all, e.g. I hope there was no runtime dependency introduced between the MacPorts installation and the modified libcurl.4.dylib. At the moment I expect that similar https related trouble which has recently occurred with a number of other ports now no longer comes back on my box. As I also have a 10.6.8 box I btw. can tell that there was not such trouble as probably the system provided curl is newer or is just more complete.

Please do not replace parts of OS X in this manner. We cannot support such custom configurations.

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign@…:

Mirroring can currently be delayed by as much as 4 days. See #50875.

Since the file has in the meantime been mirrored, I'll close this ticket since there's nothing further to do.

[udbraumann]

Replying to ryandesign@…:

MacPorts can't rely on a MacPorts-provided curl library existing. (It won't, for example, when you initially install MacPorts, since no ports have been installed yet at that point.) I've long thought that we should bundle a newer copy of libcurl with MacPorts to solve issues like this. I filed #51516 for this.

Thank you for filing this as enhancement idea! And don't forget to bundle the ca certs.

I found a hack how to update libcurl.4.dylib on my 10.5.8 PPC box. ...

Please do not replace parts of OS X in this manner. We cannot support such custom configurations.

Of course this hack is not seriously recommended. I was just curious if my speculation on the reasons for these download troubles was correct :-)