unable to fetch certain distfiles

[vkuznet (Valentin Kuznetsov)]

Hi, I encounter very annoying bug. Recently, I found that I can't upgrade certain ports. I even report a problem with curl package, see bug #39595. Turns out that it is more general problem which I'd like to put into separate ticket. So, certain packages are unable to download from certain URLs. Here is an example of erlang upgrade:

--->  Computing dependencies for erlang
--->  Fetching archive for erlang
--->  Attempting to fetch erlang-R16B01_0+ssl.darwin_11.x86_64.tbz2 from http://packages.macports.org/erlang
--->  Attempting to fetch erlang-R16B01_0+ssl.darwin_11.x86_64.tbz2 from http://mse.uk.packages.macports.org/sites/packages.macports.org/erlang
--->  Attempting to fetch erlang-R16B01_0+ssl.darwin_11.x86_64.tbz2 from http://lil.fr.packages.macports.org/erlang
--->  Fetching distfiles for erlang
--->  Attempting to fetch otp_src_R16B01.tar.gz from http://distfiles.macports.org/erlang
--->  Attempting to fetch otp_src_R16B01.tar.gz from http://sea.us.distfiles.macports.org/macports/distfiles/erlang
--->  Attempting to fetch otp_src_R16B01.tar.gz from http://mse.uk.distfiles.macports.org/sites/distfiles.macports.org/erlang
--->  Attempting to fetch otp_src_R16B01.tar.gz from http://lil.fr.distfiles.macports.org/erlang
--->  Attempting to fetch otp_src_R16B01.tar.gz from http://www.erlang.org/download/
--->  Attempting to fetch otp_src_R16B01.tar.gz from http://ykf.ca.distfiles.macports.org/MacPorts/mpdistfiles/erlang
--->  Attempting to fetch otp_src_R16B01.tar.gz from http://fco.it.distfiles.macports.org/mirrors/macports-distfiles/erlang
--->  Attempting to fetch otp_src_R16B01.tar.gz from http://her.gr.distfiles.macports.org/mirrors/macports/mpdistfiles/erlang
--->  Attempting to fetch otp_src_R16B01.tar.gz from http://aarnet.au.distfiles.macports.org/pub/macports/mpdistfiles/erlang
--->  Attempting to fetch otp_src_R16B01.tar.gz from http://cjj.kr.distfiles.macports.org/erlang
--->  Attempting to fetch otp_src_R16B01.tar.gz from http://jog.id.distfiles.macports.org/macports/mpdistfiles/erlang
--->  Attempting to fetch otp_src_R16B01.tar.gz from http://svn.macports.org/repository/macports/distfiles/erlang
Error: org.macports.fetch for port erlang returned: fetch failed
Please see the log file for port erlang for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_erlang/erlang/main.log
Error: Unable to upgrade port: 1
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets

as you can see my port was able to fetch erlang-R16B01_0+ssl.darwin_11.x86_64.tbz2, but then fail to fetch otp_src_R16B01.tar.gz from list of URLs. The main.log shows the following

from main.log
debug:fetch Fetching distfile failed:: The requested URL returned error: 403

for those URLs. Then port command exit with error status. I observed the pattern that this only happens from aforementioned URLs, while other URLs are working fine, for instance here is successful gcc upgrade

--->  Attempting to fetch gcc47-4.7.3_1.darwin_11.x86_64.tbz2 from http://packages.macports.org/gcc47
--->  Attempting to fetch gcc47-4.7.3_1.darwin_11.x86_64.tbz2 from http://mse.uk.packages.macports.org/sites/packages.macports.org/gcc47
--->  Attempting to fetch gcc47-4.7.3_1.darwin_11.x86_64.tbz2 from http://lil.fr.packages.macports.org/gcc47
--->  Fetching distfiles for gcc47
--->  Attempting to fetch gcc-4.7.3.tar.bz2 from ftp://ftp.gnu.org/gnu/gcc/gcc-4.7.3
--->  Attempting to fetch ecj-4.5.jar from http://mirrors.kernel.org/sources.redhat.com/java/
--->  Attempting to fetch ecj-4.5.jar from http://distfiles.macports.org/gcc47
--->  Attempting to fetch ecj-4.5.jar from ftp://sourceware.org/pub/java/
--->  Verifying checksum(s) for gcc47
--->  Extracting gcc47
--->  Applying patches to gcc47
--->  Configuring gcc47
--->  Building gcc47

So, I was unable to upgrade the following list: otp, curl, mongodb, clang, nginx, py27-sqlalchemy, but at the same time I was able to successfully upgrade gcc, mercurial, etc.

I really don't know what's going on, but I don't have any proxy. For the reference, I have OSX 10.7.5 and recent port version 2.1.3

Any suggestions are really welcome,
Thanks,
Valentin.

[larryv (Lawrence Velázquez)]

What happens if you try to fetch one of the failing distfiles manually? For example, this is what I see:

% curl -IL 'http://www.erlang.org/download/otp_src_R16B01.tar.gz'
HTTP/1.1 200 OK
Server: inets/5.7.1
Date: Sat, 06 Jul 2013 01:21:19 GMT
Content-Type: text/plain
Content-Length: 65813898

%

[ryandesign (Ryan Carsten Schmidt)]

Replying to vkuznet@…:

as you can see my port was able to fetch erlang-R16B01_0+ssl.darwin_11.x86_64.tbz2, but then fail to fetch otp_src_R16B01.tar.gz from list of URLs.

MacPorts was not able to fetch erlang-R16B01_0+ssl.darwin_11.x86_64.tbz2. If it had been, it wouldn't have needed to try next to download the source.

[vkuznet (Valentin Kuznetsov)]

That's what I did, I manually fetch some of the packages via curl, e.g.

vklaptop(09:05:18) macports > curl -IL 'http://www.erlang.org/download/otp_src_R16B01.tar.gz'
HTTP/1.1 200 OK
Server: inets/5.7.1
Date: Sat, 06 Jul 2013 13:05:20 GMT
Content-Type: text/plain
Content-Length: 65813898

so curl works just fine.

[vkuznet (Valentin Kuznetsov)]

Replying to ryandesign@…:

Replying to vkuznet@…:

as you can see my port was able to fetch erlang-R16B01_0+ssl.darwin_11.x86_64.tbz2, but then fail to fetch otp_src_R16B01.tar.gz from list of URLs.

MacPorts was not able to fetch erlang-R16B01_0+ssl.darwin_11.x86_64.tbz2. If it had been, it wouldn't have needed to try next to download the source.

Ahh, I see, I thought it did, anyway it worked for gcc and mercurial, so I'm pretty sure only specific URLs do not work.

[ryandesign (Ryan Carsten Schmidt)]

There is no MacPorts bug here. It sounds like a problem with your network. Please write to the macports-users mailing list if you need help diagnosing it.