getopt portfile checksums not matched

[teige-mp@…]

Having trouble installing packages that depend on getopt (e.g. texlive). OS is Yosemite, migration & reinstallation of previous ports successful.

$ sudo port install getopt attempts to fetch archive getopt-1.1.5_1.darwin_14.x86-64.tbz2, which apparently doesn't exist, then defaults to fetching distfiles getopt-1.1.5.tar.gz from frodo.looijaard.name which provides a html webpage instead of the tarball. (This is the first, less important, problem.)

--->  Fetching archive for getopt
--->  Attempting to fetch getopt-1.1.5_1.darwin_14.x86_64.tbz2 from http://packages.macports.org/getopt
--->  Attempting to fetch getopt-1.1.5_1.darwin_14.x86_64.tbz2 from http://lil.fr.packages.macports.org/getopt
--->  Attempting to fetch getopt-1.1.5_1.darwin_14.x86_64.tbz2 from http://nue.de.packages.macports.org/macports/packages/getopt
--->  Fetching distfiles for getopt
--->  Attempting to fetch getopt-1.1.5.tar.gz from http://software.frodo.looijaard.name/getopt/files/
--->  Verifying checksums for getopt
Error: Checksum (md5) mismatch for getopt-1.1.5.tar.gz
Error: Checksum (sha1) mismatch for getopt-1.1.5.tar.gz
Error: Checksum (rmd160) mismatch for getopt-1.1.5.tar.gz

Obviously the checksums fail since it's actually a webpage that is fetched.

So I tried (separately) downloading the tarball from 2 places: ​http://frodo.looijaard.name/system/files/software/getopt/getopt-1.1.5.tar.gz ​http://distfiles.macports.org/getopt/ cleaned, moved the file to /opt/local/var/macports/distfiles/getopt/getopt-1.1.5.tar.gz and sudo port install getopt, again. The checksums still don't match. The checksums for both copies I downloaded match each other, but not the portfile checksum. Error logs are identical for either file used:

:info:checksum Portfile checksum: getopt-1.1.5.tar.gz md5 546acfdb67ba16b8658a45ade9a64d5d
:info:checksum Distfile checksum: getopt-1.1.5.tar.gz md5 f6405207663f5376f52de35146d52363

:info:checksum Portfile checksum: getopt-1.1.5.tar.gz sha1 9090eb46ac92f2fd2749ca4121e81aaad40f325d
:info:checksum Distfile checksum: getopt-1.1.5.tar.gz sha1 b90321438fa0952240e25b396c7f1e3f031fafae

:info:checksum Portfile checksum: getopt-1.1.5.tar.gz rmd160 5cdef902ada45ba5e8a66a1357be8189b0cc634f
:info:checksum Distfile checksum: getopt-1.1.5.tar.gz rmd160 87eb1e0028edf0bc086fe256b1256a8bbf7eeea2

Macports still attempts to fetch archive darwin_14 files before it uses the downloaded distfile.

--->  Fetching archive for getopt
--->  Attempting to fetch getopt-1.1.5_1.darwin_14.x86_64.tbz2 from http://packages.macports.org/getopt
--->  Attempting to fetch getopt-1.1.5_1.darwin_14.x86_64.tbz2 from http://lil.fr.packages.macports.org/getopt
--->  Attempting to fetch getopt-1.1.5_1.darwin_14.x86_64.tbz2 from http://nue.de.packages.macports.org/macports/packages/getopt
--->  Fetching distfiles for getopt
--->  Verifying checksums for getopt
Error: Checksum (md5) mismatch for getopt-1.1.5.tar.gz
Error: Checksum (sha1) mismatch for getopt-1.1.5.tar.gz
Error: Checksum (rmd160) mismatch for getopt-1.1.5.tar.gz

I cannot find a file whose checksums match the portfile checksums. Perhaps the portfile checksums want to match this file getopt-1.1.5_1.darwin_14.x86_64.tbz2 which macports attempts to fetch first, and which apparently does not exist (neither I nor macports can find it on the internet). However, the error log does say "Portfile checksum: getopt-1.1.5.tar.gz". Maybe I'm misunderstanding a difference between archives and distfiles, as pertains to fetching? (I'm new to macports) I don't know if, but doubt that, this is a case of stealth updating, since the copies I downloaded were last updated on their servers in 2012 and 2013 respectively, and were both the same.

[teige-mp@…]

Error log upon sudo port install getopt with downloaded 1.1.5 in distfiles dir

[larryv (Lawrence Velázquez)]

Replying to teige-mp@…:

$ sudo port install getopt attempts to fetch archive getopt-1.1.5_1.darwin_14.x86-64.tbz2, which apparently doesn't exist, then defaults to fetching distfiles getopt-1.1.5.tar.gz from frodo.looijaard.name which provides a html webpage instead of the tarball. (This is the first, less important, problem.)

Nope, this is the only problem. For some reason, upstream redirects ​the URL to ​a webpage instead of to another distfile. The port’s master_sites should be fixed.

So I tried (separately) downloading the tarball from 2 places:

• ​http://frodo.looijaard.name/system/files/software/getopt/getopt-1.1.5.tar.gz
• ​http://distfiles.macports.org/getopt/

cleaned, moved the file to /opt/local/var/macports/distfiles/getopt/getopt-1.1.5.tar.gz and sudo port install getopt, again. The checksums still don't match. The checksums for both copies I downloaded match each other, but not the portfile checksum. Error logs are identical for either file used:

:info:checksum Portfile checksum: getopt-1.1.5.tar.gz md5 546acfdb67ba16b8658a45ade9a64d5d
:info:checksum Distfile checksum: getopt-1.1.5.tar.gz md5 f6405207663f5376f52de35146d52363

:info:checksum Portfile checksum: getopt-1.1.5.tar.gz sha1 9090eb46ac92f2fd2749ca4121e81aaad40f325d
:info:checksum Distfile checksum: getopt-1.1.5.tar.gz sha1 b90321438fa0952240e25b396c7f1e3f031fafae

:info:checksum Portfile checksum: getopt-1.1.5.tar.gz rmd160 5cdef902ada45ba5e8a66a1357be8189b0cc634f
:info:checksum Distfile checksum: getopt-1.1.5.tar.gz rmd160 87eb1e0028edf0bc086fe256b1256a8bbf7eeea2

The files you mention have the correct checksums.

% port distfiles getopt | grep rmd160
 rmd160: 5cdef902ada45ba5e8a66a1357be8189b0cc634f
% openssl dgst -rmd160 <(curl -Ls http://frodo.looijaard.name/system/files/software/getopt/getopt-1.1.5.tar.gz)
RIPEMD160(/dev/fd/11)= 5cdef902ada45ba5e8a66a1357be8189b0cc634f
% openssl dgst -rmd160 <(curl -Ls http://distfiles.macports.org/getopt/getopt-1.1.5.tar.gz)
RIPEMD160(/dev/fd/11)= 5cdef902ada45ba5e8a66a1357be8189b0cc634f
%

Do those commands produce different output for you?

Perhaps the portfile checksums want to match this file getopt-1.1.5_1.darwin_14.x86_64.tbz2

They do not.

which macports attempts to fetch first, and which apparently does not exist (neither I nor macports can find it on the internet). However, the error log does say "Portfile checksum: getopt-1.1.5.tar.gz". Maybe I'm misunderstanding a difference between archives and distfiles, as pertains to fetching? (I'm new to macports)

By default, MacPorts attempts to fetch a pre-built binary archive that is appropriate for your platform. If it cannot find one, it downloads the source code and builds. The checksums only pertain to the source code distribution; binary archives are verified by other means.

I don't know if, but doubt that, this is a case of stealth updating, since the copies I downloaded were last updated on their servers in 2012 and 2013 respectively, and were both the same.

The distfiles are the same as ever, but the upstream HTTP redirect is wrong.

[teige-mp@…]

Thanks for the help with understanding this Larry. Those commands you list do provide exactly the same output for me. I am not sure what to do next. Why would the HTTP redirect error prevent the checksums matching given I've fetched the tarball myself? (Where does macports look-up the correct checksums?)

I wonder whether I do not understand what a portfile and a distfile are. I thought the portfile checksums were the 'target' against which the checksums calculated from fetched distfiles were compared.

I came to this conclusion because the portfile checksums in the error log remained constant whether the html was erroneously fetched, or I circumvented this provided the tarball in the correct directory, whereas the distfile checksums differed.

Error Log with regular sudo port install getopt (fetches html):

:notice:fetch --->  Attempting to fetch getopt-1.1.5.tar.gz from http://software.frodo.looijaard.name/
getopt/files/
:debug:fetch Privilege de-escalation not attempted as not running as root.
:debug:checksum checksum phase started at Wed Nov 12 20:47:42 GMT 2014
:notice:checksum --->  Verifying checksums for getopt
:debug:checksum Executing org.macports.checksum (getopt)
:info:checksum --->  Checksumming getopt-1.1.5.tar.gz
:debug:checksum Calculated (md5) is f7e366d5dc3eb22d3e2901e56f7f0160
:error:checksum Checksum (md5) mismatch for getopt-1.1.5.tar.gz
:info:checksum Portfile checksum: getopt-1.1.5.tar.gz md5 546acfdb67ba16b8658a45ade9a64d5d
:info:checksum Distfile checksum: getopt-1.1.5.tar.gz md5 f7e366d5dc3eb22d3e2901e56f7f0160
:debug:checksum Calculated (sha1) is 3826c7087e7fc019243db0e9ef4ada0cd643e995
:error:checksum Checksum (sha1) mismatch for getopt-1.1.5.tar.gz
:info:checksum Portfile checksum: getopt-1.1.5.tar.gz sha1 9090eb46ac92f2fd2749ca4121e81aaad40f325d
:info:checksum Distfile checksum: getopt-1.1.5.tar.gz sha1 3826c7087e7fc019243db0e9ef4ada0cd643e995
:debug:checksum Calculated (rmd160) is e822226e00bd350a919a4d434a21e7185157d36b
:error:checksum Checksum (rmd160) mismatch for getopt-1.1.5.tar.gz
:info:checksum Portfile checksum: getopt-1.1.5.tar.gz rmd160 5cdef902ada45ba5e8a66a1357be8189b0cc634f
:info:checksum Distfile checksum: getopt-1.1.5.tar.gz rmd160 e822226e00bd350a919a4d434a21e7185157d36b
:notice:checksum ***
:notice:checksum The non-matching file appears to be HTML.

Error Log with getopt-1.1.5.tar.gz provided in distfiles/getopt directory: (identical for tarballs from different sources)

:notice:fetch --->  Fetching distfiles for getopt
:debug:fetch euid/egid changed to: 0/0
:debug:fetch chowned /opt/local/var/macports/distfiles/getopt to macports
:debug:fetch euid/egid changed to: 502/501
:debug:fetch Executing org.macports.fetch (getopt)
:debug:fetch Privilege de-escalation not attempted as not running as root.
:debug:checksum checksum phase started at Wed Nov 12 20:56:49 GMT 2014
:notice:checksum --->  Verifying checksums for getopt
:debug:checksum Executing org.macports.checksum (getopt)
:info:checksum --->  Checksumming getopt-1.1.5.tar.gz
:debug:checksum Calculated (md5) is f6405207663f5376f52de35146d52363
:error:checksum Checksum (md5) mismatch for getopt-1.1.5.tar.gz
:info:checksum Portfile checksum: getopt-1.1.5.tar.gz md5 546acfdb67ba16b8658a45ade9a64d5d
:info:checksum Distfile checksum: getopt-1.1.5.tar.gz md5 f6405207663f5376f52de35146d52363
:debug:checksum Calculated (sha1) is b90321438fa0952240e25b396c7f1e3f031fafae
:error:checksum Checksum (sha1) mismatch for getopt-1.1.5.tar.gz
:info:checksum Portfile checksum: getopt-1.1.5.tar.gz sha1 9090eb46ac92f2fd2749ca4121e81aaad40f325d
:info:checksum Distfile checksum: getopt-1.1.5.tar.gz sha1 b90321438fa0952240e25b396c7f1e3f031fafae
:debug:checksum Calculated (rmd160) is 87eb1e0028edf0bc086fe256b1256a8bbf7eeea2
:error:checksum Checksum (rmd160) mismatch for getopt-1.1.5.tar.gz
:info:checksum Portfile checksum: getopt-1.1.5.tar.gz rmd160 5cdef902ada45ba5e8a66a1357be8189b0cc634f
:info:checksum Distfile checksum: getopt-1.1.5.tar.gz rmd160 87eb1e0028edf0bc086fe256b1256a8bbf7eeea2
:info:checksum The correct checksum line may be:
:info:checksum checksums           md5     f6405207663f5376f52de35146d52363 \
                    sha1    b90321438fa0952240e25b396c7f1e3f031fafae \
                    rmd160  87eb1e0028edf0bc086fe256b1256a8bbf7eeea2
:error:checksum org.macports.checksum for port getopt returned: Unable to verify file checksums

Thanks for the help & apologies for my neophyte ways.

[ryandesign (Ryan Carsten Schmidt)]

Whoever maintains the web server for the getopt project rearranged their URLs, and did not have the foresight to install proper HTTP redirects at the old previously-valid URLs. Instead they apparently installed a catch-all HTTP redirect, sending all requests at the old server, including requests for tarballs and other non-HTML files, to the main HTML page at the new server. Had they not installed any redirects at all, MacPorts would have recognized that the old server no longer existed and continued on to the next server. Or had they installed correct redirects, MacPorts would have been able to find the file at the new server. But because they installed a redirect that delivered a 302 Found ("the file you want is at this new URL") response followed by a 200 OK ("here is the file you requested") response, MacPorts assumed it had been given the file it requested, then had to exit when the checksums of the incorrect file it was given did not match those that were expected (as listed in the portfile).

Larry fixed the problem by updating the URLs in the portfile in r128063.

[larryv (Lawrence Velázquez)]

Replying to teige-mp@…:

I am not sure what to do next.

Trash whatever distfiles you have, fetch the latest Portfile, and install again.

% sudo port clean --all getopt
% sudo port selfupdate
% sudo port install getopt
% sudo port unsetrequested getopt

[teige-mp@…]

Thanks guys!