ImageMagick @6.7.1-5_0 checksum mismatch

[gthb (Gunnlaugur Thor Briem)]

Upgrading outdated just now,

--->  Computing dependencies for ImageMagick
--->  Fetching archive for ImageMagick
--->  Attempting to fetch ImageMagick-6.7.1-5_0+q16.darwin_10.x86_64.tbz2 from http://packages.macports.org/ImageMagick
--->  Fetching ImageMagick
--->  Attempting to fetch ImageMagick-6.7.1-5.7z from http://imagemagick.linux-mirror.org/download/
--->  Verifying checksum(s) for ImageMagick
Error: Checksum (sha1) mismatch for ImageMagick-6.7.1-5.7z
Error: Checksum (rmd160) mismatch for ImageMagick-6.7.1-5.7z
Error: Target org.macports.checksum returned: Unable to verify file checksums

The distfile is 7,425,091 bytes and here are the checksums:

$ openssl dgst -sha1 /opt/local/var/macports/distfiles/ImageMagick/ImageMagick-6.7.1-5.7z 
SHA1(/opt/local/var/macports/distfiles/ImageMagick/ImageMagick-6.7.1-5.7z)= 8218b1551cc649c2607b1b95387234b04ad429e5
$ openssl dgst -rmd160 /opt/local/var/macports/distfiles/ImageMagick/ImageMagick-6.7.1-5.7z 
RIPEMD160(/opt/local/var/macports/distfiles/ImageMagick/ImageMagick-6.7.1-5.7z)= bac00fe7badd864b03716011711b50c9e6005845

[gthb (Gunnlaugur Thor Briem)]

main.log from failed ImageMagick upgrade

[gthb (Gunnlaugur Thor Briem)]

After removing the distfile and port clean ImageMagick, the build passes the checksum step (but it's getting the distfile from a different mirror):

$ sudo port upgrade outdated
--->  Computing dependencies for ImageMagick
--->  Fetching archive for ImageMagick
--->  Attempting to fetch ImageMagick-6.7.1-5_0+q16.darwin_10.x86_64.tbz2 from http://packages.macports.org/ImageMagick
--->  Fetching ImageMagick
--->  Attempting to fetch ImageMagick-6.7.1-5.7z from http://lil.fr.distfiles.macports.org/ImageMagick
--->  Verifying checksum(s) for ImageMagick
--->  Extracting ImageMagick
--->  Configuring ImageMagick
--->  Building ImageMagick

[ryandesign (Ryan Carsten Schmidt)]

I wish they would stop doing this to us.

[ryandesign (Ryan Carsten Schmidt)]

By which I meant, they've previously released different distfiles on different servers with the same version number. But in this case I don't see this. On all servers listed in the portfile, including imagemagick.linux-mirror.org, the checksums match the portfile. Perhaps your initial download got interrupted; the size you reported is smaller than the 7,445,429 bytes it should be.

[jankassens@…]

I'm getting the same error.

version:1
:msg:main --->  Computing dependencies for ImageMagick:info:main .:debug:main Checking for conflicts against ImageMagick
:debug:main Searching for dependency: cryptlib
:debug:main Didn't find receipt, going to depspec regex for: cryptlib
:debug:main Searching for dependency: p7zip
:debug:main Found Dependency: receipt exists for p7zip
 ...
:msg:main 
:debug:main Executing org.macports.main (ImageMagick)
:debug:archivefetch archivefetch phase started at Mon Aug 15 20:38:35 CEST 2011
:msg:archivefetch --->  Fetching archive for ImageMagick
:debug:archivefetch Executing org.macports.archivefetch (ImageMagick)
:info:archivefetch --->  ImageMagick-6.7.1-5_0+q16.darwin_11.x86_64.tgz doesn't seem to exist in /opt/local/var/macports/software/ImageMagick
:msg:archivefetch --->  Attempting to fetch ImageMagick-6.7.1-5_0+q16.darwin_11.x86_64.tgz from http://packages.macports.org/ImageMagick
:debug:archivefetch Fetching archive failed:: The requested URL returned error: 404
:debug:fetch fetch phase started at Mon Aug 15 20:38:36 CEST 2011
:notice:fetch --->  Fetching ImageMagick
:debug:fetch Executing org.macports.fetch (ImageMagick)
:debug:checksum checksum phase started at Mon Aug 15 20:38:36 CEST 2011
:notice:checksum --->  Verifying checksum(s) for ImageMagick
:debug:checksum Executing org.macports.checksum (ImageMagick)
:info:checksum --->  Checksumming ImageMagick-6.7.1-5.7z
:error:checksum Checksum (sha1) mismatch for ImageMagick-6.7.1-5.7z
:info:checksum Portfile checksum: ImageMagick-6.7.1-5.7z sha1 4b33f882d440bbbcc9b33eb5306fa33d1ef74fc3
:info:checksum Distfile checksum: ImageMagick-6.7.1-5.7z sha1 8218b1551cc649c2607b1b95387234b04ad429e5
:error:checksum Checksum (rmd160) mismatch for ImageMagick-6.7.1-5.7z
:info:checksum Portfile checksum: ImageMagick-6.7.1-5.7z rmd160 0053efb528ac6097b95ae47cab02ae053829957b
:info:checksum Distfile checksum: ImageMagick-6.7.1-5.7z rmd160 bac00fe7badd864b03716011711b50c9e6005845
:info:checksum The correct checksum line may be:
:info:checksum checksums           sha1    8218b1551cc649c2607b1b95387234b04ad429e5 \
                    rmd160  bac00fe7badd864b03716011711b50c9e6005845
:error:checksum Target org.macports.checksum returned: Unable to verify file checksums
:debug:checksum Backtrace: Unable to verify file checksums
    while executing
"$procedure $targetname"
:info:checksum Warning: the following items did not execute (for ImageMagick): org.macports.install org.macports.checksum org.macports.extract org.macports.patch org.macports.configure org.macports.build org.macports.destroot
:notice:checksum Log for ImageMagick is at: /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_graphics_ImageMagick/ImageMagick/main.log

[ryandesign (Ryan Carsten Schmidt)]

What server did your bad file come from? Tell me about the file. Is it a valid 7z archive? If so how do it's contents compare to the version we have on distfiles.macports.org? Or is it truncated? Or something else?

[juergen@…]

Same problem here. I could reproduce this while using the main mirror as well as the French one. The file I got there has this characteristics:

asgard:ImageMagick root# file ImageMagick-6.7.1-5.7z 
ImageMagick-6.7.1-5.7z: 7-zip archive data, version 0.3
asgard:ImageMagick root# ls -l ImageMagick-6.7.1-5.7z 
-rw-r--r--  1 macports  admin  7425091 14 Aug 01:41 ImageMagick-6.7.1-5.7z

But manually downloading the file from distfiles.macports.org got me a file with a different size:

asgard:ImageMagick root# ls -l ImageMagick-6.7.1-5.7z 
-rw-r--r--  1 macports  admin  7445429 18 Aug 19:43 ImageMagick-6.7.1-5.7z

That file passed the check and worked.

[ryandesign (Ryan Carsten Schmidt)]

juergen, same questions to you about the bad file you got.

[ryandesign (Ryan Carsten Schmidt)]

I've updated ImageMagick to 6.7.1-7 in r82707 which hopefully does not give you the same trouble. If it does, let me know, and provide the information requested above.

[juergen@…]

Replying to ryandesign@…:

juergen, same questions to you about the bad file you got.

OK, the servers I mentioned in my post, so about the file: Yes, it's a valid 7zip file. I've extracted the two file versions in two directories and diffed these directories:

asgard:tmp root# diff ImageMagick-6.7.1-5_from_distfiles/ ImageMagick-6.7.1-5_from_rsync/
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/Magick++ and ImageMagick-6.7.1-5_from_rsync/Magick++
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/PerlMagick and ImageMagick-6.7.1-5_from_rsync/PerlMagick
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/coders and ImageMagick-6.7.1-5_from_rsync/coders
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/config and ImageMagick-6.7.1-5_from_rsync/config
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/filters and ImageMagick-6.7.1-5_from_rsync/filters
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/images and ImageMagick-6.7.1-5_from_rsync/images
diff ImageMagick-6.7.1-5_from_distfiles/index.html ImageMagick-6.7.1-5_from_rsync/index.html
160c160
<   <img src="images/networkredux.png" width="140" height="31" alt="[sponsor]"
---
>   <img src="images/networkredux.png" alt="[sponsor]"
164c164
<   <a rel="follow" title="Sponsor: Autoteile" href="http://www.q11-shop.de">Autoteile</a><!-- 201205010540s suvalj -->
---
>   <a rel="follow" title="Sponsor: Web Hosting" href="http://www.micfo.com">Web Hosting</a><!-- 209901010090s golestan -->
167c167
<   <a rel="follow" title="Sponsor: Web Hosting" href="http://www.micfo.com">Web Hosting</a><!-- 209901010090s golestan -->
---
>   <a rel="follow" title="Sponsor: Autos part" href="http://www.pkwteile.de/autoteile">Autos part</a><!-- 201112010090s jr@pkwte... -->
179,181d178
<   <a rel="follow" title="Sponsor: Autos part" href="http://www.pkwteile.de/autoteile">Autos part</a><!-- 201112010090s jr@pkwte... -->
< </div>
< <div  class="sponsor">
206c203
<    <a rel="follow" title="Sponsor: Günstige Turbolader" href="http://www.autoteile-carparts.de/">Günstige Turbolader</a><!-- 20111101000030 -->
---
>    <a rel="follow" title="Sponsor: Webdesign" href="http://www.renehornig.com/">Webdesign</a><!-- 20111001000240 -->
231c228
< <p>The current release is ImageMagick 6.7.1-5.</p>
---
> <p>The current release is ImageMagick 6.7.1-1.</p>
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/ltdl and ImageMagick-6.7.1-5_from_rsync/ltdl
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/m4 and ImageMagick-6.7.1-5_from_rsync/m4
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/magick and ImageMagick-6.7.1-5_from_rsync/magick
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/scenes and ImageMagick-6.7.1-5_from_rsync/scenes
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/scripts and ImageMagick-6.7.1-5_from_rsync/scripts
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/tests and ImageMagick-6.7.1-5_from_rsync/tests
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/utilities and ImageMagick-6.7.1-5_from_rsync/utilities
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/wand and ImageMagick-6.7.1-5_from_rsync/wand
Common subdirectories: ImageMagick-6.7.1-5_from_distfiles/www and ImageMagick-6.7.1-5_from_rsync/www

Looks like an old HTML file got in there...?

[ryandesign (Ryan Carsten Schmidt)]

What are "the main mirror", "the French one", "from_distfiles" and "from_rsync"? Complete URLs to the files in question might be helpful, as would a complete diff (not a diff of just the top level directory). But again, I've already updated the port to 6.7.1-7 which should eliminate any problem you had with the 6.7.1-5 distfiles, unless they developers have posted differing files again, which as far as I can tell they have not (but as far as I can tell they did not for 6.7.1-5 either).

[juergen@…]

OK, so just in case you still want to look at this in detail:

"Main mirror" is what Macports comes preconfigured with, i.e. rsync://rsync.macports.org/release/tarballs/ports.tar. "French mirror" corresponds to rsync://lil.fr.rsync.macports.org/release/ports/ and setting

rsync_server il.fr.rsync.macports.org
rsync_dir release/base/

in macports.conf. I don't know of a way to reconstruct the complete URL of that file (is it ok to just concatenate filename and repository address?), so I hope that info is sufficient.

So these were the sources that gave me the file which triggered the error. Extracting that file gave me what I called ImageMagick-6.7.1-5_from_rsync in the diff above.

The other file version, for which the problem did not occur, was downloaded directly from distfiles.macports.org using

curl -O http://distfiles.macports.org/ImageMagick/ImageMagick-6.7.1-5.7z

and gave me what I called ImageMagick-6.7.1-5_from_distfiles above.

[juergen@…]

...and the output of a recursive diff was just attached as attachment:recursive_diff.bz2​ ; it was too large to paste in a code block.

[ryandesign (Ryan Carsten Schmidt)]

Replying to juergen@…:

"Main mirror" is what Macports comes preconfigured with, i.e. rsync://rsync.macports.org/release/tarballs/ports.tar. "French mirror" corresponds to rsync://lil.fr.rsync.macports.org/release/ports/ and setting

rsync_server il.fr.rsync.macports.org
rsync_dir release/base/

in macports.conf.

Those are the places that the portfiles are downloaded from, not the places the distfiles are downloaded from. For the latter, see the output of e.g. "port distfiles ImageMagick" (it varies by port).

[juergen@…]

Sorry! Here you go:

$ port distfiles imagemagick
--->  Distfiles for imagemagick
[ImageMagick-6.7.1-5.7z] /opt/local/var/macports/distfiles/ImageMagick/ImageMagick-6.7.1-5.7z
 sha1: 4b33f882d440bbbcc9b33eb5306fa33d1ef74fc3
 rmd160: 0053efb528ac6097b95ae47cab02ae053829957b
  http://image_magick.veidrodis.com/image_magick/ImageMagick-6.7.1-5.7z
  http://imagemagick.linux-mirror.org/download/ImageMagick-6.7.1-5.7z
  ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-6.7.1-5.7z
  ftp://ftp.fifi.org/pub/ImageMagick/ImageMagick-6.7.1-5.7z
  http://distfiles.macports.org/ImageMagick/ImageMagick-6.7.1-5.7z
  http://aarnet.au.distfiles.macports.org/pub/macports/mpdistfiles/ImageMagick/ImageMagick-6.7.1-5.7z
  http://her.gr.distfiles.macports.org/mirrors/macports/mpdistfiles/ImageMagick/ImageMagick-6.7.1-5.7z
  http://lil.fr.distfiles.macports.org/ImageMagick/ImageMagick-6.7.1-5.7z
  http://sea.us.distfiles.macports.org/macports/mpdistfiles/ImageMagick/ImageMagick-6.7.1-5.7z
  http://ykf.ca.distfiles.macports.org/MacPorts/mpdistfiles/ImageMagick/ImageMagick-6.7.1-5.7z
  http://svn.macports.org/repository/macports/distfiles/ImageMagick/ImageMagick-6.7.1-5.7z

[ryandesign (Ryan Carsten Schmidt)]

Replying to juergen@…:

Sorry! Here you go:

Yes, I know that is the list of available servers. I was trying to find out which of those you got the bad file from. As far as I have been able to determine, all of those servers are serving the correct file now. But:

Replying to juergen@…:

...and the output of a recursive diff was just attached as attachment:recursive_diff.bz2​ ; it was too large to paste in a code block.

Ok, I see lots of differences in html files, and a couple differences in source files. The developers have released different versions of allegedly the same version of the software in the past, for example ​6.5.6-0. Hopefully 6.7.1-7 has been released correctly. Let's call this issue closed, unless you experience problems with 6.7.1-7.