Checksum mismatches for generated tarballs from GitHub

[raimue (Rainer Müller)]

Apparently, GitHub upgraded their software and changed the way git tarballs are generated due to a ​bugfix in upstream git. The produced tarballs will differ from those that have been generated before.

The checksums of the following ports have changed due to this (ran against [ffdb5922289aca17f166d14e3e91ecbe52278c10/macports-ports]):

Port	Maintainer
ALPSCore	galexv@…
apbs	howarth.at.macports@…,openmaintainer
box2d	stromnov@…,openmaintainer
certbot	mps@… Schamschula,openmaintainer
cgminer
chapel	takeshi@…,openmaintainer
cjdns	l2dy@… l2dy,openmaintainer
coffee-script	ryandesign@…,openmaintainer
cot	glenn@…
csvToLedger	ben@…
cusp	sean@…,openmaintainer
DepreciateForLedger	ben@…
docker
docker-machine
elixir	me@… milmazz
expense.txt	ben@…
faust-devel	ryandesign@…,aggraef@…
faust2-devel	ryandesign@…,aggraef@…
fbthrift	sean@…,openmaintainer
fbthrift	sean@…,openmaintainer
git-lfs	raimue@… raimue,openmaintainer
gst-plugins-cairovis	aronnax@… lpsinger
GTDtoCSV	ben@…
Gyoto	thibaut@…,openmaintainer
HermesApp	ryandesign@… ryandesign,openmaintainer
htcondor	aronnax@… lpsinger
hub	lists@…
ifeffit	joe.fowler@…,openmaintainer
Io
iTerm2	emer@…
jake	ryandesign@…,openmaintainer
just	casey@…
keybase	sean@…,openmaintainer
keybinder	ryandesign@…,openmaintainer
keybinder-3.0	ryandesign@…,openmaintainer
kops	pebrc,openmaintainer
LedgerScheduler	ben@…
libcdata
libcdatetime
libcerror
libcfile
libclocale
libcnotify
libcsplit
libcthreads
libewf
libgit2	juanrgar@…,devans@… dbevans
libiodbc	snc@…,openmaintainer
libphonenumber-cpp	devans@… dbevans,openmaintainer
litecoin	mk@…,openmaintainer
lua-cgilua
lua-rings
lua-wsapi
lua-xavante
LucenePlusPlus
mame
meson
mogenerator
mongo-cxx-driver	ryandesign@…,openmaintainer
mongo-tools	ryandesign@… ryandesign,openmaintainer
MongoHub	ryandesign@…,openmaintainer
mongrel2
mosesdecoder
myrepos	gjasny@…,openmaintainer
mythtv-core.25	ctreleaven@…,openmaintainer
mythtv-core.26	ctreleaven@…,openmaintainer
nu2
ocaml-async-core
OpenCoarrays	fanfarillo.gcc@…
pear-TCPDF
pficommon
phoenix	g5pw@…,openmaintainer
phpsh
povray
QGIS	vince@…
QLStephen
qoauth
racer	sean@…,openmaintainer
rekall
root5	jonesc@… cjones051073
scm_breeze	g5pw@…,openmaintainer
ShiftIt
subsurface
syncthing-0.14	leonardo@…,openmaintainer
synergy
tDOM	neumann@…
tbb	sean@…
teleport	graemelawes@… gclawes,openmaintainer
texworks	mojca@…,openmaintainer
tidyp
time.txt	ben@…
todotxt	snc@…,openmaintainer
tpkg	jheiss@…,openmaintainer
uzbl	ryandesign@… ryandesign,openmaintainer
webkit-sharp	ryandesign@…,openmaintainer
xctool	sean@…,openmaintainer
xmlroff
zorba

I do not know if GitHub actually invalidated the generated tarballs from their download caches, so the following might not be conclusive and checksums that are deemed correct might be subject to change in the future.

To track the progress, please do not edit above table as-is but remove fixed ports from the port field in this ticket.

See also this similar report for Homebrew: ​https://github.com/Homebrew/homebrew-core/issues/18044

For reference, above list was generated with these commands:

$ grep -l -R '^github\.setup' . | xargs -I{} sh -c 'basename $(dirname {})' | while read port; do sudo port clean --all \; fetch --no-mirrors \ checksum || echo $port > github-failed.txt; done
$ while read port; do port info --line --name --maintainer $port; done < <(sort -f github-failed.txt) | awk -F$'\t' '{printf "|| %-40s || %s ||\n", $2, $1}'

[raimue (Rainer Müller)]

In order to fix these ports, please apply the recipe from wiki:PortfileRecipes#stealth-updates by adding a dist_subdir.

[raimue (Rainer Müller)]

In 73194e3f3ba6e6435f28a4de68060cb7c98827d5/macports-ports:

git-lfs: Fix checksums for stealth distfile update

See: #54839

[nerdling (Jeremy Lavergne)]

I did get a changed distfile for todotxt (07842f8/macports-ports), but libiodbc seems to still be correct.

[raimue (Rainer Müller)]

Replying to nerdling:

but libiodbc seems to still be correct.

You are right, I only checked if the command was successful and the problem with libiodbc was unrelated and temporary (502 Bad Gateway). I noticed now that the list also contains some other errors such as 404 Not Found for some ports. However, they need to be looked into anyway, so I am not re-running the whole thing.

[Schamschula (Marius Schamschula)]

In 8de513662f8647017c9f850c0146606ed9121154/macports-ports:

certbot: revision bump got stealth update 2 GitHub

See: #54839

[lbschenkel (Leonardo Brondani Schenkel)]

There's a new syncthing release scheduled to happen in 3 days; since installing the pre-built binaries still works and this port has no variants that would force building from source, I think that the number of impacted users is on the low side and we can afford to wait so I can kill two birds with one stone. Sounds reasonable?

[raimue (Rainer Müller)]

Users of a non-default prefix will not receive binary archives, although I agree there are probably not many. Fixing the checksums will not affect existing installations in any way. We can assume the content of the distfiles has not changed as they are still based on the same git commit. There is no need to increase the revision when fixing the checksums and it will not cause any rebuilds.

[mojca (Mojca Miklavec)]

I planned to seize the opportunity to update texworks to 0.6.2 to avoid doing workarounds, but I cannot make it compile. I'll see if I can fix the problems quickly. If not, I'll do a workaround.

[l2dy (Zero King)]

In d7d06e889adaad74b8bd92d356d0bcd7f9aea989/macports-ports:

cjdns: Fix checksums for stealth distfile update

See: #54839

[Schamschula (Marius Schamschula)]

In 14721aee908d3a7a98becf1e6119d7c7893aebce/macports-ports:

iTerm2: update to version 3.0.15 (maintainer timeout)

Closes: #54806
also fixes GitHub stealth update:
See: #54839

[lbschenkel (Leonardo Brondani Schenkel)]

syncthing-0.14 has now been upgraded to a new upstream version, that also addresses the checksum problem

[joefowler (Joe Fowler)]

See #54890: I tried to follow the instructions and update port ifeffit. But as @ryandesign has politely pointed out, I did not actually have to change the checksums.

So can you check again that ifeffit belongs on this list? If it does, then I guess I am totally confused about what to do.

[raimue (Rainer Müller)]

I can confirm ifeffit was a false positive, follow-up in #54890.

[Dan R. K. Ports <drkp@…>]

In 7c281354f28c9ddbcdb19dddec135e7df2f030ff/macports-ports:

osxfuse: set dist_subdir to prevent github-caused checksum mismatch

See #54839
Closes: #54892
Closes: #54898

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #54910.

[cjones051073 (Chris Jones)]

tbb fixed (also updated to 2017_U8).

[bokertov]

Any update on when this will get fixed? This is preventing me from install packages such as Julia (see #55117). Thanks!

[raimue (Rainer Müller)]

As soon as you provide updated checksums. It just needs someone to work through these ports and update the checksums.

[lbschenkel (Leonardo Brondani Schenkel)]

Btw: syncthing-0.14 has been updated a while ago, I forgot to notify in this ticket.

[michaellass (Michael Lass)]

At least for libgit2 I created a pull request some time ago which just needs to be merged: ​https://github.com/macports/macports-ports/pull/900

[Michael Lass <michael.lass@…>]

In 1f90ea9f36086e27cbbf90ba36652d1202ea8c7d/macports-ports:

libgit2: set dist_subdir and update checksum

See #54839

[ctreleaven (Craig Treleaven)]

In aabf04dc178a3c445472383a47446ffecb4d7aab/macports-ports:

mythtv-core.28: update to fixes as of 2017-10-10

see #54839

[ctreleaven (Craig Treleaven)]

mythtv-core.28 was not on the list but was affected by this issue. Now fixed.

Will sudo port distcheck blah verify if a port suffers from this issue? Because I would expect mythweb.28 to be affected and it appears not to be.

[neverpanic (Clemens Lang)]

In e6a8d60bb52bcba99ddb01fc6470a7f9a9ce5ac9/macports-ports:

keybase: update to 1.0.33, openmaintainer

Fixes checksum failure due to GitHub regenerating their tarballs.

Closes: #54807
See: #54839

[mojca (Mojca Miklavec)]

In c95b459817df041b5281fc14ae9ce65e4bfd659c/macports-ports:

texworks: update to 0.6.2

See: #54839

[Alexander Gaenko <galexv@…>]

In e2515ef1caa7184e3e0fd46eb8ee30833488d0f0/macports-ports:

ALPSCore: update to 1.0.0

A new release. Also should resolve the tarball checksum mismatch.
See: #54839

[ctreleaven (Craig Treleaven)]

In 0185d8ff5ae99fbc3454163b59b8c65e72e21410/macports-ports:

myth*.27: bitrot fixes

mythtv-core.27, mythtv-plugins.27, mythweb.27 -

Update to most recent commit in Fixes/0.27 branch for each
which fixes mangled checksums problem (54839)
Update to use Python 5.2.6 (55208)
Change MythWeb paths to account for recent changes to Apache2

See #54839
See #55208

[ctreleaven (Craig Treleaven)]

Ports now obsolete and therefore checksums no longer relevant:

• mythtv-core.25: [14829fec8a6e4d888910197a0f47fff10b17032f/macports-ports]
• mythtv-core.26: [056d04d825a60266f409e0c94d89c48e627389c4/macports-ports]
• mythweb.25: [98b164f39ceeb9f7ebd2ff0866defde9fc74be5a/macports-ports]
• mythweb.26: [01c9b70862011ca711cda1bd3a298e29cd23a5d9/macports-ports]

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #55412 (docker).

[ryandesign (Ryan Carsten Schmidt)]

In 8449249ddc3ab5f608c3e3a5ecdcd19504ff68b2/macports-ports:

docker: Update to 17.05.0-ce

See: #54839

[mojca (Mojca Miklavec)]

root5 looks ok to me.

[lpsinger (Leo Singer)]

I fixed htcondor in [f52105c01a0a2905b62d04d9fb98b3bb4e418553/macports-ports].

[Eric A. Borisch <eborisch@…>]

In acca590955b08c0922fbfb3dc43855f38fceeb50/macports-ports:

rclone: Fix checksum & upgrade to 1.3.8

See also #54839 and #54959