Opened 11 years ago

Closed 5 years ago

#40518 closed enhancement (fixed)

github portgroup: offer the option to use archive URLs instead of tarball URLs

Reported by: fred.brunner@… Owned by: macports-tickets@…
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc: ryandesign (Ryan Carsten Schmidt), gorticus (Jason Mitchell), RJVB (René Bertin), larryv (Lawrence Velázquez), ctreleaven (Craig Treleaven), yan12125 (Chih-Hsuan Yen), chrstphrchvz (Christopher Chavez), MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)
Port: github

Description

Trying to update a Github-hosted project via Macports, the tarball pathname seems to have changed from 'tarball' to 'archive'. With the default option in place for the github.master_sites config parameter, attempted fetch of the tarball fails, if changed to 'archive' it works fine. I may be missing something about how this is supposed to work, but I have attached a diff file for the portgroup definition with this change in case it is useful.

Attachments (1)

github-1.0.tcl.diff (591 bytes) - added by fred.brunner@… 11 years ago.
updated Githib portgroup template

Download all attachments as: .zip

Change History (20)

Changed 11 years ago by fred.brunner@…

Attachment: github-1.0.tcl.diff added

updated Githib portgroup template

comment:1 Changed 11 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: ryandesign@… added
Port: github added; git-extras removed

The portgroup works fine for me as is.

Making the suggested change seems to cause the checksums of existing ports' distfiles to change, which would necessitate changes to every port using the github portgroup, so before considering doing so I would want to understand the benefit.

comment:2 in reply to:  1 Changed 10 years ago by gorticus (Jason Mitchell)

Replying to ryandesign@…:

The portgroup works fine for me as is.

Making the suggested change seems to cause the checksums of existing ports' distfiles to change, which would necessitate changes to every port using the github portgroup, so before considering doing so I would want to understand the benefit.

Perhaps I'm restating things that are already known, but from the discussion over on #46241 ...

While playing w/ the GitHub API, I discovered that the old API is redirected to a legacy API, e.g., the prior

https://github.com/user/repo/{tarball,zipball}/{branch,tag,sha}

is redirected to

https://codeload.github.com/user/repo/legacy.{tar.gz,zip}/{branch,tag,sha}

However, based on the file structure of the returned archive, it's not clear this is a drop-in replacement, i.e., the top level director is user-repo-sha/. So it seems likely that checksums using the existing would have to be revisited.

Alternatively, for GitHub'ers using the v3 API, perhaps just by default, could you grant github.tarball_from a new archive or v3 option, e.g.,

archive {
  github.master_sites ${github.homepage}/archive/${version}
}
default { ...

It may also be possible to query the repo via the API to determine what method it used to to provide distfiles, e.g., gitflow AVH Edition v3 items of interest:

"has_downloads": true,
"archive_url": "https://api.github.com/repos/petervanderdoes/gitflow/{archive_format}{/ref}",
"releases_url": "https://api.github.com/repos/petervanderdoes/gitflow/releases{/id}",
"downloads_url": "https://api.github.com/repos/petervanderdoes/gitflow/downloads",

unfortunately, querying downloads_url returns an empty set, e.g.,

$ curl -o- https://api.github.com/repos/petervanderdoes/gitflow/downloads
[

]

comment:3 Changed 10 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: jason-macports@… added

I've never looked at any version of github API. I've just looked at the github web site to determine what URLs exist, and coded the github portgroup to match.

I may some day:

  • add archive URL support to the github portgroup via a new option "github.tarball_from archive"
  • add "github.tarball_from tags" to all ports currently relying on that being the default, along with a comment advising the portfile author to remove it and switch to archives with the next update
  • change the default for github.tarball_from to archive

However, since "tags" and "archive" tarballs have exactly the same contents (with just a different enclosing directory name, and thus a different checksum), it is not a high priority for me.

Really, the most convincing argument I have right now for doing this is so that I will no longer need to tell people to stop submitting portfiles in which they have overridden the github portgroup's default master_sites with the archive URL, which has happened now more often than I can remember.

comment:4 Changed 10 years ago by gorticus (Jason Mitchell)

git's interface is more stable than GitHub's API. I suspect we'll be a generation or two down the road in VCS tools before git clone is deprecated.

comment:5 Changed 10 years ago by ryandesign (Ryan Carsten Schmidt)

Yes, but in MacPorts we prefer to use distfiles, because they can be mirrored and checksummed and don't have to be fetched every time.

comment:6 Changed 10 years ago by ryandesign (Ryan Carsten Schmidt)

Summary: github portgroup github.master_sites should point to .../archive/... not .../tarball/...github portgroup: offer the option to use archive URLs instead of tarball URLs
Type: defectenhancement
Version: 2.2.0

comment:7 Changed 9 years ago by larryv (Lawrence Velázquez)

Cc: rjvbertin@… added

Has duplicate #47895.

comment:8 Changed 8 years ago by larryv (Lawrence Velázquez)

Cc: larryv@… added

Cc Me!

comment:9 Changed 8 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)

Cc: MarcusCalhoun-Lopez added

comment:10 Changed 7 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)

comment:11 Changed 7 years ago by ctreleaven (Craig Treleaven)

Cc: ctreleaven added

comment:12 in reply to:  3 Changed 7 years ago by l2dy (Zero King)

Replying to ryandesign:

However, since "tags" and "archive" tarballs have exactly the same contents (with just a different enclosing directory name, and thus a different checksum), it is not a high priority for me.

Really, the most convincing argument I have right now for doing this is so that I will no longer need to tell people to stop submitting portfiles in which they have overridden the github portgroup's default master_sites with the archive URL, which has happened now more often than I can remember.

One more argument: NeoMutt provides sha256 checksum and GPG signature for "archive" tarballs, which obviously can't be verified with "tags" tarballs. e.g. https://github.com/neomutt/neomutt/releases/tag/neomutt-20180323

comment:13 Changed 6 years ago by yan12125 (Chih-Hsuan Yen)

Cc: yan12125 added

comment:14 Changed 6 years ago by chrstphrchvz (Christopher Chavez)

Cc: chrstphrchvz added

comment:15 Changed 6 years ago by chrstphrchvz (Christopher Chavez)

A PR to add this has been open for several months: https://github.com/macports/macports-ports/pull/2587

comment:16 Changed 5 years ago by herbygillot (Herby Gillot)

The Github PR has been closed and merged, this ticket can be closed. You can now do:

github.tarball_from archive
Last edited 5 years ago by herbygillot (Herby Gillot) (previous) (diff)

comment:17 Changed 5 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)

Cc: MarcusCalhoun-Lopez removed

comment:18 Changed 5 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)

Cc: MarcusCalhoun-Lopez added

comment:19 Changed 5 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)

Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.