#14891 closed enhancement (fixed)
PATCH: use the fastest mirror in fetch phase
| Reported by: | jmroot (Joshua Root) | Owned by: | macports-tickets@… |
|---|---|---|---|
| Priority: | Normal | Milestone: | MacPorts 1.7.0 |
| Component: | base | Version: | 1.7.0 |
| Keywords: | Cc: | raimue (Rainer Müller) | |
| Port: |
Description
Here's a patch which makes the fetch phase ping each of the candidate download sites, and then sort the list in ascending order of ping time. This is my first attempt at writing Tcl, so no doubt it could be done better, and I welcome any advice.
Some issues that need to be thought about:
- We probably want to special-case some mirrors. We always want to try MacPorts svn last, and the primary site for each mirror group should be second-last.
- Caching. Is it worth the effort? Maybe pinging every mirror every time is fine. When do we refresh cache entries?
- User hints/overrides. If the user tells us that certain mirrors are good, we should always try them first when eligible.
Attachments (5)
Change History (16)
comment:1 Changed 17 years ago by wsiegrist@…
Changed 17 years ago by jmroot (Joshua Root)
| Attachment: | sortsites.diff added |
|---|
comment:2 Changed 17 years ago by jmroot (Joshua Root)
Updated the patch to try svn.macports.org last, ping with only one packet, and use a 1-second timeout in the interest of speed. Slow mirrors won't be sorted in quite the correct order, but I don't think that that's a real problem. Working mirrors that just don't respond to ping will go last in any case, but again, things should still generally be better with the patch than without.
It would be ideal to run all the pings in parallel, but I don't know how to do that.
Changed 17 years ago by raimue (Rainer Müller)
| Attachment: | sortsites2.diff added |
|---|
comment:3 Changed 17 years ago by raimue (Rainer Müller)
Added 'sortsites2.diff‘; an updated version of the patch which avoids hardcoding 'svn.macports.org', but uses the fallback_mirror_list where this host actually comes from.
comment:4 Changed 17 years ago by jmroot (Joshua Root)
Updated: don't re-ping hosts that have already been pinged. Saves a lot of time in cases where hosts appear in the list several times.
Changed 17 years ago by jmroot (Joshua Root)
| Attachment: | sortsites3.diff added |
|---|
comment:7 Changed 17 years ago by jmroot (Joshua Root)
Updated: Recognise invalid hostnames like image_magick.veidrodis.com. (Sigh.) Check that the output from ping can be parsed as a number. Switched to using 3 ping packets and a 3-second timeout, since we effectively only have to wait for it once now that the pings run in parallel. Also simplified the code a bit.
Changed 17 years ago by raimue (Rainer Müller)
| Attachment: | sortsites4.diff added |
|---|
Changed 17 years ago by raimue (Rainer Müller)
| Attachment: | sortsites4.2.diff added |
|---|
comment:8 Changed 17 years ago by raimue (Rainer Müller)
Updated sortsites4.diff: If there is only one host (besides the fallbacks), don't ping or sort.
comment:9 Changed 17 years ago by raimue (Rainer Müller)
| Cc: | raimue@… added |
|---|
This patch works just great. I encourage you to commit it, jmr!
comment:10 Changed 17 years ago by jmroot (Joshua Root)
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
Committed in r35748. Thanks for the help!
comment:11 Changed 16 years ago by blb@…
| Milestone: | MacPorts base enhancements → MacPorts 1.7.0 |
|---|
I tested with and without the patch on 2 random ports: less and php5.
less: the svn.macosforge.org sites won and were tried first, even though they dont have the files. You should probably exclude macosforge and macports sites from the ping test until we have mirroring going.
php5: it took longer to fetch because it waited up to 5s for each mirror it seems. Maybe try to find a winner with "-t1", then "-t5" to minimize the impact of downed mirrors.
Overall I like the idea though.