Make master_site_local and patch_site_local able to be suffixed with dist_subdir

[snowflake (Dave Evans)]

I have two Macs and several VMs

One Mac is set up as a server for distfiles. This is working well, but I have to set up the URL in the ports tree in file _resources/port1.0/fetch /mirror_sites.tcl

It would be nice if there was a configuration file for this setting which could be in /opt/local/etc/macports. I've looked for one but have not found it in the documentation.

This is a low priority enhancement request.

[ryandesign (Ryan Carsten Schmidt)]

The configuration option for this in macports.conf is called master_site_local. You probably also want to set patch_site_local to the same value. Example:

master_site_local   http://distfiles.local/:mirror
patch_site_local    http://distfiles.local/:mirror

These do not appear to be documented in the guide, so let's make this ticket be about the documentation request.

There is also archive_site_local which is supposed to usable to specify a local site for binaries. Example:

archive_site_local  http://packages.local/:tbz2

However we also have archive_sites.conf and I'm not sure of the relationship between those two—which one takes precedence for example.

[snowflake (Dave Evans)]

I've tried setting master_site_local and patch_site_local and they do not work. I tried them as upper-case environment variables and that does not work. I've put debugging ui_notice's in the Tcl to print out the list of sites at various places in the code and get nothing.

archive_site_local does work. Putting the sites in the file _resources/port1.0/fetch/mirror_sites.tcl works.

[snowflake (Dave Evans)]

I have looked at the code in fetch_common.tcl and portfetch.tcl. I have put lots of debugging prints in the code. I see that my local site is listed in various lists, then it disappears and I do not know why.

I have devised a crude yet effective fix. The idea is to append my site to the macports_distfiles list in fetch_common.tcl proc portfetch::mirrorsites

Here is the first part of the proc

# For a given mirror site type, e.g. "gnu" or "x11", check to see if there's a                                  
# pre-registered set of sites, and if so, return them.                                                          
proc portfetch::mirror_sites {mirrors tag subdir mirrorfile} {
    global name dist_subdir global_mirror_site env
    ui_notice "dje: get mirror sites mirrors: $mirrors, tag: $tag, subdir: $subdir\n"
    if {[file exists $mirrorfile]} {
        source $mirrorfile
    }

    if {![info exists portfetch::mirror_sites::sites($mirrors)]} {
        if {$mirrors ne $global_mirror_site} {
            ui_warn "[format [msgcat::mc "No mirror sites on file for class %s"] $mirrors]"
        }
        return {}
    }
# new code starts here
    if {$mirrors eq "macports_distfiles"} {
        if {  [ array names env MASTER_SITE_LOCAL ] != ""  } {
            lappend portfetch::mirror_sites::sites($mirrors) $env(MASTER_SITE_LOCAL)
        }
    }
# new code ends here

My knowledge of Tcl is not great and this patch will probably offend the Tcl purists.

[ryandesign (Ryan Carsten Schmidt)]

Replying to snowflake:

I've tried setting master_site_local and patch_site_local and they do not work. I tried them as upper-case environment variables and that does not work.

They "work", but not in a way that seems useful to me. Since it was never documented how they were intended to work, it's unclear to me if it "works" correctly or not.

If you set master_site_local http://distfiles.local/ or patch_site_local http://distfiles.local/ then MacPorts will look for distfiles or patchfiles there without adding dist_subdir to the end. For example, it would try to download the distfile for the zlib port from http://distfiles.local/zlib-1.2.11.tar.xz when you might actually want it to try to download from http://distfiles.local/zlib/zlib-1.2.11.tar.xz if your web server is serving the contents of /opt/local/var/macports/distfiles for example.

If you specify master_site_local http://distfiles.local/:mirror or patch_site_local http://distfiles.local/:mirror as I suggested earlier, then MacPorts interprets mirror as the name of a tag and only considers that URL when fetching distfiles that have that tag assigned (and probably no ports have a distfile with that tag which is why it doesn't end up being used). That's not what I expected or what I intended when I made that suggestion and probably nobody ever uses this behavior intentionally. Rather, I had intended for :mirror to be interpreted as the directive to treat this URL as a mirror and thus add dist_subdir to the end.

Your proposal adds master_site_local a second time and processes it in the way I intended, but that doesn't handle patch_site_local and it doesn't deal with the fact that an incorrectly-processed entry has already been added.

I think I can refactor the code so that the code that interprets the :mirror and :nosubdir directives can be reused when handling master_site_local and patch_site_local. The way MacPorts currently handles them is the same way it handles entries in mirror_sites.tcl that have the :nosubdir directive, but we might want to change it so that URLs that don't specify a behavior default to the :mirror behavior since that's probably what people want.

Moved documentation request to #62952.

[ryandesign (Ryan Carsten Schmidt)]

Support for MASTER_SITE_LOCAL was added in r5003 ([d9e3c3fca5c9db42b9cf7d581405ec342ae9e85f/macports-base])

Support for master_site_local was added in r5004 ([ee9f7e1045c8c6d56e3215bc1f18306e6e0a9cd4/macports-base])

Support for PATCH_SITE_LOCAL was added in r10223 ([72f038602927e68dd2d1e25a01e18c887947ff69/macports-base])

Support for patch_site_local was added in r74924 ([a90913b57f65f7f56cca661f8ca395307bef1f1b/macports-base])