`port -f archive foo` doesn't actually force the archive step if the state file says it's already done

[macports@…]

Daniel Luke asked me to file this report in ​this email.

I'll just demonstrate this with nmap +gtk2, but I'm pretty sure it will happen for any port. Sorry, no patch; I'm really busy and just trying to make stuff work.

Note that I make the archive, check the timestamp, then rerun the archive step with -f, but the timestamp is not updated. I then rerun it with -d for good measure.

~# port archive nmap +gtk2
--->  Fetching nmap
--->  Verifying checksum(s) for nmap
--->  Extracting nmap
--->  Configuring nmap
--->  Building nmap with target all
--->  Staging nmap into destroot
--->  Packaging tgz archive for nmap 4.20_0+gtk2

~# ls -l /opt/local/var/db/dports/packages/darwin/powerpc/nmap-4.20_0+gtk2.powerpc.tgz
-rw-rw----   1 root  wheel  680569 Dec 12 18:54 /opt/local/var/db/dports/packages/darwin/powerpc/nmap-4.20_0+gtk2.powerpc.tgz

~# cat /opt/local/var/db/dports/build/_opt_local_var_db_dports_sources_rsync.rsync.darwinports.org_dpupdate_dports_net_nmap/work/.darwinports.nmap.state 
variant: +gtk2
target: com.apple.main
target: com.apple.fetch
target: com.apple.checksum
target: com.apple.extract
target: com.apple.patch
target: com.apple.configure
target: com.apple.build
target: com.apple.destroot
target: com.apple.archive

~# port -f archive nmap +gtk2

~# ls -l /opt/local/var/db/dports/packages/darwin/powerpc/nmap-4.20_0+gtk2.powerpc.tgz
-rw-rw----   1 root  wheel  680569 Dec 12 18:54 /opt/local/var/db/dports/packages/darwin/powerpc/nmap-4.20_0+gtk2.powerpc.tgz

~# port -df archive nmap +gtk2
DEBUG: Found port in file:///opt/local/var/db/dports/sources/rsync.rsync.darwinports.org_dpupdate_dports/net/nmap
DEBUG: Changing to port directory: /opt/local/var/db/dports/sources/rsync.rsync.darwinports.org_dpupdate_dports/net/nmap
DEBUG: Requested variant powerpc is not provided by port nmap.
DEBUG: Requested variant darwin is not provided by port nmap.
DEBUG: Requested variant macosx is not provided by port nmap.
DEBUG: Executing variant gtk2 provides gtk2
DEBUG: Found port in file:///opt/local/var/db/dports/sources/rsync.rsync.darwinports.org_dpupdate_dports/net/libpcap
DEBUG: Changing to port directory: /opt/local/var/db/dports/sources/rsync.rsync.darwinports.org_dpupdate_dports/net/libpcap
DEBUG: Searching for dependency: libpcap
DEBUG: Found Dependency: receipt exists for libpcap
DEBUG: Found port in file:///opt/local/var/db/dports/sources/rsync.rsync.darwinports.org_dpupdate_dports/devel/openssl
DEBUG: Changing to port directory: /opt/local/var/db/dports/sources/rsync.rsync.darwinports.org_dpupdate_dports/devel/openssl
DEBUG: Searching for dependency: openssl
DEBUG: Found Dependency: receipt exists for openssl
DEBUG: Found port in file:///opt/local/var/db/dports/sources/rsync.rsync.darwinports.org_dpupdate_dports/devel/pcre
DEBUG: Changing to port directory: /opt/local/var/db/dports/sources/rsync.rsync.darwinports.org_dpupdate_dports/devel/pcre
DEBUG: Searching for dependency: pcre
DEBUG: Found Dependency: receipt exists for pcre
DEBUG: Found port in file:///opt/local/var/db/dports/sources/rsync.rsync.darwinports.org_dpupdate_dports/archivers/zlib
DEBUG: Changing to port directory: /opt/local/var/db/dports/sources/rsync.rsync.darwinports.org_dpupdate_dports/archivers/zlib
DEBUG: Searching for dependency: zlib
DEBUG: Found Dependency: receipt exists for zlib
DEBUG: Found port in file:///opt/local/var/db/dports/sources/rsync.rsync.darwinports.org_dpupdate_dports/x11/gtk2
DEBUG: Changing to port directory: /opt/local/var/db/dports/sources/rsync.rsync.darwinports.org_dpupdate_dports/x11/gtk2
DEBUG: Searching for dependency: gtk2
DEBUG: Found Dependency: receipt exists for gtk2
DEBUG: Skipping completed com.apple.main (nmap)
DEBUG: Skipping completed com.apple.fetch (nmap)
DEBUG: Skipping completed com.apple.checksum (nmap)
DEBUG: setting option extract.cmd to /opt/local/bin/bzip2
DEBUG: Skipping completed com.apple.extract (nmap)
DEBUG: Skipping completed com.apple.patch (nmap)
DEBUG: Skipping completed com.apple.configure (nmap)
DEBUG: Skipping completed com.apple.build (nmap)
DEBUG: Skipping completed com.apple.destroot (nmap)
DEBUG: Skipping completed com.apple.archive (nmap)

~#

This is with OS 10.4.8 on an iBook G4.

~% uname -a
Darwin maunakea.magicpacket.net 8.8.0 Darwin Kernel Version 8.8.0: Fri Sep  8 17:18:57 PDT 2006; root:xnu-792.12.6.obj~1/RELEASE_PPC Power Macintosh powerpc
~% port version
Version: 1.320
~%

I don't demonstrate it here, but if you delete the last line of .darwinports.nmap.state (target: com.apple.archive) and rerun port -f archive nmap +gtk2, I believe it will appropriately rebuild the archive and clobber the .tgz.

Summary: When I run port -f archive foo, I expect the archive step to happen regardless of the state file. If the state file says the archive is already built, nothing happens.

[danielluke (Daniel J. Luke)]

Something like this untested patch should fix this:

=== portarchive.tcl
==================================================================
--- portarchive.tcl     (revision 21020)
+++ portarchive.tcl     (local)
@@ -87,7 +87,7 @@
 
        # Determine if archive should be skipped
        set skipped 0
-       if {[check_statefile target com.apple.archive $target_state_fd]} {
+       if {[check_statefile target com.apple.archive $target_state_fd] && ![info exists options(ports_force)]} {
                return 0
        } elseif {[check_statefile target com.apple.unarchive $target_state_fd] && ([info exists ports_binary_only] && $ports_binary_only == "yes")} {
                ui_debug "Skipping archive ($portname) since binary-only is set"

[danielluke (Daniel J. Luke)]

Ok, so that fix doesn't work.

It looks like all of the targets work that way (force doesn't override the statefile - see line 587 portutil.tcl)

So, it looks like it needs to be fixed in both places? I'm not sure why portarchive is doing its own statefile checking ...

[kballard (Lily Ballard)]

See ​my email to the list about what I think proper behaviour here is.

[jmpalacios (Juan Manuel Palacios)]

I was looking into what I thought was a related bug but came up empty handed. I've found that in order to successfully force some steps a clean action is necessary in advance (to remove the state file, as indeed we seem to respect it a tad too much). So maybe we could refocus this bug on that, deciding when we should respect the state file and when to ignore it, if forcing is requested?

Other than that, forcing does work if there's no state file (as said, if the work dir is pruned by cleaning).

-jmpp

[danielluke (Daniel J. Luke)]

Going through my old tickets, I think I agree with Kevin's email (linked to above) and what needs to be fixed in this case is the port manpage (which I think is the only place where force is documented to override the state file).

[jmroot (Joshua Root)]

Marking as a dupe of #16061 since it's more general.