Travis CI: Sometimes declared dependencies aren't installed

[ryandesign (Ryan Carsten Schmidt)]

At Zero King's request, I'm breaking this issue out from #56300.

Travis sometimes experiences errors that should be impossible, such a dependency which is definitely declared not being installed, which then results in a failure.

Here is an example of this:

• The PR: ​https://github.com/macports/macports-ports/pull/2874
• One of the resulting Travis CI builds: ​https://travis-ci.org/macports/macports-ports/jobs/446238426
• The log of the failure of that build: ​https://paste.macports.org/9e2079261585
• The relevant error from that log:

--->  Extracting coreutils
Error: Failed to extract coreutils: Failed to locate 'xz' in path: '/opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin' or at its MacPorts configuration time location, did you move it?
DEBUG: Error code: NONE

The coreutils port definitely has an extract dependency on xz, added by MacPorts because of its .xz distfile, and 2 of the other builds started from this PR built fine. (The 4th failed due to an unrelated issue.)

This issue is specific to whatever we're doing on Travis CI; I've never seen this problem with the buildbot.

[ryandesign (Ryan Carsten Schmidt)]

This problem continues to happen intermittently on Travis. ​Here's another example. ​The log says:

--->  Building py27-mistune

Traceback (most recent call last):
  File "setup.py", line 4, in <module>
    from setuptools import setup
ImportError: No module named setuptools

py27-mistune definitely declares a dependency on py27-setuptools.

[raimue (Rainer Müller)]

I took a closer look at your last example with py27-mistune. mpbb runs the following command to install the port. All dependencies should already be installed at this point, but all ports are deactivated.

if ! "${option_prefix}/bin/port" -dn install --unrequested "$depname" $deprequestedvariants; then
  ...

However, the log does not mention py27-setuptools or py27-cython in the list of dependencies in the log on paste.macports.org:

--->  Computing dependencies for py27-mistune
...
--->  Dependencies to be installed: python27 bzip2 db48 expat gettext libiconv ncurses libedit libffi openssl zlib python2_select python_select sqlite3

These two ports are the dependencies added only in the subport conditional. Are we maybe somehow parsing the Portfile with an incorrect ${subport} or ${name} which leads to the condition being evaluated to false...?

By the way, the only reason why Travis tried to build it from source appears to be some network/DNS outage. I would assume on the other macOS versions mpbb successfully downloaded the binary archive in this step and thus succeeded.

--->  py27-mistune-0.8.4_0.darwin_15.x86_64.tbz2 doesn't seem to exist in /opt/local/var/macports/incoming/verified
--->  Attempting to fetch py27-mistune-0.8.4_0.darwin_15.x86_64.tbz2 from https://packages.macports.org/py27-mistune
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
^M  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0DEBUG: Fetching archive failed:     Could not resolve host: packages.macports.org

[ryandesign (Ryan Carsten Schmidt)]

If we had the wrong subport (i.e. the py-mistune stub port), the python27 dependency wouldn't be there either, but it is.

What I'm noticing is that in the original report about coreutils, an extract dependency was missing, and in the py27-mistune report, a build dependency was missing. Both would not be needed if we were installing from an archive. Somewhere in MacPorts we have code that removes build and extract dependencies when we are using an archive, and I suspect that's where the problem is. Do you know where that code is?

[ryandesign (Ryan Carsten Schmidt)]

In src/macports1.0/macports.tcl we have:

# Determine dependency types required for target
proc macports::_deptypes_for_target {target workername} {
    switch -- $target {

        install     -
        activate    -
        {}          {
            if {[global_option_isset ports_binary_only] ||
                [$workername eval {registry_exists $subport $version $revision $portvariants}]
                || (![global_option_isset ports_source_only] && [$workername eval {_archive_available}])} {
                return "depends_lib depends_run"
            } else {
                return "depends_fetch depends_extract depends_patch depends_build depends_lib depends_run"
            }
        }

If an archive hasn't already been downloaded, [$workername eval {_archive_available}] uses curl to check if an archive is available on the server. I think the problem is that the Travis network connection was unavailable briefly—while we were running port archivefetch (all attempts to download an archive failed) and when we started running port fetch (the first distfile download attempt failed; the second one succeeded). By the time we ran port install the network was back up so it could see that an archive was available so it only added lib and run dependencies, but by that point we had already marked the archivefetch phase as complete so we tried a source build which failed because the build dependencies weren't there.

[ryandesign (Ryan Carsten Schmidt)]

Ok yes, with that in mind, I can reproduce the issue on my own machine:

• disconnect network

  $ sudo port clean --all py27-mistune
  $ sudo port -f deactivate py27-setuptools
  Warning: Deactivate forced.  Proceeding despite dependencies.
  --->  Deactivating py27-setuptools @40.8.0_0
  --->  Cleaning py27-setuptools
  $ sudo port archivefetch py27-mistune
  --->  Fetching archive for py27-mistune
  --->  Attempting to fetch py27-mistune-0.8.4_0.darwin_17.x86_64.tbz2 from https://packages.macports.org/py27-mistune
  

• reconnect network

  $ sudo port install py27-mistune
  --->  Computing dependencies for py27-mistune
  --->  Fetching distfiles for py27-mistune
  --->  Attempting to fetch mistune-0.8.4.tar.gz from https://files.pythonhosted.org/packages/source/m/mistune
  --->  Verifying checksums for py27-mistune
  --->  Extracting py27-mistune
  --->  Configuring py27-mistune
  --->  Building py27-mistune
  Error: Failed to build py27-mistune: command execution failed
  Error: See /opt/local/var/macports/logs/.../py27-mistune/main.log for details.
  Error: Follow https://guide.macports.org/#project.tickets to report a bug.
  Error: Processing of port py27-mistune failed
  $
  

I think the fix is in proc _archive_available, before it checks for an archive on the server, it should check if the archivefetch phase has been completed. If it has, it should return no.

[l2dy (Zero King)]

So this is a defect in base?

[ryandesign (Ryan Carsten Schmidt)]

Yes, you're off the hook :) Though Travis is still at fault for having an unreliable network. Could you maybe report that to them again?

[jmroot (Joshua Root)]

In 45620f022b4f5ca2754b45fbbf74941e488f30e1/macports-base (master):

Only mark archivefetch complete if it succeeded

Fixes: #57457