port selfupgrade to base v2.3.0 fails on G4

[paulccobb@…]

Today I've been trying to run 'port selfupgrade' on my PowerBook G4. It has Mac OS 10.5.8, and XCode version 3.1.4

After the command failed on several attempts, I reran with -v to get some information; at the outset, I see:

MacPorts base version 2.2.1 installed,
MacPorts base version 2.3.0 downloaded.

All appears to proceed ok until:

===> staging to destroot in vendor/tcllib-1.15
/usr/bin/tclsh8.6-64 `echo ./installer.tcl` \
		-pkg-path   /opt/local/var/macports/sources/rsync.macports.org/release/base/vendor/vendor-destroot/opt/local/libexec/macports/lib/. \
		-no-examples -no-html -no-nroff \
		-no-wait -no-gui -no-apps
/bin/sh: /usr/bin/tclsh8.6-64: Bad CPU type in executable
make[2]: *** [install-libraries] Error 126
make[1]: *** [destroot-tcllib] Error 2
make: *** [all] Error 1
Command failed: cd /opt/local/var/macports/sources/rsync.macports.org/release/base && CC=/usr/bin/cc OBJC=/usr/bin/cc ./configure --prefix=/opt/local --with-tclpackage=/Library/Tcl --with-install-user=root --with-install-group=admin --with-directory-mode=0755 --enable-readline && make SELFUPDATING=1 && make install SELFUPDATING=1
Exit code: 2
Error: Error installing new MacPorts base: command execution failed

The invocation of /usr/bin/tclsh8.6-64 seems incorrect; the 'file' command says:

/usr/bin/tclsh8.6-64: Mach-O universal binary with 2 architectures: [ppc64: Mach-O ppc64 executable] [x86_64: Mach-O 64-bit x86_64 executable]

On my PowerBook G4, clearly neither of those architectures is going to be any help.

Looks like a better choice would be:

/usr/bin/tclsh8.6:    Mach-O universal binary with 2 architectures: [ppc: Mach-O ppc executable] [i386: Mach-O i386 executable]

I'll continue to pursue this myself, in an effort to understand why the 64-bit tclsh8.6 is being selected on what's clearly a 32-bit host; but I'm sure someone familiar with MacPorts internals could get there much more quickly. I can provide more info if needed.

[ryandesign (Ryan Carsten Schmidt)]

MacPorts does not support Tcl 8.6. Mac OS X 10.4 and 10.5 ship with Tcl 8.4. Mac OS X 10.6, 10.7, 10.8 and 10.9 ship with Tcl 8.5. MacPorts 2.3.0 for the first time includes its own copy of Tcl 8.5, which is what we intend to use to avoid differences between Tcl versions and insulate ourselves from a hypothetical future version of OS X in which Apple upgrades Tcl to 8.6.

It seems you have installed a version of Tcl 8.6 into /usr/bin. That's probably a bad idea, and you should uninstall it. Possibly it has overwritten Apple's copy of Tcl 8.4 that was there. If so, you should put that back by reinstalling your OS and combo updates.

It does seem like a bug that we're trying to use /usr/bin/tclsh8.6-64 (or any tclsh from /usr/bin) and we should fix that if we can figure out a way to do so.

[neverpanic (Clemens Lang)]

This isn't a bug – the Tcllib package that we now install into our own private copy of Tcl needs a Tcl interpreter to install. Unfortunately we cannot use our private copy of Tcl to do this, because it hasn't been installed at this point.

The Tcllib configure script searches for a copy of tclsh in standard paths using ls $dir/tclsh[[8-9]]* and ls $dir/tclsh* and selects the first one that's a real file, so it's no surprise that it picks up /usr/bin/tclsh8.6-64 for you. If you need a newer Tcl, please install it in a directory other than /usr/bin.

[paulccobb@…]

Many thanks for coming back quickly with these two replies. I had picked up a new Tcl because I needed >=8.5 for something I was working on, but I can live with it being somewhere other than /usr/bin, at least as a temporary workaround.

I understand the point that you can't rely on any particular tlcsh path and/or version in the early stages of the installation. Couple of quick thoughts, for whatever they're worth:

1. Would it be possible to have the configure script do the search for a tclsh as at present, BUT also do a quick sanity check of whether the Tcl version is appropriate for the Mac OS version? If the check fails, exit with a message explaining that the installation process needs Tcl <major>.<minor> in /usr/bin (or wherever). Ideally the sanity check could also consider architecture, and not try to use a 64-bit Tcl on a 32-bit host (or vice versa).

2. Could there be a way e.g. a config option to specify the Tcl interpreter for use during the installation, so that if I know I've got a suitable Tcl somewhere on my machine, I can tell the MacPorts installer about it?

I let the new Tcl go into /usr/bin only because that was the default for the installer (think I used the package from ActiveState, but couldn't swear to it), so I'd guess others will run into this same issue sooner or later.

Again, many thanks for the explanations.

[ryandesign (Ryan Carsten Schmidt)]

Replying to paulccobb@…:

Many thanks for coming back quickly with these two replies. I had picked up a new Tcl because I needed >=8.5 for something I was working on, but I can live with it being somewhere other than /usr/bin, at least as a temporary workaround.

For example, you could install it using MacPorts. (sudo port install tcl)

I understand the point that you can't rely on any particular tlcsh path and/or version in the early stages of the installation. Couple of quick thoughts, for whatever they're worth:

1. Would it be possible to have the configure script do the search for a tclsh as at present, BUT also do a quick sanity check of whether the Tcl version is appropriate for the Mac OS version? If the check fails, exit with a message explaining that the installation process needs Tcl <major>.<minor> in /usr/bin (or wherever). Ideally the sanity check could also consider architecture, and not try to use a 64-bit Tcl on a 32-bit host (or vice versa).

It's probably not our job to verify that OS-provided software hasn't been tampered with. I mean, that exercise would have no end. We either have to move to being completely self-contained (which admittedly seems appealing sometimes), or we have to assume that certain OS-provided components are available.

But, it might not be unreasonable that we pick not the first tclsh found but the first tclsh that actually works. I'm still not quite clear on why we don't just use tclsh or /usr/bin/tclsh; not sure why we look for tclsh*.

2. Could there be a way e.g. a config option to specify the Tcl interpreter for use during the installation, so that if I know I've got a suitable Tcl somewhere on my machine, I can tell the MacPorts installer about it?

We do offer a lot of options like that already; from ./configure --help:

  --with-bsdmake=PATH     Path to alternate bsdmake/pmake command
  --with-bzip2=PATH       Path to alternate bzip2 command
  --with-cvs=PATH         Path to alternate cvs command
  --with-gnumake=PATH     Path to alternate gnumake command
  --with-gnutar=PATH      Path to alternate gnutar command
  --with-lzma=PATH        Path to alternate lzma command
  --with-make=PATH        Path to alternate make command
  --with-mtree=PATH       Path to alternate mtree command
  --with-open=PATH        Path to alternate open command
  --with-openssl=PATH     Path to alternate openssl command
  --with-rsync=PATH       Path to alternate rsync command
  --with-sed=PATH         Path to alternate sed command
  --with-svn=PATH         Path to alternate svn command
  --with-swig=PATH        Path to alternate swig command
  --with-tar=PATH         Path to alternate tar command
  --with-xar=PATH         Path to alternate xar command
  --with-xz=PATH          Path to alternate xz command

So we could conceivably let the user pick the tclsh path as well. On the other hand, the situation should never arise on OS X that tclsh doesn't work. All versions of OS X that MacPorts works on have a working tclsh as shipped by Apple; and until the just-released MacPorts 2.3.0, it was being used every time you used the port command.

I let the new Tcl go into /usr/bin only because that was the default for the installer (think I used the package from ActiveState, but couldn't swear to it), so I'd guess others will run into this same issue sooner or later.

It is an error for you to install software into /usr/bin. That directory is for Apple, not the user, to modify.

[neverpanic (Clemens Lang)]

Replying to ryandesign@…:

1. Would it be possible to have the configure script do the search for a tclsh as at present, BUT also do a quick sanity check of whether the Tcl version is appropriate for the Mac OS version? If the check fails, exit with a message explaining that the installation process needs Tcl <major>.<minor> in /usr/bin (or wherever). Ideally the sanity check could also consider architecture, and not try to use a 64-bit Tcl on a 32-bit host (or vice versa).

The problem with this check is that the file that picks up the broken Tcl in this case isn't actually written by MacPorts – it is part of a tarball of Tcllib which we ship unmodified, and we currently have no code to apply patches to these files before configuring. I agree that the code (it's SC_PROG_TCLSH from ​http://core.tcl.tk/tcllib/artifact/b93d99315ce5f4758a13021bc708a54da766fd94) could be better, e.g. by checking that the detected tclsh actually works, but that's for upstream tcllib to fix, IMO. We could file a bug with the tcllib developers at ​http://core.tcl.tk/tcllib/reportlist.

But, it might not be unreasonable that we pick not the first tclsh found but the first tclsh that actually works. I'm still not quite clear on why we don't just use tclsh or /usr/bin/tclsh; not sure why we look for tclsh*.

For the same reasons as above: We don't control the file that does that, and we don't have code in place to patch these files before running configure.

2. Could there be a way e.g. a config option to specify the Tcl interpreter for use during the installation, so that if I know I've got a suitable Tcl somewhere on my machine, I can tell the MacPorts installer about it?

We do offer a lot of options like that already; from ./configure --help:

We do not offer an option for the tclsh used by tcllib for installation, and the tcllib configure script does not have such an option. However, since MacPorts' configure script passes (almost) all its arguments to tcllib's configure script and that has a autoconf cache variable for the path, ./configure ac_cv_path_tclsh=/usr/bin/tclsh might work.

[cooljeanius (Eric Gallager)]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Ah, I didn't look at the code, didn't see it was tcllib doing the finding of tclsh. Yes, we should report this problem to the developers of tcllib. We might be wise to specify ac_cv_path_tclsh=/usr/bin/tclsh when the MacPorts build configures tcllib, to avoid the problem.

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign@…:

Yes, we should report this problem to the developers of tcllib.

Done: ​http://core.tcl.tk/tcllib/tktview/ba3b0d913c430b5cb2182eedbea80eba8a6ac87a

[ryandesign (Ryan Carsten Schmidt)]

The developers of tcllib resolved their bug in November 2014.

We updated our bundled tcllib to 1.17 which contained that fix in [03052c190065b04ebe81567f10868a8dfbcf8c26/macports-base].

We changed our configuration process to explicitly specify to use our tclsh in [8a5c8a2e01bf978162914c03f104d7353ddaf079/macports-base].

These fixes were part of MacPorts 2.4.0.