PortIndex2MySQL.tcl: can't find package Tclx

[ryandesign (Ryan Carsten Schmidt)]

With MacPorts base trunk, the PortIndex2MySQL.tcl script​ is failing as follows:

$ cat /tmp/portsdb.log 
To: PortIndex2MySQL@example.com
From: macports-mgr@lists.macosforge.org
Subject: PortIndex2MySQL run failure on Thursday 2014-04-03 at 20:56:39


can't find package Tclx
    while executing
"package require Tclx"
    (file "/opt/local/share/macports/Tcl/macports1.0/macports.tcl" line 39)
    invoked from within
"source /opt/local/share/macports/Tcl/macports1.0/macports.tcl"
    ("package ifneeded macports 1.0" script)
    invoked from within
"package require macports "
Failed to load the macports1.0 Tcl package: can't find package Tclx

I'm worried other contrib scripts may be affected as well.

[jmroot (Joshua Root)]

Currently the shebang says to use /opt/local/bin/tclsh, which was never right. It needs to use the same tclsh as port(1), which would now be ${prefix}/libexec/macports/bin/tclsh8.5.

OTOH port_binary_distributable.tcl still works with /usr/bin/tclsh, though that should be changed too.

[ryandesign (Ryan Carsten Schmidt)]

Replying to jmr@…:

Currently the shebang says to use /opt/local/bin/tclsh, which was never right.

Well, it used to work fine that way.

It needs to use the same tclsh as port(1), which would now be ${prefix}/libexec/macports/bin/tclsh8.5.

Damn. Because the script I actually want to use requires Tcl 8.6.

[ryandesign (Ryan Carsten Schmidt)]

I would love if we could find a way to fix all the Tcl scripts so that they use MacPorts tclsh8.5, but that the MacPorts prefix does not need to be hardcoded into them. For example, we already know the technique of having the script use #!/bin/sh as the first line, then re-launching the script under tclsh. Perhaps this technique could be modified to determine the MacPorts prefix programmatically (using e.g. dirname $(dirname $(which port))).

[jmroot (Joshua Root)]

Replying to ryandesign@…:

Replying to jmr@…:

It needs to use the same tclsh as port(1), which would now be ${prefix}/libexec/macports/bin/tclsh8.5.

Damn. Because the script I actually want to use requires Tcl 8.6.

You definitely can’t use 8.6 to run any script that uses the macports API because of the try incompatibility. But apart from that, probably any Tcl installation that has Tclx and tcllib would more or less work.

[ryandesign (Ryan Carsten Schmidt)]

Replying to jmr@…:

Replying to ryandesign@…:

Replying to jmr@…:

It needs to use the same tclsh as port(1), which would now be ${prefix}/libexec/macports/bin/tclsh8.5.

Damn. Because the script I actually want to use requires Tcl 8.6.

You definitely can’t use 8.6 to run any script that uses the macports API because of the try incompatibility. But apart from that, probably any Tcl installation that has Tclx and tcllib would more or less work.

I already have been running my script under tcl 8.6 for over a year. But I accept that I'll need to change the script to use MacPorts tclsh.

Replying to ryandesign@…:

I would love if we could find a way to fix all the Tcl scripts so that they use MacPorts tclsh8.5, but that the MacPorts prefix does not need to be hardcoded into them. For example, we already know the technique of having the script use #!/bin/sh as the first line, then re-launching the script under tclsh. Perhaps this technique could be modified to determine the MacPorts prefix programmatically (using e.g. dirname $(dirname $(which port))).

I'm attaching a proposed patch to implement this, which works for me. The executable bit should also be set. If this seems acceptable, I would like similar changes to be made to all the Tcl scripts that need it (i.e. in contrib).

I would also like to request that a tclsh symlink be created in ${prefix}/libexec/macports/bin so that later on, once we upgrade MacPorts to 8.6, we don't have to go modifying each script again to change tclsh8.5 to tclsh8.6.

[ryandesign (Ryan Carsten Schmidt)]

proposed patch

[neverpanic (Clemens Lang)]

We could extend port(1) with an argument to act as a shebang that executes the correct tclsh so we could use #!port tclsh as shebang and be done with it.

That doesn't really solve the problem of finding the right path to macports_fastload.tcl. I wonder if we even need it anymore, though. The first block of it overriding the package command is dead because of a version constraint and then all it does is preload the pkgIndex.tcl files in the MacPorts Tcl dir (which probably made sense when there used to be a few others in the Tcl package dir, but now there aren't).

The leaves the problem of getting the MacPorts prefix, but since the Tcl packages can be loaded at that point, that's not hard to come by anymore.

[cooljeanius (Eric Gallager)]

Cc Me!

[neverpanic (Clemens Lang)]

I've implemented what I wrote in comment:6 in r118559. Using #!/usr/bin/env port-tclsh should now automatically fix your problems and avoid the need to source macports_fastload.tcl.

[neverpanic (Clemens Lang)]

This change should probably also be backported into 2.3. I'm looking into the failure caused on the buildbots.

[neverpanic (Clemens Lang)]

OK, this was a little harder than I expected it to be. The relevant commits are r118562, r118563, r118564, r118565, r118566, r118567, r118568 and r118569.

I think it makes sense to backport these to 2.3 since they're related to providing a way for contrib scripts to use macports1.0 correctly.

[ryandesign (Ryan Carsten Schmidt)]

PortIndex2MySQL.tcl now begins with the line:

#!@TCLSH@

This means I cannot run it without first manually modifying it (or, presumably, running make in that directory, though that fails for me with Makefile:9: ../../Mk/macports.autoconf.mk: No such file or directory).

I would prefer to be able to run the script without having to modify it. (Well, I know I might still have to modify it for the location to the password file and to set SPAM_LOVERS.) But can't the first line be:

#!/usr/bin/env port-tclsh

as you proposed above? Making that change, the script runs for me.

Also, I see that /opt/local/bin/port-tclsh is a shell script containing:

#!/bin/sh
exec /opt/local/libexec/macports/bin/tclsh8.5 "$@"

Is there a reason why that's preferable to just being a symlink to that?

[neverpanic (Clemens Lang)]

Yes, we could usr #!/usr/bin/env port-tclsh, but that could lead to the situation where you configured a source tree for prefix $A and have prefix $B first in path and running it (from the configured source tree) still uses the installation in $B. I thought that was counter-intuitive for a script that's in our main source tree (as opposed to being in contrib/). Feel free to change that.

As for the symlink, yes, that would have been the better solution. See ​https://lists.macosforge.org/pipermail/macports-dev/2014-April/026392.html where I also noticed and suggested that.

[neverpanic (Clemens Lang)]

Oh, and btw, the step that creates PortIndex2MySQL.tcl from PortIndex2MySQL.tcl.in is configure – that also explains why the macports.autoconf.mk file cannot be found – it also it created by configure.

[neverpanic (Clemens Lang)]

Made port-tclsh a symlink in r119169. Also merged to source:branches/release_2_3/base​. My statement in comment:13 was wrong – the make step creates PortIndex2MySQL from PortIndex2MySQL.tcl and that requires a configured source tree.

As for whether the shebang should be #!@TCLSH@ or #!/usr/bin/env port-tclsh I don't have a strong opinion either way. Feel free to change it how you see fit.