Tips and Tricks for committers

This page provides some useful hints how to work with our infrastructure. And they can also make your work a lot easier.

Contents

1. Set svn properties automatically on new Portfiles
2. Create an experimental users directory in the MacPorts Subversion repository
3. Apply patches directly from Trac URL
4. Do Explorative Programming in tclsh with Readline Support
5. Debugging Tcl scripts
6. Portfile syntax highlighting
7. Checksum tips
8. Testing Port Phases
9. Searching for ports of a specific maintainer

Set svn properties automatically on new Portfiles

In the configuration for your Subversion client, enable automatic property setting and, for all files named Portfile, setting "svn:eol-style" to "native" and "svn:keywords" to "Id". If you are not using Subversion's own svn command-line client, see its documentation. For svn, you can make the appropriate changes by editing ~/.subversion/config as follows:

...

[miscellany]
enable-auto-props = yes

...

[auto-props]
Portfile = svn:eol-style=native;svn:keywords=Id

Create an experimental users directory in the MacPorts Subversion repository

Use the Trac Browser to explore the MacPorts Subversion repository. The repository root is located at https://svn.macosforge.org/repository/macports.

See also the explanation of ​local development port trees in the guide.

To create a sandbox for experimental development, create a users directory:

svn mkdir https://svn.macosforge.org/repository/macports/users/<YourMacPortsUserName>

Then, checkout the user directory (suggested location is ~/myports):

$ svn co https://svn.macosforge.org/repository/macports/users/<YourMacPortsUserName> ~/myports

Then edit the MacPorts sources.conf file /opt/local/etc/macports/sources.conf to add ~/myports to the list before the main rsync source at rsync.macports.org, e.g.:

file:///Users/<YourSystemUserName>/myports [nosync]

To work on a copy of a port from the MacPorts trunk, use svn copy. For example, to test changes on the cableswig port, copy the repository trunk (at the HEAD revision) into the user branch:

$ svn mkdir ~/myports/devel
$ svn copy https://svn.macosforge.org/repository/macports/trunk/dports/devel/cableswig ~/myports/devel/

Another option for a large branch is to copy everything at the server side, where the copy is equivalent to a unix hard link in the file system; e.g.:

$ svn mkdir https://svn.macosforge.org/repository/macports/users/<YourMacPortsUserName>/devel
$ svn copy \
      https://svn.macosforge.org/repository/macports/trunk/dports/devel/cableswig \
      https://svn.macosforge.org/repository/macports/users/<YourMacPortsUserName>/devel/cableswig \
      -m "experimental modifications to cableswig"
$ cd ~/myports
$ svn update

After the copy, run portindex in ~/myports (do this anytime ports are added or removed from ~/myports), e.g.:

$ cd ~/myports
$ svn update
$ portindex
$ port file cableswig

Now make experimental changes to the port in the sandbox. Whenever a signficant change is made, note the change with a commit to the sandbox and when all the changes are complete commit the final change to the sandbox with a meaningful commit message.

The main trunk Portfile and the sandbox Portfile are now out of sync. Someone may make changes to the main trunk Portfile while changes are made to the sandbox Portfile. Let's assume that all the experimental changes are successful, so the Portfile needs to be integrated back into the main trunk, where other changes may have occurred. This merge process requires a few steps. Let's assume there are checkouts for the sandbox in ~/myports and the trunk in ~/macports.

First update the sandbox and the trunk checkouts, e.g.:

cd ~/myports
svn update
cd ~/macports
svn update

Note that the revision numbers should be the same (because the sandbox and the main trunk belong to the same svn repository).

Next, find out where the experimental changes left off from the trunk and create a bash variable (any shell variable) to store the value of the revision number where the branch copy occurred (the revision number will be the last one in the log):

cd ~/myports/devel/cableswig
svn log --stop-on-copy Portfile
EXP_BRANCH_REV=54362

For the curious, leave off the --stop-on-copy to see the entire commit log history (it should all be there if there is a continuous and reliable use of svn to manage the file from it's "birth" in the svn repository).

Now take a look at the commit log for the main trunk, to get some hints about what has changed to the file since the experimental branch was forked off.

cd ~/macports/dports/devel/cableswig
svn log Portfile | more

Look through the log to find the revision number where the experimental copy took place. If that is the first revision number in the log, there have been no changes in the trunk. Otherwise, changes have occurred in the trunk that are not in the experimental branch. There could be trunk changes that complement the experimental branch, or they may be in conflict. Try to merge all these changes using the revision number of the experimental copy:

cd ~/macports/dports/devel/cableswig
svn merge  -r ${EXP_BRANCH_REV}:HEAD  https://svn.macosforge.org/repository/macports/users/<YourMacPortsUserName>/devel/cableswig/Portfile Portfile

Now resolve any conflicts, if any, in the trunk Portfile. All work continues in the trunk (until the next time a sandbox is required). Once the merge looks good, commit all the changes to the trunk with a meaningful commit message,

cd ~/macports/dports/devel/cableswig
svn commit "merged sandbox changes from ${EXP_BRANCH_REV}:HEAD back into trunk" Portfile

Finally, to have the port system revert to using the rsync distribution, it's possible to remove the sandbox port. No information is lost from the svn system, so removing the sandbox port is really harmless (in any case, all the changes are now back in the trunk). For example,

sudo port clean --all cableswig
cd ~/myports/devel
svn remove cableswig
svn commit -m "done with sandbox for cableswig"
svn update
cd ~/myports
portindex

For more information about branches and merging, see ​http://svnbook.red-bean.com/en/1.5/svn.branchmerge.html

Apply patches directly from Trac URL

Installation

Add the following functions to your .bashrc in order to apply patches directly from Trac.

function trac-get {
    local url=$1
    local dir=$2

    if [ -z $dir ]; then
        dir=.
    fi  

    curl "$url?format=raw" --create-dirs -o $dir/$(basename $1)
}

function trac-patch {
    local cmd=""
    while [[ $1 == -* ]]; do
        if [ "$1" == "--" ]; then
            break
        fi

        cmd="$cmd $1"
        shift
    done

    if [ -z $cmd ]; then
        cmd="-p0"
    fi  

    trac-get $1
    patch $cmd < $(basename $1)
}

Usage

Use like this:

1. Copy the URL to the patch from the Trac ticket page
2. Switch to the ports directory

   $ cd $(port dir foo)
   

3. Apply the patch

   $ trac-patch https://trac.macports.org/attachment/ticket/.../Portfile.diff
   

You can also add options to trac-patch which will get passed through to the patch tool. This is especially useful when the patch needs another prefix level.

$ trac-patch -p1 https://trac.macports.org/attachment/ticket/.../Portfile.diff

If you don't add any option, -p0 is used as a default. This should be the most common case.

trac-get can also download to another directory. If the directory does not yet exist, it will be created. Just add a second parameter with the name of the directory. If you omit the second parameter, the current directory is used.

$ trac-get https://trac.macports.org/attachment/ticket/.../Portfile new-port

Do Explorative Programming in tclsh with Readline Support

tclsh does not offer readline support by itself, which is quite annoying. When writing portfiles or tinkering with changes to Macports base, I need to experiment in a Tcl shell all the time to tests small things. But for that, command history and Emacs-like navigation within the line are essential. Unfortunately tclsh does not offer this.

You can also use Emacs’ tcl-mode: from within a tcl buffer, the tcl shell is just a C-t away. More info in ​tcler’s wiki.

Solution: use port rlwrap together with tclsh. When you invoke tclsh via rlwrap you get all the convenience you know from bash.

rlwrap tclsh

Or, put even an alias into ~/.bashrc

alias tclsh='rlwrap tclsh'

and don't think about it ever again.

If you want to test MacPorts Tcl extensions, you need to require the appropriate packages. For access to the 'strsed', 'reinplace', and other macport commands, put the following into ~/bin/macports_testing.tcl:

package require macports
set portarchivemode no
mportinit
#package require port
package require Pextlib

[Note, Oct 2009] There may be a bug in the port 1.0 package, see ​http://lists.macosforge.org/pipermail/macports-dev/2009-September/010132.html

Then source the file in the Apple-supplied tclsh, i.e.:

$ rlwrap /usr/bin/tclsh
% source ~/bin/macports_testing.tcl
/opt/local
% strsed "foo" "s/f/m/"
moo
% 

Check self contained script too.

Debugging Tcl scripts

Tcl scripts can be debugged with the expect tool, which provides a debugging interface similar to gdb for those who are familiar with that. For a command overview, just type h at the prompt.

Run with:

$ expect -D1 <file> <args>

For example:

$ expect -D1 foo.tcl -bar
1: proc main {args} {
...
dbg1.0>

Portfile syntax highlighting

• BBEdit, TextWrangler
• ​TextMate
• ​vim

Checksum tips

Checksums for port updates

If you're updating a port's version and simply need the checksums for the new version, run the following while in the port's directory:

$ port -v checksum
...
The correct checksum line may be:
checksums           md5     1ca0cf40350913fa620f73cbd906378a \
                    sha1    2921531ece0eacb303b0c9a4978d0d050dcfd5d2 \
                    rmd160  6cacbcf37581b5e19fe9fe674dd7b0722f1a19de

You can then copy/paste the result into the Portfile. You should probably then manually tweak the whitespace of these lines to match the existing whitespace of the portfile to keep things consistent.

A bash script for checksums

#!/bin/bash
if [ -f "$1" ]; then
    basename $1 | sed -e "s/\(.*\)/\1 \\\/"
    md5 $1 | sed -e "s/^MD5.*=/md5/" | sed -e "s/\(.*\)/\1 \\\/"
    openssl sha1 $1 | sed -e "s/^SHA1.*=/sha1/" | sed -e "s/\(.*\)/\1 \\\/"
    openssl rmd160 $1 | sed -e "s/^R.*=/rmd160/"
fi

For example, assume this is in ~/bin/macports_checksum.bash, then we get:

$ macports_checksum.bash ~/Downloads/libpqxx-3.0.1.tar.gz 
libpqxx-3.0.1.tar.gz \
md5 23557f306821bf4cae39cca45acdf9e1 \
sha1 a37874511946ba340d5df2d92252177f9eb906f6 \
rmd160 1f842ea95ad6dd2cba2cdc2d2bd8e0be5063fb9b

Testing Port Phases

Individual port phases (extract, patch, configure, build, etc.) can be run by using the appropriate keyword, e.g.:

port build python26

All phases that have not already been performed up to the designated phase will be run. If you want to rerun a port command that completed successfully, you can edit ./work/.macports.[port name].state and remove the lines up to the completed stage you want to keep.

By default, if the portfile is changed, you will get the message:

Portfile changed since last build; discarding previous state.

and the build directory will be removed (cleaned) before proceeding. When debugging a portfile, this can be avoided by using the '-o' flag (from the manpage):

-o       honor state files older than Portfile

Searching for ports of a specific maintainer

To find ports maintained by a particular maintainer, you can use the "maintainer:" pseudoport. Maintainers can list their email address in a port in ​several different ways, and you must search for all of them to be sure you find all relevant ports. For example, to print the name and maintainer(s) of all ports maintained by julesverne:

HANDLE=julesverne
port info --name --maintainer '(' \
"maintainer:(\s|^)$HANDLE(\s|$)" or \
"maintainer:(\s|^)$HANDLE@macports.org(\s|$)" or \
"maintainer:(\s|^)macports.org:$HANDLE(\s|$)" ')'

which will now find only those ports which HANDLE@… maintains.