Intro

The MacPorts testing framework uses tcltest [0] for its unit tests as well as regression tests. Maintainer: marius [at] macports.org

Running tests

The easiest way to run all the tests, is to use the target in the Makefile.

• make test

Each 'tests/' directory has a 'test.tcl' file, used by the make target to run all tests and format the output, making it easy to read. The file can be used also to:

• run all tests: ' tclsh test.tcl '
• get debug info: ' tclsh test.tcl -debug \[0-3\] '
• list individual test files: ' tclsh test.tcl -l '
• run specific test files: ' tclsh test.tcl -t macports.test '
• print help message: ' tclsh test.tcl -h '

Specific test cases can be run using the '-match' argument for the file that contains the test, from its parent directory.

• tclsh macports.test -match mportclose

Regression tests can be found in ' trunk/base/tests/ ' and can be run just as unit tests.

Must know

• regression tests have their own directory, found in ' trunk/base/tests/ '
• each module of MacPorts (port1.0, macports1.0, package1.0) has its own ‘tests/’ directory where the test files are located and also additional files needed (Portfile, test.tcl)
• each file in a module has a corresponding test file (.test extension) in the ‘tests/’ directory
• each proc in a file has a corresponding test case (test proc_name) in the
• each test case must be independent from each other, so they can be run individually if needed
• each test must clan all auxiliary files or directories it creates and revert all port it installs
• use a single test proceduce for each tested proc; sub-test cases should be included in the same body
• when adding new regression tests, make sure to specify its name in the test_suite list of ' trunk/base/tests/test.tcl '

Sample file

# include required tcltest package and set namespace
package require tcltest 2
namespace import tcltest::*

# get absolute path to current ‘tests/’ directory
set pwd [file normalize $argv0]
set pwd [eval file join {*}[lrange [file split $pwd] 0 end-1]]

# debug options
# ports_debug and ports_verbose are commented out as default
# need to be set before ‘mportinit’
array set ui_options {}
#set ui_options(ports_debug)   yes
#set ui_options(ports_verbose) yes
mportinit ui_options

# source/require tested/needed files
source ../../port1.0/portutil.tcl

# additional procs needed for testing
proc registry_exists {name version {revision 0} {variants ""}} {
        global macports::registry.format
        return [${macports::registry.format}::entry_exists $name $version $revision $variants]
}


# test case example
# the test name must reflect the tested proc (remove namespaces if any)
# the test description should list specific values from the tested proc on which it depends
# or the partial cases it tests
test mportclose {
    Mport close unit test.
# this branch is optional and you can use other constraints too
} -constraints {
    root
# the setup branch is optional
} -setup {
    set mport [mportopen file://.]
# please make output as useful as possible (even error cases)
# all sub-test cases should be part of the body branch
} -body {
    if {[catch {mportclose $mport}] != 0} {
        return "FAIL: cannot run mportclose"
    }
    return "Mport close successful."
# the cleanup branch is optional
} -cleanup {
    file delete -force $pwd/work
} -result "Mport close successful."


# print test results
cleanupTests

Additional files

• In all tests we use this ​ Portfile.
• This is the ​ test.tcl file used to run and parse the output of all the tests in a module.
• A worker_init copy, without using sub-interpreters ​ library.tcl .
• Example of ​ sources.conf .
• An example of a test file ​ macports.test .
• A ​ library.tcl of useful procs in regression testing.

Resources

[0] - ​ Tcltest official wiki page
[1] - ​ Getting started with tcltest
[2] - ​ Official tcltest documentation