wiki:Tests

Version 9 (modified by marius@…, 11 years ago) (diff)

more improvements

Intro

The MacPorts testing framework uses ​ tcltest for its unit tests as well as regression tests. Maintainer: marius [at] macports.org

Running tests

The easiest way to run all the tests, is to use the target in the Makefile.

  • make test

Each 'tests/' directory has a ​ test.tcl file, used by the make target to run all tests and format the output, making it easy to read. The file can be used also to:

  • run all tests: ' tclsh test.tcl '
  • get debug info: ' tclsh test.tcl -debug \[0-3\] '
  • list individual test files: ' tclsh test.tcl -l '
  • run specific test files: ' tclsh test.tcl -t macports.test '
  • print help message: ' tclsh test.tcl -h '

Specific test cases can be run using the '-match' argument for the file that contains the test, from its parent directory.

  • tclsh macports.test -match mportclose

Regression tests can be found in ' trunk/base/tests/ ' and can be run just as unit tests.

Must know

  • regression tests have their own directory, found in ' trunk/base/tests/ '
  • each module of MacPorts (port1.0, macports1.0, package1.0) has its own ‘tests/’ directory where the test files are located and also additional files needed (​ Portfile, ​ test.tcl)
  • each file in a module has a corresponding test file (.test extension) in the ‘tests/’ directory
  • each proc in a file has a corresponding test case (test proc_name) in the
  • each test case must be independent from each other, so they can be run individually if needed
  • each test must clan all auxiliary files or directories it creates and revert all port it installs
  • use a single test proceduce for each tested proc; sub-test cases should be included in the same body
  • when adding new regression tests, make sure to specify its name in the test_suite list of ' trunk/base/tests/test.tcl '

Sample file

# include required tcltest package and set namespace
package require tcltest 2
namespace import tcltest::*

# get absolute path to current ‘tests/’ directory
set pwd [file normalize $argv0]
set pwd [eval file join {*}[lrange [file split $pwd] 0 end-1]]

# debug options
# ports_debug and ports_verbose are commented out as default
# need to be set before ‘mportinit’
array set ui_options {}
#set ui_options(ports_debug)   yes
#set ui_options(ports_verbose) yes
mportinit ui_options

# source/require tested/needed files
# source ../../port1.0/portutil.tcl
package require portutil 1.0

# use custom macports.conf and sources.conf
# you need to provide the sources.conf (see additional files) file
makeDirectory $pwd/tmpdir
makeDirectory $pwd/tmpdir/share
makeDirectory $pwd/tmpdir/var/macports/registry
set fd [open $pwd/tmpdir/macports.conf w+]
puts $fd "portdbpath $pwd/tmpdir/var/macports"
puts $fd "prefix $pwd/tmpdir"
puts $fd "variants_conf $pwd/tmpdir/variants.conf"
puts $fd "sources_conf $pwd/sources.conf"
puts $fd "applications_dir $pwd/tmpdir/Applications"
puts $fd "frameworks_dir $pwd/tmpdir/Library/Frameworks"
close $fd
set env(PORTSRC) $pwd/tmpdir/macports.conf
file link -symbolic $pwd/tmpdir/share/macports $macports::autoconf::prefix/share/macports
close [open $pwd/tmpdir/variants.conf w+]

# if you need to use procs from macports namespace, that are just aliases, you can
# always source library.tcl (see additional files) which provides a copy macports::worker_init
# without sub-interpreters; it also sets some important environment variables like
# os.platform, os.major, os.arch, workpath, destpath, portpath
# some other option would be to get the $workername from a $mport and use it directly

# additional procs needed for testing go before the actual test cases


# test case example
# the test name must reflect the tested proc (remove namespaces if any)
# the test description should list specific values from the tested proc on which it depends
# or the partial cases it tests
test mportclose {
    Mport close unit test.
# this branch is optional and you can use other constraints too
} -constraints {
    root
# the setup branch is optional
} -setup {
    set mport [mportopen file://.]
# please make output as useful as possible (even error cases)
# all sub-test cases should be part of the body branch
} -body {
    if {[catch {mportclose $mport}] != 0} {
        return "FAIL: cannot run mportclose"
    }
    return "Mport close successful."
# the cleanup branch is optional
} -cleanup {
    file delete -force $pwd/work
} -result "Mport close successful."


# print test results
cleanupTests

Additional files

Resources