tcltest Part 8: Recursive Test Suites

The Problem

We want to organize tests in a hierarchy of directories and execute them in all at once.

The Solution

We can organize tests in sub directories of arbitrary depths, but in order for tcltest to include a directory, that directory must contain a file named all.tcl.

In this example, we only create two sub directories, but in practice, the number of directories and the depth can be arbitrary. Let say under the tcltest_part8 directory we have two sub directories: suiteA and suiteB. Each of these directory contains an all.tcl file and a number of *.test files:

• tcltest_part8 (dir)
• all.tcl
• suiteA (dir)
• all.tcl
• a.test
• suiteB (dir)
• all.tcl
• b.test

The contents of all.tcl for each directory is the same, unless we have reasons to customize them:

package require tcltest
tcltest::configure -testdir [file dirname [file normalize [info script]]]
eval tcltest::configure $argv
tcltest::runAllTests

Once we have the directory and their files in place, we can execute all tests by issuing the following command at tcltest_part8 directory:

tclsh all.tcl

Because all.tcl processes command-line parameters, we can pass any parameter to tcltest::configure. Below are a couple of examples of command line usages:

# Skip tests in directory suiteB
tclsh all.tcl -asidefromdir suiteB

# Skips tests in certain files
tclsh all.tcl -notfile undone_*.test

Discussion

The documentation for tcltest is confusing, but if we read carefully, the following requirements must be met to setup hierarchical test directories:

1. Each directory, including the root directory, must contain a file named all.tcl. Without this file, tcltest will skip the tests in that directory.
2. If a directory does not contain all.tcl, tcltest ignores only tests within that directory, it might include tests in the sub directories. For example, if directory suiteB does not have all.tcl, tcltest will ignore tests in this directory, but might include test in suiteB’s sub directories.
3. Consequently, if a directory does not have any test, it does not require to have all.tcl.

If for some reason, tests in a directory is not running, make sure that it has all.tcl and the contents must at least include the four lines in the sample all.tcl shown above.

Conclusion

The key to executing tests in a hierarchy of directories is the presence of the file all.tcl. We can use the command line parameters to tailor the test run to include or exclude tests as discussed in earlier installments. At this point, we know quite a bit about tcltest. However, there are still many other aspects of tcltest which we have not explored. In our next installment, we will discuss some of the more useful commands.

About these ads

Posted in: Programming, Tcl