tcltest Part 8: Recursive Test Suites

The Problem

We want to organize tests in a hierarchy of directories and execute them in all at once.

The Solution

We can organize tests in sub directories of arbitrary depths, but in order for tcltest to include a directory, that directory must contain a file named all.tcl.

In this example, we only create two sub directories, but in practice, the number of directories and the depth can be arbitrary. Let say under the tcltest_part8 directory we have two sub directories: suiteA and suiteB. Each of these directory contains an all.tcl file and a number of *.test files:

  • tcltest_part8 (dir)
    • all.tcl
    • suiteA (dir)
      • all.tcl
      • a.test
    • suiteB (dir)
      • all.tcl
      • b.test

The contents of all.tcl for each directory is the same, unless we have reasons to customize them:

package require tcltest
tcltest::configure -testdir [file dirname [file normalize [info script]]]
eval tcltest::configure $argv
tcltest::runAllTests

Once we have the directory and their files in place, we can execute all tests by issuing the following command at tcltest_part8 directory:

tclsh all.tcl

Because all.tcl processes command-line parameters, we can pass any parameter to tcltest::configure. Below are a couple of examples of command line usages:

# Skip tests in directory suiteB
tclsh all.tcl -asidefromdir suiteB

# Skips tests in certain files
tclsh all.tcl -notfile undone_*.test

Discussion

The documentation for tcltest is confusing, but if we read carefully, the following requirements must be met to setup hierarchical test directories:

  1. Each directory, including the root directory, must contain a file named all.tcl. Without this file, tcltest will skip the tests in that directory.
  2. If a directory does not contain all.tcl, tcltest ignores only tests within that directory, it might include tests in the sub directories. For example, if directory suiteB does not have all.tcl, tcltest will ignore tests in this directory, but might include test in suiteB’s sub directories.
  3. Consequently, if a directory does not have any test, it does not require to have all.tcl.

If for some reason, tests in a directory is not running, make sure that it has all.tcl and the contents must at least include the four lines in the sample all.tcl shown above.

Conclusion

The key to executing tests in a hierarchy of directories is the presence of the file all.tcl. We can use the command line parameters to tailor the test run to include or exclude tests as discussed in earlier installments. At this point, we know quite a bit about tcltest. However, there are still many other aspects of tcltest which we have not explored. In our next installment, we will discuss some of the more useful commands.

One thought on “tcltest Part 8: Recursive Test Suites

  1. chris

    Hi Hai,

    Great posting. Our group (Cisco, datacenter automation testing) has an opening for automation test manager at San Jose, CA. If you are interested, please let me know and I can forward your resume to the hiring manager:

    Required Skills/Experience:
    * BS CS/EE degree with a total of 10+ years engineering experience including a minimum of 2 years in first level management role
    * Demonstrated experience building out and managing a high performance development/QA team, including: hiring, setting up development environment / testing facilities, software configuration management, release management, and project management
    * Experience developing large-scale, complex automation frameworks
    * Proficient programming in at least one scripting language (TCL, Perl, Python, etc.)

    Highly Desired Skills/Experience:
    * Experience in managing global teams overseas
    * Experience with a full spectrum of languages, from machine-level (some assembly) to OO-based languages (Python/Java/C++), and everything in between (C/shell/Perl)
    * Strong background and experience in software testing methodology, test
    development and test execution
    * Strong hands-on experience in one or more data center areas: Server, Networking, and Storage
    * Host/Server side product experience
    * Practical skills in L2/L3 networking (testing or administration)
    * Understanding of data center operation

    Regards
    Chris

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s