EXERCISE(1) General Commands Manual EXERCISE(1)
NAME
exercise - simple but flexible test runner
SYNOPSIS
exercise [-q] [-p k=pattern] [path ...]
DESCRIPTION
The exercise utility searches for and executes software (unit) tests in
or below the current directory. It displays a running status and passes
any error messages to standard error.
If given a directory argument, exercise searches for tests therein. If
given a file argument, it executes it as a single test (or test suite).
To be recognized as tests, files must match the pattern *.t. To be run
as test suites (see below), files must match the pattern *.tests.
The options are as follows:
-p d=pattern
When searching for tests, only descend into directories that
match the given pattern. Defaults to `*' (all directories).
-p s=pattern
Process (only) files matching the given pattern as test suites.
Defaults to `*.tests'.
-p t=pattern
Process (only) files matching the given pattern as tests.
Defaults to `*.t'.
-q Do not print status counters to standard output.
Test Files
Tests are represented simply as executable files that are expected to
exit 0 on success. Non-zero exits as well as any output to standard
error are interpreted as negative results (failure). Standard output is
ignored.
As a special case, exit status 127 is interpreted as an error that
precluded the proper execution of the test, implying an inconclusive
result.
Test Suites
Multiple tests can be combined into one executable. A test suite should
pass error messages from individual tests unchanged to its standard
error. Its exit status should indicate whether all tests passed.
A test suite should only exit 127 if there was a fatal issue in executing
the suite itself. Errors among the individual tests should be reported
through the status counters as detailed below.
The standard output from a test suite executable should consist of lines
of the form `total o run n ok m fail l error k', representing running
counts as printed by exercise itself. Consequently, the exercise utility
can be used as a (nested) test suite runner if desired.
After running a test suite, its final report is checked for consistency.
If any of the conditions (listed below) are not satisfied, the entire
test suite is discarded and counted as a single error, as if it had
exited with status 127.
Status Counters
Unless given the -q option, exercise prints the following running
counters to its standard output:
total The number of tests that are planned to execute. This is first
set to the number of test files found, including test suites.
The latter are initially counted as single tests because the
number of tests they contain is not known beforehand. The total
counter will increase at run time as test suites report their
totals.
run The number of tests that have been started so far. This is
normally the sum of ok, fail, and error plus any tests that are
currently in progress. Test suites may increment this counter
before or after a test finishes, so it may or may not include all
tests in progress.
ok The number of tests that have passed successfully, i.e. exited 0
without any output to standard error.
fail The number of tests that have yielded a negative result, i.e.
exited with a non-zero status (except 127) or printed anything to
standard error.
error The number of tests that could not be executed despite the
attempt or that exited with the special status 127.
The following conditions should hold at the end of a run:
1. run <= total
2. run = ok + fail + error
The second condition is relaxed to an inequality at run time to account
for tests in progress, as explained above. The run count is permitted to
ultimately remain short of total as the result of test suites or exercise
terminating early.
DEPENDENCIES
The exercise utility requires Tcl 8.4 or above.
EXIT STATUS
The exercise utility exits 0 if all tests passed, 1 if some tests failed
or encountered errors, and 127 if an error occured outside of the tests.
EXAMPLES
See the tests/ directory distributed with the program. It shows a useful
idiom where a number of test (shell) scripts make use of a helper,
assert-exercise. The helper runs exercise with some test (shell) script
to execute and compares the outcome to expectations.
SEE ALSO
sh(1)
AUTHORS
Sven M. Hallberg <pesco@khjk.org>
May 25, 2025
