Tree


Makefilecommits | blame
READMEcommits | blame
exercise*commits | blame
exercise.1commits | blame
lang/
tests/

README

EXERCISE(1)                 General Commands Manual                EXERCISE(1)

NAME
     exercise - simple but flexible test runner

SYNOPSIS
     exercise [-q] [-p k=pattern] [path ...]

DESCRIPTION
     The exercise utility searches for and executes software (unit) tests in
     or below the current directory.  It displays a running status and passes
     any error messages to standard error.

     If given a directory argument, exercise searches for tests therein.  If
     given a file argument, it executes it as a single test (or test suite).
     To be recognized as tests, files must match the pattern *.t.  To be run
     as test suites (see below), files must match the pattern *.tests.

     The options are as follows:

     -p d=pattern
             When searching for tests, only descend into directories that
             match the given pattern.  Defaults to `*' (all directories).

     -p s=pattern
             Process (only) files matching the given pattern as test suites.
             Defaults to `*.tests'.

     -p t=pattern
             Process (only) files matching the given pattern as tests.
             Defaults to `*.t'.

     -q      Do not print status counters to standard output.

   Test Files
     Tests are represented simply as executable files that are expected to
     exit 0 on success.  Non-zero exits as well as any output to standard
     error are interpreted as negative results (failure).  Standard output is
     ignored.

     As a special case, exit status 127 is interpreted as an error that
     precluded the proper execution of the test, implying an inconclusive
     result.

   Test Suites
     Multiple tests can be combined into one executable.  A test suite should
     pass error messages from individual tests unchanged to its standard
     error.  Its exit status should indicate whether all tests passed.

     A test suite should only exit 127 if there was a fatal issue in executing
     the suite itself.  Errors among the individual tests should be reported
     through the status counters as detailed below.

     The standard output from a test suite executable should consist of lines
     of the form `total o run n ok m fail l error k', representing running
     counts as printed by exercise itself.  Consequently, the exercise utility
     can be used as a (nested) test suite runner if desired.

     After running a test suite, its final report is checked for consistency.
     If any of the conditions (listed below) are not satisfied, the entire
     test suite is discarded and counted as a single error, as if it had
     exited with status 127.

   Status Counters
     Unless given the -q option, exercise prints the following running
     counters to its standard output:

     total   The number of tests that are planned to execute.  This is first
             set to the number of test files found, including test suites.
             The latter are initially counted as single tests because the
             number of tests they contain is not known beforehand.  The total
             counter will increase at run time as test suites report their
             totals.

     run     The number of tests that have been started so far.  This is
             normally the sum of ok, fail, and error plus any tests that are
             currently in progress.  Test suites may increment this counter
             before or after a test finishes, so it may or may not include all
             tests in progress.

     ok      The number of tests that have passed successfully, i.e. exited 0
             without any output to standard error.

     fail    The number of tests that have yielded a negative result, i.e.
             exited with a non-zero status (except 127) or printed anything to
             standard error.

     error   The number of tests that could not be executed despite the
             attempt or that exited with the special status 127.

     The following conditions should hold at the end of a run:

           1.   run <= total

           2.   run = ok + fail + error

     The second condition is relaxed to an inequality at run time to account
     for tests in progress, as explained above.  The run count is permitted to
     ultimately remain short of total as the result of test suites or exercise
     terminating early.

DEPENDENCIES
     The exercise utility requires Tcl 8.4 or above.

EXIT STATUS
     The exercise utility exits 0 if all tests passed, 1 if some tests failed
     or encountered errors, and 127 if an error occured outside of the tests.

EXAMPLES
     See the tests/ directory distributed with the program.  It shows a useful
     idiom where a number of test (shell) scripts make use of a helper,
     assert-exercise.  The helper runs exercise with some test (shell) script
     to execute and compares the outcome to expectations.

SEE ALSO
     sh(1)

AUTHORS
     Sven M. Hallberg <pesco@khjk.org>

                                 May 25, 2025