commit 42d07c50c84c3496b4c9b9f4c44c6339df2e6932
from: Sven M. Hallberg <pesco@khjk.org>
date: Sun May 25 14:46:28 2025 UTC

add counters section to manpage

commit - 9c8a0916ad7697e4a5e36371593f0c2ea4840efb
commit + 42d07c50c84c3496b4c9b9f4c44c6339df2e6932
blob - 9e7aa74f656e35decd920eeadd0d8fe6e441ee25
blob + 781c6b54fb49de865596d2140475fd85c3785aad
--- exercise.1
+++ exercise.1
@@ -73,14 +73,83 @@ status counters as detailed below.
 .Pp
 The standard output from a
 test suite executable should consist of lines of the form
-.Sq run Va n No ok Va m No fail Va l No error Va k No ,
+.Sq total Va o No run Va n No ok Va m No fail Va l No error Va k ,
 representing running counts as printed by
 .Nm
 itself.
 Consequently, the
 .Nm
 utility can be used as a (nested) test suite runner if desired.
+.Pp 
+After running a test suite, its final report is checked for consistency.
+If any of the conditions (listed below) are not satisfied,
+the entire test suite is discarded and counted as a single
+.Sy error ,
+as if it had exited with status 127.
 .
+.Ss Status Counters
+Unless given the
+.Fl q
+option,
+.Nm
+prints the following running counters to its standard output:
+.Bl -tag
+.It Sy total
+The number of tests that are planned to execute.
+This is first set to the number of test files found,
+including test suites.
+The latter are initially counted as single tests
+because the number of tests they contain is not known beforehand.
+The
+.Sy total
+counter will increase at run time as test suites report their totals.
+.It Sy run
+The number of tests that have been started so far.
+This is normally the sum of
+.Sy ok , fail ,
+and
+.Sy error
+plus any tests that are currently in progress.
+Test suites may increment this counter before or after a test finishes,
+so it may or may not include all tests in progress.
+.It Sy ok
+The number of tests that have passed successfully,
+i.e. exited 0 without any output to standard error.
+.It Sy fail
+The number of tests that have yielded a negative result,
+i.e. exited with a non-zero status (except 127)
+or printed anything to standard error.
+.It Sy error
+The number of tests that could not be executed despite the attempt
+or that exited with the special status 127.
+.El
+.Pp
+The following conditions should hold at the end of a run:
+.Bl -enum -offset Ds
+.It
+.Sy run
+\(<=
+.Sy total
+.It
+.Sy run
+=
+.Sy ok
++
+.Sy fail
++
+.Sy error
+.El
+.Pp
+The second condition is relaxed to an inequality at run time to account
+for tests in progress, as explained above.
+The
+.Sy run
+count is permitted to ultimately remain short of
+.Sy total
+as the result of test suites or
+.Nm
+terminating early.
+.
 .Sh DEPENDENCIES
 The
 .Nm