www.delorie.com/gnu/docs/greg/greg_13.html   search  
 
Buy GNU books!


Greg testing framework

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.1 Output

While Greg may produce more verbose output in response to various settings, the basic output of a test run is a series of lines describing the success/failure state of each testcase encountered, followed by a summary of all test cases.

In `normal' mode, only unexpected results are displayed, but in `verbose' output mode, results for all results are displayed.

`greg' flags the outcome of each test case with a line consisting of one of the following codes followed by a colon, a space, and then the testcase description.

PASS
The most desirable outcome: the test succeeded, and was expected to succeed.

UPASS
A pleasant kind of failure: a test was expected to fail, but succeeded. This may indicate progress; inspect the test case to determine whether you should amend it to stop expecting failure.

FAIL
A test failed, although it was expected to succeed. This may indicate regress; inspect the test case and the failing software to locate the bug.

XFAIL
A test failed, but it was expected to fail. This result indicates no change in a known bug. If a test fails because the operating system where the test runs lacks some facility required by the test, the outcome is UNSUPPORTED instead.

UNRESOLVED
Output from a test requires manual inspection; the test suite could not automatically determine the outcome. For example, your tests can report this outcome is when a test does not complete as expected.

UNTESTED
A test case is not yet complete, and in particular cannot yet produce a PASS or FAIL. You can also use this outcome in dummy "tests" that note explicitly the absence of a real test case for a particular property.

UNSUPPORTED
A test depends on a conditionally available feature that does not exist (in the configured testing environment). For example, you can use this outcome to report on a test case that does not work on a particular target because its operating system support does not include a required subroutine.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

  webmaster     delorie software   privacy  
  Copyright 2003   by The Free Software Foundation     Updated Jun 2003