GNU Go Documentation
21. Regression testing
The standard purpose of regression testing is to avoid getting the same
bug twice. When a bug is found, the programmer fixes the bug and adds a
test to the test suite. The test should fail before the fix and pass
after the fix. When a new version is about to be released, all the tests
in the regression test suite are run and if an old bug reappears, this
will be seen quickly since the appropriate test will fail.
The regression testing in GNU Go is slightly different. A typical test
case involves specifying a position and asking the engine what move it
would make. This is compared to one or more correct moves to decide
whether the test case passes or fails. It is also stored whether a test
case is expected to pass or fail, and deviations in this status signify
whether a change has solved some problem and/or broken something
else. Thus the regression tests both include positions highlighting some
mistake being done by the engine, which are waiting to be fixed, and
positions where the engine does the right thing, where we want to detect
if a change breaks something.
