www.delorie.com/gnu/docs/autoconf/autoconf_167.html   search  
Buy the book!


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

16. Generating Test Suites with Autotest

Note: This section describes an experimental feature which will
be part of Autoconf in a forthcoming release.  Although we believe
Autotest is stabilizing, this documentation describes an interface which
might change in the future: do not depend upon Autotest without
subscribing to the Autoconf mailing lists.

It is paradoxical that portable projects depend on nonportable tools to run their test suite. Autoconf by itself is the paragon of this problem: although it aims at perfectly portability, up to 2.13, its test suite was using DejaGNU, a rich and complex testing framework, but which is far from being standard on Unix systems. Worse yet, it was likely to be missing on the most fragile platforms, the very platforms that are most likely to torture Autoconf and exhibit deficiencies.

To circumvent this problem many package maintainers have developed their own testing framework, based on simple shell scripts whose sole output are their exit status: the test succeeded, or failed. In addition, most of these tests share some common patterns, what results in lots of duplicated code, tedious maintenance etc.

Following exactly the same reasoning that yielded to the inception of Autoconf, Autotest provides a test suite generation frame work, based on M4 macros, building a portable shell script. The suite itself is equipped with automatic logging and tracing facilities which greatly diminish the interaction with bug reporters, and simple timing reports.

Autoconf itself has been using Autotest for years, and we do attest that it has considerably improved the strength of the test suite, and the quality of bug reports. Other projects are known to use some generation of Autotest, such as Bison, Free Recode, Free Wdiff, GNU Tar, each of them having different needs, what slowly polishes Autotest as a general testing framework.

Nonetheless, compared to DejaGNU, Autotest is inadequate for interactive tool testing, which is probably its main limitation.

16.1 Using an Autotest Test Suite  Autotest and the user
16.2 Writing `testsuite.at'  Autotest macros
16.3 Running testsuite Scripts  Running testsuite scripts
16.4 Making testsuite Scripts  Using autom4te to create testsuite

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

  webmaster   donations   bookstore     delorie software   privacy  
  Copyright 2003   by The Free Software Foundation     Updated Jun 2003