<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html><head><title>Using and writing Ghostscript testing scripts</title><!-- $Id: Testing.htm,v 1.37 2005/10/20 19:46:23 ray Exp $ --><link rel="stylesheet" type="text/css" href="gs.css" title="Ghostscript Style"></head> <body><!-- [1.0 begin visible header] ============================================ --> <!-- [1.1 begin headline] ================================================== --> <h1>Using and writing Ghostscript testing scripts</h1> <!-- [1.1 end headline] ==================================================== --> <!-- [1.2 begin table of contents] ========================================= --> <!-- [1.2 end table of contents] =========================================== --> <h2>Table of contents</h2> <blockquote><ul><li><a href="#General_overview">General overview</a><li><a href="#Running_tests">Running tests</a><ul><li><a href="#Individual_tests">Individual tests</a><li><a href="#Regression_testing">Regression testing</a></ul><li><a href="#Writing_new_tests">Writing new tests</a></ul></blockquote> <!-- [1.3 begin hint] ====================================================== --> <p>This document describes how to use the scripts located in the <ahref="../toolbin/tests">toolbin/tests</a> directory, and conventions forwriting new testing scripts. <p>For other information, see the <a href="Readme.htm">Ghostscriptoverview</a> and the documentation related to <ahref="Maintain.htm">maintaining Ghostscript</a>. <!-- [1.3 end hint] ======================================================== --> <hr> <!-- [1.0 end visible header] ============================================== --> <!-- [2.0 begin contents] ================================================== --> <h2><a name="General_overview"></a>General overview</h2> <p>The test scripts discussed here are written in Python, a language whoseimplementation is freely available from <a class="offsite"href="http://www.python.org"><tt>http://www.python.org</tt></a>.  Thescripts require Python version 2.1 or later. <h2><a name="Running_tests"></a>Running tests</h2> <p>On Unix and Linux systems, test scripts written in Python can be executeddirectly simply by typing their name into the shell, e.g., <blockquote><pre>toolbin/tests/check_source.py</pre></blockquote> <p>On other systems, it may be necessary to invoke Python explicitly, e.g., <blockquote><pre>python toolbin/tests/check_source.py</pre></blockquote> <p>The test scripts will print information about any failures that occur. <h3><a name="Individual_tests"></a>Individual tests</h3> <p>The individual test scripts are named<b><tt>toolbin/tests/check_</tt></b><em>xxx</em><b><tt>.py</tt></b> (if theydo not run Ghostscript) or<b><tt>toolbin/tests/gscheck_</tt></b><em>xxx</em><b><tt>.py</tt></b> (ifthey do run Ghostscript).  Any script with such a name can be runindividually, and is also normally run as part of regression testing(described in the next section). <p>We don't list the individual test scripts here, because any suchdocumentation would inevitably be out of date most of the time.  Each ofthese scripts contains documentation about what it tests: we suggest youread the documentation in the scripts. <h3><a name="Regression_testing"></a>Regression testing</h3> <p>We run a nightly regression test to discover any obvious problems caused bycode checked in the previous day.  Here is a list of the scripts andsupporting files that make up the regression test. <h4>Top-level scripts</h4> <dl><dl><dt><a href="../toolbin/tests/dump_testdb"><tt>dump_testdb</tt></a><dd>A debugging script that will print the contents of the database definedby gsconf.testdatadb.</dl> <dl><dt><a href="../toolbin/tests/make_testdb"><tt>make_testdb</tt></a><dd>This script creates an initial test database.  It uses gsconf.baselinegsto create raster data from the test files and computes their MD5 sums andstores them in the gsconf.testdatadb database.</dl> <dl><dt><a href="../toolbin/tests/make_two_versions"><tt>make_two_versions</tt></a><dd>A helper script to make two versions of a particular file for visualdiffing or manual analysis.  When a test fails nightly regression, this isgenerally the first investigative step.</dl> <dl><dt><a href="../toolbin/tests/make_two_pdfversions"><tt>make_two_pdfversions</tt></a><dd>Same as above, except for pdfwrite regressions.</dl> <dl><dt><a href="../toolbin/tests/testdiff"><tt>testdiff</tt></a><dd>this script provides the difference between two sets of regression results.if end date is omitted, the current date will be used.</dl> <dl><dt><a href="../toolbin/tests/revert_baseline"><tt>revert_baseline</tt></a><dd>In cases where a baseline has been accidentally updated, this scriptwill revert the database entry to the MD5 sum computed withgsconf.baselinegs.</dl> <dl><dt><a href="../toolbin/tests/revert_pdfbaseline"><tt>revert_pdfbaseline</tt></a><dd>Same as above, except for pdfwrite baselines.</dl> <dl><dt><a href="../toolbin/tests/run_nightly"><tt>run_nightly</tt></a><dd>This is the control script (usually invoked by cron) that controls thenightly test run.  It's responsible for checking the latest code out of CVS,building a new Ghostscript to compare with and launching the test suite viarun_regression.</dl> <dl><dt><a href="../toolbin/tests/run_regression"><tt>run_regression</tt></a><dd>This script runs the full gamut of regression tests using files fromgsconf.comparedir.  It differentiates files by extension and controls whattests get run and with what options.</dl> <dl><dt><a href="../toolbin/tests/update_baseline"><tt>update_baseline</tt></a><dd>This script is invoked to update the MD5 sum in the test database when anightly regression is really a progression.  Generally after noticing thatthe output from make_two_versions is acceptable or better, this script isrun to log the changes to the database.</dl> <dl><dt><a href="../toolbin/tests/update_pdfbaseline"><tt>update_pdfbaseline</tt></a><dd>Same as above, except for pdfwrite baselines.</dl> <dl><dt><a href="../toolbin/tests/update_specific"><tt>update_specific</tt></a> <tt><em>cvs-date-spec</em></tt><dd>Update a series of baselines to a specific datestamp; accepts a series of "flag filename"lines on stdin, where flag is either N or P to specify whether the normal or pdfwrite baselinesshould be updated.<dl> <dt><a href="../toolbin/tests/get_baselines"><tt>get_baselines</tt></a><dd>This script returns the change log for the test database startingat a given date.</dl></dl> <h4>Support files</h4> <dl> <dl><dt><a href="../toolbin/tests/fuzzy.c"><tt>fuzzy.c</tt></a><dd>A fuzzy comparison tool appropriate for tests where exact binary matchesaren't appropriate.</dl> <dl><dt><a href="../toolbin/tests/cmpi.py"><tt>cmpi.py</tt></a><dd>An image comparison tool for exploring the differences between tworaster files. It can switch between the two, display only the differencesand localize, highlight and jump between areas where differences occur.This can be a useful way of examining regression difference to decideif they are progressions or regressions. Requires python imaging andpython Tkinter.</dl> <dl><dt><a href="../toolbin/tests/gsconf.py"><tt>gsconf.py</tt></a><dd>This is the configuration framework for the scripts above.  It readsconfiguration files and makes available those configuration options tothe rest of the testing framework.</dl> <dl><dt><a href="../toolbin/tests/testing.cfg.example"><tt>testing.cfg.example</tt></a><dd>This is an example configuration file for the scripts above.  It controlswhere files are found, where Ghostscript executables are and the location ofthe test database.  Most test configuration will be in this file.  It mustbe copied to testing.cfg in order for the tests to find it.</dl> <dl><dt><a href="../toolbin/tests/testing.cfg"><tt>testing.cfg</tt></a><dd>This is the configuration file for the scripts above.  It controlswhere files are found, where Ghostscript executables are and the location ofthe test database.  Most test configuration will be in this file. The filetesting.cfg.example above can be used as a template.</dl> <dl><dt><a href="../toolbin/tests/gstestgs.py"><tt>gstestgs.py</tt></a><dd>This provides classes for running tests that actually executeGhostcript.</dl> <dl><dt><a href="../toolbin/tests/gsparamsets.py"><tt>gsparamsets.py</tt></a><dd>What parameters tests get run with by default are stored in this file.Between this configuration information and the information in gsconf.py allconfigurable testing parameters should be covered.</dl> <dl><dt><a href="../toolbin/tests/gsutil.py"><tt>gsutil.py</tt></a><dd>Various utility routines used by the regression test scripts.</dl> <dl><dt><a href="../toolbin/tests/gssum.py"><tt>gssum.py</tt></a><dd>Helper functions that compute, compare and store MD5 sums.</dl> <dl><dt><a href="../toolbin/tests/rasterdb.py"><tt>rasterdb.py</tt></a><dd>Hepler functions for accessing the raster database.</dl> </dl> <h2><a name="Writing_new_tests"></a>Writing new tests</h2> <p>Some of Ghostscript's test scripts follow a set of conventions that allowthem to be run either stand-alone or as part of a suite; in particular, theycan be run as part of the nightly regression test suite.  In this section,we provide pointers to documentation on how to write new tests that followthese conventions, since that will make them the most useful. <p>The test scripts are based on Python's <b><tt>unittest</tt></b> module.  Wesuggest that if you are not familiar with this module, you read thedocumentation, which is available at <a class="offsite"href="http://www.python.org/doc/current/lib/module-unittest.html">http://www.python.org/doc/current/lib/module-unittest.html</a>. <p>Ghostscript specializes the <b><tt>unittest</tt></b> module by definingsubclasses, which all individual tests use in place of those in<b><tt>unittest</tt></b>.  These subclasses are defined in <ahref="../toolbin/tests/gstestutils.py">toolbin/tests/gstestutils.py</a>. <p>Since code documentation separate from the code itself is always out ofdate, we have decided to maintain the primary documentation for writing newtests in <a href="../toolbin/tests/gstestutils.py">gstestutils.py</a> ratherthan here in a separate document.  Please read that file for moreinformation. <!-- [2.0 end contents] ==================================================== --> <!-- [3.0 begin visible trailer] =========================================== --><hr> <p><small>Copyright &copy; 2002 artofcode LLC.  All rights reserved.</small> <p><small>This file is part of AFPL Ghostscript.  See the <ahref="Public.htm">Aladdin Free Public License</a> (the "License") for fulldetails of the terms of using, copying, modifying, and redistributing AFPLGhostscript.</small> <p><small>Ghostscript version 8.53, 20 October 2005 <!-- [3.0 end visible trailer] ============================================= --> </body></html>