# Tests

## Contents

1. A Brief Guide to Running Tests
2. A Brief Guide to Writing Tests
3. Quartz Backend Tests

## A Brief Guide to Running Tests

After a successful "make", try "make check". If you set the environment variable OM_TEST_BACKEND, apitest will only run with that backend.

Individual test programs optionally take one or more test names as arguments (if the number of the test is omitted, all tests with that basename are run, so "./apitest deldoc" runs deldoc1, deldoc2, ...). You can also pass "-v" to a test to get more verbose output from some tests.

By default, the testsuite catches signals and handles them gracefully - the current test is failed, and the testsuite moves onto the next test. If you want to suppress this (some debugging tools may work better if the signal is not caught) set the environment variable XAPIAN_SIG_DFL to any value to prevent the testsuite from installing its own signal handling.

## A Brief Guide to Writing Tests

Test programs live in tests/. They mostly use a standard test rig, in testsuite/, which wraps each test, reports results, and generally packages things up nicely. The test rig counts passes, fails, and skips, catches signals and unhandled exceptions and so forth. On x86 Linux is can also check for memory leaks and accesses to uninitialised values by making use of valgrind, if it is installed.

A typical test program has three parts: the tests themselves (at the top), a table of tests (at the bottom), and a tiny main which sets the test harness in motion. It uses the table to figure out what the tests are called, and what function to call to run them.

The most important test system for most people will be apitest. This also uses the test rig, but has several tables of tests to be run depending what facilities each backend supports. A lot of the work is done by macros and helper functions, which may make it hard to work out quite what is going on, but make life easier once you've grasped what's going on. The main() and other bits are in apitest.cc, and tests themselves are in various other C++ files starting api_. Each one of these has its own tables for various different groups of tests (eg: api_db.cc, which performs tests on the API that require a database backend, has basic tests, a few specialised groups that only contain one or two tests, tests that require a writable database, tests that require a local database, and finally tests that require a remote database).

To add a new api test, figure out what the test will be dependent on and put it in the appropriate place (eg: if adding a test for a bug that occurs while writing to a database, you want a writable database, so you add a test to api_db.cc and reference it in the writabledb_tests table).

Currently, there's api_nodb.cc (no db required, largely testing query construction and boundary conditions), api_posdb.cc (db with positional information required) and api_db.cc (everything else, with lots of subgroups of tests). It's easiest to base a test on an existing one.

You'll notice in apitest.cc that it runs all appropriate test groups against each backend that is being built. We don't build any of the Muscat 3.6 compatibility backends by default any more, so the backends are inmemory, quartz and remote. If you need to create a new test group with different requirements to any current ones, put it in the appropriate api_ file (or create a new one, and add it into Makefile.am) and remember to add the group to all pertinent backends in apitest.cc.

Incidentally, when fixing bugs, it's often better to write the test before fixing the bug. Firstly, it's easier to assure yourself that the bug is (a) genuine, and (b) fixed, because you see the test go from fail to pass (though sometimes you don't get the testcase quite right, so this isn't doesn't always work as well as it should). Secondly you're more likely to write the test carefully, because once you've fixed something there's often a feeling that you should commit it for the good of the world, which tends to distract you.

## Quartz Backend Tests

Tests of the btree's functionality go in tests/btreetest.cc; tests of quartz itself in tests/quartztest.cc.

Okay. Basically, most of it you can ignore, because the framework is done for you. You are responsible for doing two things:

1. writing a minimal test or tests for the feature
2. adding that test to the list of tests to be run

Adding the test is simple. There's a test_desc array in each file that comprises a set of tests (I'll come to that in a minute), and you just add another entry. The entry is an array consisting of a name for the test and a pointer to the function that is the test. Easy.

Look at the bottom of btreetest.cc for the test_desc array. Now look up about 20 lines where there's currently just one test (which is why this is a good example). You need to write a function which will return true or false depending on whether it failed or not.

In addition, there are a bunch of macros you want to use. Things like TEST_EQUAL are all in testsuite/testsuite.h. They're pretty simple to use.