DataparkSearch Engine 4.28 reference manual: The Web searching software
Prev | Chapter 3. Indexing | Next

# Storing compressed document copies

In DataparkSearch it is possible to store compressed copies of indexed documents. Copies are stored and retrieved by the new daemon - stored, that is installed into `sbin` directory of DataparkSearch installation (default: `/usr/local/dpsearch/sbin`).

stored is also used to make documents excerpts for search results.

To enable documents copies archiving without stored usage, place DoStore yes command in your `indexer.conf` file instead of stored daemon configuration.

Stored document copies are retrieved by means of `storedoc.cgi` CGI script. It requests a saved copy of a documents from stored, then a copy is displayed with user's web browser with search keywords highlighted.

To enable stored support, compile DataparkSearch with `zlib` support:

    ./configure --with-zlib <other arguments>

## Configure stored

To start using stored, please do the following:

- Copy `/usr/local/dpsearch/etc/stored.conf-dist` to `/usr/local/dpsearch/etc/stored.conf`.

  Edit `/usr/local/dpsearch/etc/stored.conf`

  There are several commands available for using with stored:

  - Listen tells stored which address and/or port to bind to. By default stored listens to port 7004 and any address. It is possible to specify port only:

    `Listen 7004`

    Or address only:

    `Listen 127.0.0.2`

    Or both address and port:

    `Listen 127.0.0.2:7004`

  - VarDir command specifies an alternate `var/` working directory, e.g.

    `VarDir /mnt/d/dpsearch/var/`

  - StoredFiles command specifies number of stored data files created in `var/stored/` directory, e.g.

    `StoredFiles 256`

  - OptimizeInterval command specifies interval in seconds between attempts to optimize a stored datafile, e.g.

    `OptimizeInterval 300`

  - OptimizeRatio command specifies defragmentation threshold for a stored datafile optimization.

    `OptimizeRatio 3`

- Run stored:

  `/usr/local/dpsearch/sbin/stored &`

- Configure `indexer.conf` and `search.htm` (or `searchd.conf`, if searchd is used). Specify address and port that `indexer` will use to connect to stored. Use `stored` parameter for DBAddr command, e.g.:

  `DBAddr mysql://localhost/search/?dbmode=cache&stored=localhost:7004`

## How stored works

After you have successfully configured stored, the `indexer` pass downloaded documents to stored daemon. After that, stored will compress the received documents and save them.

## Using stored during search

To enable displaying stored documents during search, do the following:

- Configure `storedoc.htm` (`storedoc.cgi` template) if necessary.

- Add a `storedoc.cgi` link to `search.htm` `<!--res-->` section: e.g. `<A HREF="$(stored_href)">Cached copy</A>`

- Specify `storedoc.cgi` CGI script URL in `search.htm` (by default `$(stored_href)` will return `/cgi-bin/storedoc.cgi`). In case you have to specify other URL, add to `search.htm` `variables` section the following line:

  `StoredocURL /path/to/storedoc.cgi`

  Or an absolute path:

  `StoredocURL http://servername/path/to/storedoc.cgi`

This is how stored works during search, if everything configured correctly:

1. `search.htm` displays a link to `storedoc.cgi`;

2. When user clicks the link, `storedoc.cgi` sends a query to stored daemon to the address, specified in `storedoc.htm` with the `Stored` parameter of DBAddr command;

3. After the query, stored will decompress the requested saved copy and send it to `storedoc.cgi`;

4. `storedoc.cgi` parses the received document and apply highlighting of search keywords. Highlighting method is specified with `storedoc.htm` HlBeg and HlEnd commands;

Prev | Home | Next
Using syslog | Up | DataparkSearch HTML parser