DataparkSearch Engine 4.50: Reference manual
Prev | Chapter 3. Indexing | Next

# 3.13. Storing compressed document copies

In DataparkSearch it is possible to store compressed copies of indexed documents. Copies are stored and retrieved by the new daemon - stored, that is installed into `sbin` directory of DataparkSearch installation (default: `/usr/local/dpsearch/sbin`).

stored is also used to make documents excerpts for search results. You may use ExcerptSize command in `search.htm` template to specify average excerpt size in bytes. Value by default: 256. You may switch off document excerpts (but retain ability to show stored copies) using DoExcerpt no command in your search template.

To enable documents copies archiving without stored usage, place DoStore yes command in your `indexer.conf` file instead of stored daemon configuration.

Stored document copies are retrieved by means of `storedoc.cgi` CGI script. It requests a saved copy of a documents from stored, then a copy is displayed with user's web browser with search keywords highlighted.

To enable stored support, compile DataparkSearch with `zlib` support:

    ./configure --with-zlib <other arguments>

You may use the Store and NoStore commands to allow or disallow storing several files by pattern. For arguments of those commands are exactly the same as for the Allow command (see Section 3.10.14). All documents are stores by defaults, if support for stored is enabled.

## 3.13.1. Configure stored

To start using stored, please do the following:

- Copy `/usr/local/dpsearch/etc/stored.conf-dist` to `/usr/local/dpsearch/etc/stored.conf`.

  Edit `/usr/local/dpsearch/etc/stored.conf`

  There are several commands available for using with stored:

  - Listen tells stored which address and/or port to bind to. By default stored listens to port 7004 and any address. It is possible to specify port only:

    `Listen 7004`

    Or address only:

    `Listen 127.0.0.2`

    Or both address and port:

    `Listen 127.0.0.2:7004`

  - VarDir command specifies an alternate `var/` working directory, e.g.

    `VarDir /mnt/d/dpsearch/var/`

  - StoredFiles command specifies number of stored data files created in `var/stored/` directory, e.g.

    `StoredFiles 256`

  - OptimizeInterval command specifies interval in seconds between attempts to optimize a stored datafile, e.g.

    `OptimizeInterval 300`

  - OptimizeRatio command specifies defragmentation threshold for a stored datafile optimization.

    `OptimizeRatio 3`

- Run stored:

  `/usr/local/dpsearch/sbin/stored &`

- Configure `indexer.conf` and `search.htm` (or `searchd.conf`, if searchd is used). Specify address and port that `indexer` will use to connect to stored. Use `stored` parameter for DBAddr command, e.g.:

  `DBAddr mysql://localhost/search/?dbmode=cache&stored=localhost:7004`

## 3.13.2. How stored works

After you have successfully configured stored, the `indexer` pass downloaded documents to stored daemon. After that, stored will compress the received documents and save them.

## 3.13.3. Using stored during search

To enable displaying stored documents during search, do the following:

- Configure `storedoc.htm` (`storedoc.cgi` template) if necessary.

- Add a `storedoc.cgi` link to `search.htm` `<!--res-->` section: e.g. `<A HREF="$(stored_href)">Cached copy</A>`

- Specify `storedoc.cgi` CGI script URL in `search.htm` (by default `$(stored_href)` will return `/cgi-bin/storedoc.cgi`). In case you have to specify other URL, add to `search.htm` `variables` section the following line:

  `StoredocURL /path/to/storedoc.cgi`

  Or an absolute path:

  `StoredocURL http://servername/path/to/storedoc.cgi`

This is how stored works during search, if everything configured correctly:

1. `search.htm` displays a link to `storedoc.cgi`;

2. When user clicks the link, `storedoc.cgi` sends a query to stored daemon to the address, specified in `storedoc.htm` with the `Stored` parameter of DBAddr command;

3. After the query, stored will decompress the requested saved copy and send it to `storedoc.cgi`;

4. `storedoc.cgi` parses the received document and apply highlighting of search keywords. Highlighting method is specified with `storedoc.htm` HlBeg and HlEnd commands;

Prev | Home | Next
Using syslog | Up | DataparkSearch HTML parser