DataparkSearch Engine 4.26 reference manual: The Web searching software
Prev | Chapter 8. Searching documents | Next

# Fuzzy search

## Ispell

When DataparkSearch is used with ispell support all words are normalized. It allows finding different grammatical forms of the same words. During indexing all words are stored as in the database. During the search all forms of the given keyword are selected and are taken into account. E.g. search front-end will try to find the word "test" if "testing" or "tests" is given in search query.

### Two types of ispell files

DataparkSearch understands two types of ispell files: affixes and dictionaries. Ispell affixes file contains rules for words and has approximately the following format:

    Flag V:
           E   > -E, IVE      # As in create> creative
          [^E] > IVE          # As in prevent > preventive
    Flag *N:
           E   > -E, ION      # As in create > creation
           Y   > -Y, ICATION  # As in multiply > multiplication
         [^EY] > EN           # As in fall > fallen

Ispell dictionary file contains words themselves and has the following format:

    wop/S
    word/DGJMS
    wordage/S
    wordbook
    wordily
    wordless/P

### Using Ispell

To make DataparkSearch support ispell you must specify Affix and Spell commands in `search.htm` file. The format of commands:

    Affix [lang] [charset] [ispell affixes file name]
    Spell [lang] [charset] [ispell dictionary filename]

The first parameter of both commands is two letters language abbreviation. The second is ispell files charset. The third one is filename. File names are relative to DataparkSearch `/etc` directory. Absolute paths can be also specified.

> Note: Simultaneous loading of several languages is supported, e.g.:

>     Affix en iso-8859-1 en.aff
>     Spell en iso-8859-1 en.dict
>     Affix de iso-8859-1 de.aff
>     Spell de iso-8859-1 de.dict

> Will load support for both English and German languages.

If you use `searchd`, add the same commands to `searchd.conf`.

When DataparkSearch is used with ispell support it is recommended to use `searchd`, especially for several languages support. Otherwise the starting time of `search.cgi` increases.

### Customizing dictionary

It is possible that several rare words are found in your site which are not in ispell dictionaries. You may create the list of such words in plain text file with the following format (one word per line):

    rare.dict:
    ----------
    webmaster
    intranet
    .......
    www
    http
    ---------

You may also use ispell flags in this file (for ispell flags refer to ISpell documentation). This will allow not writing the same word with different endings to the rare words file, for example "webmaster" and "webmasters". You may choose the word which has the same changing rules from existing ispell dictionary and just to copy flags from it. For example, English dictionary has this line:

`postmaster/MS`

So, webmaster with MS flags will be probably OK:

`webmaster/MS`

Then copy this file to `/etc` directory of DataparkSearch and add this file by Spell command in DataparkSearch configuration:

During next reindexing using of all documents new words will be considered as words with correct spelling. The only really incorrect words will remain.

### Where to get Ispell files

You may find ispell files for many of languages at this page.

## Synonyms

DataparkSearch also support a synonyms-based fuzzy search.

Synonyms files are installed into `etc/synonym` subdirectory of DataparkSearch installation.

To enable synonyms, add to `search.htm` search template commands like `Synonym <filename>`, e.g.:

    Synonym synonym/english.syn
    Synonym synonym/russian.syn

Filenames are relative to `etc` directory of DataparkSearch installation or absolute if begin with /

If you use `searchd`, add the same commands to `searchd.conf`.

You may create your own synonyms lists. As an example you may take the English synonyms file. In the beginning of the list please specify the following two commands:

    Language: en
    Charset:  us-ascii

- `Language` - standard (ISO 639) two-letter language abbreviation.

- `Charset` - any charset supported by DataparkSearch (see the Section called Character sets in Chapter 7).

## Accent insensitive search

Since version 4.17 DataparkSearch also support an accent insensitive search.

To enable this extension, use AccentExtensions command in your `search.htm` (or in `searchd.conf`, if searchd is used) to make automatically accent-free copies for query words, and in your `indexer.conf` config file to produce accent-free word's copies to store in database.

    AccentExtensions yes

Prev | Home | Next
Search results cache | Up | Miscellaneous