mnoGoSearch 3.2.31 reference manual: Full-featured search engine software
Prev | Chapter 7. Languages support | Next

# Segmenters for Chinese, Thai and Japanese languages
mnoGoSearch - Full Featured Web site Open Source Search Engine Software over the Internet and Intranet Web Sites Based on SQL Database. It is a Free search software covered by GNU license.
Keywords: shareware, freeware, download, internet, unix, utilities, search engine, text retrieval, knowledge retrieval, text search, information retrieval, database search, mining, intranet, webserver, index, spider, filesearch, meta, free, open source, full-text, udmsearch, website, find, opensource, search, searching, software, udmsearch, engine, indexing, system, web, ftp, http, cgi, php, SQL, MySQL, database, php3, FreeBSD, Linux, Unix, mnoGoSearch, MacOS X, Mac OS X, Windows, 2000, NT, 95, 98, GNU, GPL, url, grabbing

Traditional Chinese, Thai and Japanese writing have no spaces between words in phrase, unlike western languages. Thus, while indexing documents in these languages, the indexer needs to segment phrases into words.

## Japanese language phrase segmenter

For Japanese language phrase segmenting, one of ChaSen, a morphological system for japanes language, or MeCab, a Japanese morphological analyzer, is used. Thus, you need one of these systems to be installed before configuring and building mnoGoSearch.

To enable Japanese language phrase segmenting, use --enable-chasen or --enable-mecab switch for configure.

## Chinese language phrase segmenter

For Chinese language phrase segmenting, the frequency dictionary of Chinese words is used. Segmenting itself is done by a dynamic programming method to maximize the cumulative frequency of produced words.

To enable Chinese language phrase segmenting, you need to enable the GB2312 charset support while configuring mnoGoSearch, if you want to use `mandarin.freq`, a simplified Chinese dictionary, or enable the Big5 charset support, to use `TraditionalChinese.freq`, a traditional Chinese dictionary. You also need to specify the frequency dictionary of Chinese words with LoadChineseList in `indexer.conf` file.

    LoadChineseList [charset dictionaryfilename]

> The GB2312charset and `mandarin.freq`dictionary are used by default.

## Thai language phrase segmenter

For Thai language phrase segmenting, the frequency dictionary of Thai words is used. And segmenting itself is done as for Chinese language.

To enable Thai language phrase segmenting, you need to specify the frequency dictionary for Thai words with LoadThaiList in `indexer.conf` file.

    LoadThaiList [charset dictionaryfilename]

> The TIS-620charset and `thai.freq`dictionary are used by default

Prev | Home | Next
Making multi-language search pages | Up | Multilingual servers support