## Uses of Class
org.apache.lucene.analysis.TokenFilter

Packages that use TokenFilter
org.apache.lucene.analysis: API and code to convert text into indexable/searchable tokens.
org.apache.lucene.analysis.ar: Analyzer for Arabic.
org.apache.lucene.analysis.bg: Analyzer for Bulgarian.
org.apache.lucene.analysis.br: Analyzer for Brazilian Portuguese.
org.apache.lucene.analysis.cn: Analyzer for Chinese, which indexes unigrams (individual chinese characters).
org.apache.lucene.analysis.cn.smart: Analyzer for Simplified Chinese, which indexes words.
org.apache.lucene.analysis.compound: A filter that decomposes compound words you find in many Germanic languages into the word parts.
org.apache.lucene.analysis.cz: Analyzer for Czech.
org.apache.lucene.analysis.de: Analyzer for German.
org.apache.lucene.analysis.el: Analyzer for Greek.
org.apache.lucene.analysis.en: Analyzer for English.
org.apache.lucene.analysis.es: Analyzer for Spanish.
org.apache.lucene.analysis.fa: Analyzer for Persian.
org.apache.lucene.analysis.fi: Analyzer for Finnish.
org.apache.lucene.analysis.fr: Analyzer for French.
org.apache.lucene.analysis.gl: Analyzer for Galician.
org.apache.lucene.analysis.hi: Analyzer for Hindi.
org.apache.lucene.analysis.hu: Analyzer for Hungarian.
org.apache.lucene.analysis.icu: Analysis components based on ICU
org.apache.lucene.analysis.id: Analyzer for Indonesian.
org.apache.lucene.analysis.in: Analysis components for Indian languages.
org.apache.lucene.analysis.it: Analyzer for Italian.
org.apache.lucene.analysis.lv: Analyzer for Latvian.
org.apache.lucene.analysis.miscellaneous: Miscellaneous TokenStreams
org.apache.lucene.analysis.ngram: Character n-gram tokenizers and filters.
org.apache.lucene.analysis.nl: Analyzer for Dutch.
org.apache.lucene.analysis.payloads: Provides various convenience classes for creating payloads on Tokens.
org.apache.lucene.analysis.position: Filter for assigning position increments.
org.apache.lucene.analysis.pt: Analyzer for Portuguese.
org.apache.lucene.analysis.reverse: Filter to reverse token text.
org.apache.lucene.analysis.ru: Analyzer for Russian.
org.apache.lucene.analysis.shingle: Word n-gram filters
org.apache.lucene.analysis.snowball: `TokenFilter` and `Analyzer` implementations that use Snowball stemmers.
org.apache.lucene.analysis.standard: The `org.apache.lucene.analysis.standard` package contains three fast grammar-based tokenizers constructed with JFlex:
org.apache.lucene.analysis.stempel: Stempel: Algorithmic Stemmer
org.apache.lucene.analysis.sv: Analyzer for Swedish.
org.apache.lucene.analysis.synonym: Analysis components for Synonyms.
org.apache.lucene.analysis.th: Analyzer for Thai.
org.apache.lucene.analysis.tr: Analyzer for Turkish.
org.apache.lucene.collation: `CollationKeyFilter` converts each token into its binary `CollationKey` using the provided `Collator`, and then encode the `CollationKey` as a String using `IndexableBinaryStringTools`, to allow it to be stored as an index term.
org.apache.lucene.search.highlight: The highlight package contains classes to provide "keyword in context" features typically used to highlight search terms in the text of results pages.

Uses of TokenFilter in org.apache.lucene.analysis

Subclasses of TokenFilter in org.apache.lucene.analysis
`class` `ASCIIFoldingFilter` This class converts alphabetic, numeric, and symbolic Unicode characters which are not in the first 127 ASCII characters (the "Basic Latin" Unicode block) into their ASCII equivalents, if one exists.
`class` `CachingTokenFilter` This class can be used if the token attributes of a TokenStream are intended to be consumed more than once.
`class` `FilteringTokenFilter` Abstract base class for TokenFilters that may remove tokens.
`class` `ISOLatin1AccentFilter` Deprecated. If you build a new index, use `ASCIIFoldingFilter` which covers a superset of Latin 1. This class is included for use with existing indexes and will be removed in a future release (possibly Lucene 4.0).
`class` `KeywordMarkerFilter` Marks terms as keywords via the `KeywordAttribute`.
`class` `LengthFilter` Removes words that are too long or too short from the stream.
`class` `LimitTokenCountFilter` This TokenFilter limits the number of tokens while indexing.
`class` `LowerCaseFilter` Normalizes token text to lower case.
`class` `PorterStemFilter` Transforms the token stream as per the Porter stemming algorithm.
`class` `StopFilter` Removes stop words from a token stream.
`class` `TeeSinkTokenFilter` This TokenFilter provides the ability to set aside attribute states that have already been analyzed.

Uses of TokenFilter in org.apache.lucene.analysis.ar

Subclasses of TokenFilter in org.apache.lucene.analysis.ar
`class` `ArabicNormalizationFilter` A `TokenFilter` that applies `ArabicNormalizer` to normalize the orthography.
`class` `ArabicStemFilter` A `TokenFilter` that applies `ArabicStemmer` to stem Arabic words..

Uses of TokenFilter in org.apache.lucene.analysis.bg

Subclasses of TokenFilter in org.apache.lucene.analysis.bg
`class` `BulgarianStemFilter` A `TokenFilter` that applies `BulgarianStemmer` to stem Bulgarian words.

Uses of TokenFilter in org.apache.lucene.analysis.br

Subclasses of TokenFilter in org.apache.lucene.analysis.br
`class` `BrazilianStemFilter` A `TokenFilter` that applies `BrazilianStemmer`.

Uses of TokenFilter in org.apache.lucene.analysis.cn

Subclasses of TokenFilter in org.apache.lucene.analysis.cn
`class` `ChineseFilter` Deprecated. Use `StopFilter` instead, which has the same functionality. This filter will be removed in Lucene 5.0

Uses of TokenFilter in org.apache.lucene.analysis.cn.smart

Subclasses of TokenFilter in org.apache.lucene.analysis.cn.smart
`class` `WordTokenFilter` A `TokenFilter` that breaks sentences into words.

Uses of TokenFilter in org.apache.lucene.analysis.compound

Subclasses of TokenFilter in org.apache.lucene.analysis.compound
`class` `CompoundWordTokenFilterBase` Base class for decomposition token filters.
`class` `DictionaryCompoundWordTokenFilter` A `TokenFilter` that decomposes compound words found in many Germanic languages.
`class` `HyphenationCompoundWordTokenFilter` A `TokenFilter` that decomposes compound words found in many Germanic languages.

Uses of TokenFilter in org.apache.lucene.analysis.cz

Subclasses of TokenFilter in org.apache.lucene.analysis.cz
`class` `CzechStemFilter` A `TokenFilter` that applies `CzechStemmer` to stem Czech words.

Uses of TokenFilter in org.apache.lucene.analysis.de

Subclasses of TokenFilter in org.apache.lucene.analysis.de
`class` `GermanLightStemFilter` A `TokenFilter` that applies `GermanLightStemmer` to stem German words.
`class` `GermanMinimalStemFilter` A `TokenFilter` that applies `GermanMinimalStemmer` to stem German words.
`class` `GermanStemFilter` A `TokenFilter` that stems German words.

Uses of TokenFilter in org.apache.lucene.analysis.el

Subclasses of TokenFilter in org.apache.lucene.analysis.el
`class` `GreekLowerCaseFilter` Normalizes token text to lower case, removes some Greek diacritics, and standardizes final sigma to sigma.
`class` `GreekStemFilter` A `TokenFilter` that applies `GreekStemmer` to stem Greek words.

Uses of TokenFilter in org.apache.lucene.analysis.en

Subclasses of TokenFilter in org.apache.lucene.analysis.en
`class` `EnglishMinimalStemFilter` A `TokenFilter` that applies `EnglishMinimalStemmer` to stem English words.
`class` `EnglishPossessiveFilter` TokenFilter that removes possessives (trailing 's) from words.
`class` `KStemFilter` A high-performance kstem filter for english.

Uses of TokenFilter in org.apache.lucene.analysis.es

Subclasses of TokenFilter in org.apache.lucene.analysis.es
`class` `SpanishLightStemFilter` A `TokenFilter` that applies `SpanishLightStemmer` to stem Spanish words.

Uses of TokenFilter in org.apache.lucene.analysis.fa

Subclasses of TokenFilter in org.apache.lucene.analysis.fa
`class` `PersianNormalizationFilter` A `TokenFilter` that applies `PersianNormalizer` to normalize the orthography.

Uses of TokenFilter in org.apache.lucene.analysis.fi

Subclasses of TokenFilter in org.apache.lucene.analysis.fi
`class` `FinnishLightStemFilter` A `TokenFilter` that applies `FinnishLightStemmer` to stem Finnish words.

Uses of TokenFilter in org.apache.lucene.analysis.fr

Subclasses of TokenFilter in org.apache.lucene.analysis.fr
`class` `ElisionFilter` Removes elisions from a `TokenStream`.
`class` `FrenchLightStemFilter` A `TokenFilter` that applies `FrenchLightStemmer` to stem French words.
`class` `FrenchMinimalStemFilter` A `TokenFilter` that applies `FrenchMinimalStemmer` to stem French words.
`class` `FrenchStemFilter` Deprecated. Use `SnowballFilter` with `FrenchStemmer` instead, which has the same functionality. This filter will be removed in Lucene 5.0

Uses of TokenFilter in org.apache.lucene.analysis.gl

Subclasses of TokenFilter in org.apache.lucene.analysis.gl
`class` `GalicianStemFilter` A `TokenFilter` that applies `GalicianStemmer` to stem Galician words.

Uses of TokenFilter in org.apache.lucene.analysis.hi

Subclasses of TokenFilter in org.apache.lucene.analysis.hi
`class` `HindiNormalizationFilter` A `TokenFilter` that applies `HindiNormalizer` to normalize the orthography.
`class` `HindiStemFilter` A `TokenFilter` that applies `HindiStemmer` to stem Hindi words.

Uses of TokenFilter in org.apache.lucene.analysis.hu

Subclasses of TokenFilter in org.apache.lucene.analysis.hu
`class` `HungarianLightStemFilter` A `TokenFilter` that applies `HungarianLightStemmer` to stem Hungarian words.

Uses of TokenFilter in org.apache.lucene.analysis.icu

Subclasses of TokenFilter in org.apache.lucene.analysis.icu
`class` `ICUFoldingFilter` A TokenFilter that applies search term folding to Unicode text, applying foldings from UTR#30 Character Foldings.
`class` `ICUNormalizer2Filter` Normalize token text with ICU's `Normalizer2`
`class` `ICUTransformFilter` A `TokenFilter` that transforms text with ICU.

Uses of TokenFilter in org.apache.lucene.analysis.id

Subclasses of TokenFilter in org.apache.lucene.analysis.id
`class` `IndonesianStemFilter` A `TokenFilter` that applies `IndonesianStemmer` to stem Indonesian words.

Uses of TokenFilter in org.apache.lucene.analysis.in

Subclasses of TokenFilter in org.apache.lucene.analysis.in
`class` `IndicNormalizationFilter` A `TokenFilter` that applies `IndicNormalizer` to normalize text in Indian Languages.

Uses of TokenFilter in org.apache.lucene.analysis.it

Subclasses of TokenFilter in org.apache.lucene.analysis.it
`class` `ItalianLightStemFilter` A `TokenFilter` that applies `ItalianLightStemmer` to stem Italian words.

Uses of TokenFilter in org.apache.lucene.analysis.lv

Subclasses of TokenFilter in org.apache.lucene.analysis.lv
`class` `LatvianStemFilter` A `TokenFilter` that applies `LatvianStemmer` to stem Latvian words.

Uses of TokenFilter in org.apache.lucene.analysis.miscellaneous

Subclasses of TokenFilter in org.apache.lucene.analysis.miscellaneous
`class` `StemmerOverrideFilter` Provides the ability to override any `KeywordAttribute` aware stemmer with custom dictionary-based stemming.

Uses of TokenFilter in org.apache.lucene.analysis.ngram

Subclasses of TokenFilter in org.apache.lucene.analysis.ngram
`class` `EdgeNGramTokenFilter` Tokenizes the given token into n-grams of given size(s).
`class` `NGramTokenFilter` Tokenizes the input into n-grams of the given size(s).

Uses of TokenFilter in org.apache.lucene.analysis.nl

Subclasses of TokenFilter in org.apache.lucene.analysis.nl
`class` `DutchStemFilter` Deprecated. Use `SnowballFilter` with `DutchStemmer` instead, which has the same functionality. This filter will be removed in Lucene 5.0

Uses of TokenFilter in org.apache.lucene.analysis.payloads

Subclasses of TokenFilter in org.apache.lucene.analysis.payloads
`class` `DelimitedPayloadTokenFilter` Characters before the delimiter are the "token", those after are the payload.
`class` `NumericPayloadTokenFilter` Assigns a payload to a token based on the `Token.type()`
`class` `TokenOffsetPayloadTokenFilter` Adds the `Token.setStartOffset(int)` and `Token.setEndOffset(int)` First 4 bytes are the start
`class` `TypeAsPayloadTokenFilter` Makes the `Token.type()` a payload.

Uses of TokenFilter in org.apache.lucene.analysis.position

Subclasses of TokenFilter in org.apache.lucene.analysis.position
`class` `PositionFilter` Set the positionIncrement of all tokens to the "positionIncrement", except the first return token which retains its original positionIncrement value.

Uses of TokenFilter in org.apache.lucene.analysis.pt

Subclasses of TokenFilter in org.apache.lucene.analysis.pt
`class` `PortugueseLightStemFilter` A `TokenFilter` that applies `PortugueseLightStemmer` to stem Portuguese words.
`class` `PortugueseMinimalStemFilter` A `TokenFilter` that applies `PortugueseMinimalStemmer` to stem Portuguese words.
`class` `PortugueseStemFilter` A `TokenFilter` that applies `PortugueseStemmer` to stem Portuguese words.

Uses of TokenFilter in org.apache.lucene.analysis.reverse

Subclasses of TokenFilter in org.apache.lucene.analysis.reverse
`class` `ReverseStringFilter` Reverse token string, for example "country" => "yrtnuoc".

Uses of TokenFilter in org.apache.lucene.analysis.ru

Subclasses of TokenFilter in org.apache.lucene.analysis.ru
`class` `RussianLightStemFilter` A `TokenFilter` that applies `RussianLightStemmer` to stem Russian words.
`class` `RussianLowerCaseFilter` Deprecated. Use `LowerCaseFilter` instead, which has the same functionality. This filter will be removed in Lucene 4.0
`class` `RussianStemFilter` Deprecated. Use `SnowballFilter` with `RussianStemmer` instead, which has the same functionality. This filter will be removed in Lucene 4.0

Uses of TokenFilter in org.apache.lucene.analysis.shingle

Subclasses of TokenFilter in org.apache.lucene.analysis.shingle
`class` `ShingleFilter` A ShingleFilter constructs shingles (token n-grams) from a token stream.

Uses of TokenFilter in org.apache.lucene.analysis.snowball

Subclasses of TokenFilter in org.apache.lucene.analysis.snowball
`class` `SnowballFilter` A filter that stems words using a Snowball-generated stemmer.

Uses of TokenFilter in org.apache.lucene.analysis.standard

Subclasses of TokenFilter in org.apache.lucene.analysis.standard
`class` `ClassicFilter` Normalizes tokens extracted with `ClassicTokenizer`.
`class` `StandardFilter` Normalizes tokens extracted with `StandardTokenizer`.

Uses of TokenFilter in org.apache.lucene.analysis.stempel

Subclasses of TokenFilter in org.apache.lucene.analysis.stempel
`class` `StempelFilter` Transforms the token stream as per the stemming algorithm.

Uses of TokenFilter in org.apache.lucene.analysis.sv

Subclasses of TokenFilter in org.apache.lucene.analysis.sv
`class` `SwedishLightStemFilter` A `TokenFilter` that applies `SwedishLightStemmer` to stem Swedish words.

Uses of TokenFilter in org.apache.lucene.analysis.synonym

Subclasses of TokenFilter in org.apache.lucene.analysis.synonym
`class` `SynonymFilter` Matches single or multi word synonyms in a token stream.

Uses of TokenFilter in org.apache.lucene.analysis.th

Subclasses of TokenFilter in org.apache.lucene.analysis.th
`class` `ThaiWordFilter` `TokenFilter` that use `BreakIterator` to break each Token that is Thai into separate Token(s) for each Thai word.

Uses of TokenFilter in org.apache.lucene.analysis.tr

Subclasses of TokenFilter in org.apache.lucene.analysis.tr
`class` `TurkishLowerCaseFilter` Normalizes Turkish token text to lower case.

Uses of TokenFilter in org.apache.lucene.collation

Subclasses of TokenFilter in org.apache.lucene.collation
`class` `CollationKeyFilter` Converts each token into its `CollationKey`, and then encodes the CollationKey with `IndexableBinaryStringTools`, to allow it to be stored as an index term.
`class` `ICUCollationKeyFilter` Converts each token into its `CollationKey`, and then encodes the CollationKey with `IndexableBinaryStringTools`, to allow it to be stored as an index term.

Uses of TokenFilter in org.apache.lucene.search.highlight

Subclasses of TokenFilter in org.apache.lucene.search.highlight
`class` `OffsetLimitTokenFilter` This TokenFilter limits the number of tokens while indexing by adding up the current offset.

Copyright © 2000-2011 Apache Software Foundation. All Rights Reserved.