## Uses of Class
org.apache.lucene.analysis.TokenStream

Packages that use TokenStream
org.apache.lucene.analysis: API and code to convert text into indexable/searchable tokens.
org.apache.lucene.analysis.standard: A fast grammar-based tokenizer constructed with JFlex.
org.apache.lucene.document: The logical representation of a `Document` for indexing and searching.

Uses of TokenStream in org.apache.lucene.analysis

Subclasses of TokenStream in org.apache.lucene.analysis
`class` `CachingTokenFilter` This class can be used if the Tokens of a TokenStream are intended to be consumed more than once.
`class` `CharTokenizer` An abstract base class for simple, character-oriented tokenizers.
`class` `ISOLatin1AccentFilter` A filter that replaces accented characters in the ISO Latin 1 character set (ISO-8859-1) by their unaccented equivalent.
`class` `KeywordTokenizer` Emits the entire input as a single token.
`class` `LengthFilter` Removes words that are too long and too short from the stream.
`class` `LetterTokenizer` A LetterTokenizer is a tokenizer that divides text at non-letters.
`class` `LowerCaseFilter` Normalizes token text to lower case.
`class` `LowerCaseTokenizer` LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together.
`class` `PorterStemFilter` Transforms the token stream as per the Porter stemming algorithm.
`class` `SinkTokenizer` A SinkTokenizer can be used to cache Tokens for use in an Analyzer
`class` `StopFilter` Removes stop words from a token stream.
`class` `TeeTokenFilter` Works in conjunction with the SinkTokenizer to provide the ability to set aside tokens that have already been analyzed.
`class` `TokenFilter` A TokenFilter is a TokenStream whose input is another token stream.
`class` `Tokenizer` A Tokenizer is a TokenStream whose input is a Reader.
`class` `WhitespaceTokenizer` A WhitespaceTokenizer is a tokenizer that divides text at whitespace.

Fields in org.apache.lucene.analysis declared as TokenStream
`protected TokenStream` `TokenFilter.input` The source of tokens for this filter.

Methods in org.apache.lucene.analysis that return TokenStream
`TokenStream` `WhitespaceAnalyzer.reusableTokenStream(String fieldName, Reader reader)`
`TokenStream` `StopAnalyzer.reusableTokenStream(String fieldName, Reader reader)`
`TokenStream` `KeywordAnalyzer.reusableTokenStream(String fieldName, Reader reader)`
`TokenStream` `SimpleAnalyzer.reusableTokenStream(String fieldName, Reader reader)`
`TokenStream` `PerFieldAnalyzerWrapper.reusableTokenStream(String fieldName, Reader reader)`
`TokenStream` `Analyzer.reusableTokenStream(String fieldName, Reader reader)` Creates a TokenStream that is allowed to be re-used from the previous time that the same thread called this method.
`TokenStream` `WhitespaceAnalyzer.tokenStream(String fieldName, Reader reader)`
`TokenStream` `StopAnalyzer.tokenStream(String fieldName, Reader reader)` Filters LowerCaseTokenizer with StopFilter.
`TokenStream` `KeywordAnalyzer.tokenStream(String fieldName, Reader reader)`
`TokenStream` `SimpleAnalyzer.tokenStream(String fieldName, Reader reader)`
`TokenStream` `PerFieldAnalyzerWrapper.tokenStream(String fieldName, Reader reader)`
`abstract TokenStream` `Analyzer.tokenStream(String fieldName, Reader reader)` Creates a TokenStream which tokenizes all the text in the provided Reader.

Constructors in org.apache.lucene.analysis with parameters of type TokenStream
`CachingTokenFilter(TokenStream input)`
`ISOLatin1AccentFilter(TokenStream input)`
`LengthFilter(TokenStream in, int min, int max)` Build a filter that removes words that are too long or too short from the text.
`LowerCaseFilter(TokenStream in)`
`PorterStemFilter(TokenStream in)`
`StopFilter(TokenStream in, Set stopWords)` Constructs a filter which removes words from the input TokenStream that are named in the Set.
`StopFilter(TokenStream input, Set stopWords, boolean ignoreCase)` Construct a token stream filtering the given input.
`StopFilter(TokenStream input, String[] stopWords)` Construct a token stream filtering the given input.
`StopFilter(TokenStream in, String[] stopWords, boolean ignoreCase)` Constructs a filter which removes words from the input TokenStream that are named in the array of words.
`TeeTokenFilter(TokenStream input, SinkTokenizer sink)`
`TokenFilter(TokenStream input)` Construct a token stream filtering the given input.

Uses of TokenStream in org.apache.lucene.analysis.standard

Subclasses of TokenStream in org.apache.lucene.analysis.standard
`class` `StandardFilter` Normalizes tokens extracted with `StandardTokenizer`.
`class` `StandardTokenizer` A grammar-based tokenizer constructed with JFlex

Methods in org.apache.lucene.analysis.standard that return TokenStream
`TokenStream` `StandardAnalyzer.reusableTokenStream(String fieldName, Reader reader)`
`TokenStream` `StandardAnalyzer.tokenStream(String fieldName, Reader reader)` Constructs a `StandardTokenizer` filtered by a `StandardFilter`, a `LowerCaseFilter` and a `StopFilter`.

Constructors in org.apache.lucene.analysis.standard with parameters of type TokenStream
`StandardFilter(TokenStream in)` Construct filtering in.

Uses of TokenStream in org.apache.lucene.document

Methods in org.apache.lucene.document that return TokenStream
`TokenStream` `Fieldable.tokenStreamValue()` The value of the field as a TokenStream, or null.
`TokenStream` `Field.tokenStreamValue()` The value of the field as a TokesStream, or null.

Methods in org.apache.lucene.document with parameters of type TokenStream
`void` `Field.setValue(TokenStream value)` Expert: change the value of this field.

Constructors in org.apache.lucene.document with parameters of type TokenStream
`Field(String name, TokenStream tokenStream)` Create a tokenized and indexed field that is not stored.
`Field(String name, TokenStream tokenStream, Field.TermVector termVector)` Create a tokenized and indexed field that is not stored, optionally with storing term vectors.

Copyright © 2000-2008 Apache Software Foundation. All Rights Reserved.