## Uses of Class
org.apache.lucene.analysis.Token

Packages that use Token
org.apache.lucene.analysis: API and code to convert text into indexable/searchable tokens.
org.apache.lucene.analysis.br: Analyzer for Brazilian.
org.apache.lucene.analysis.cjk: Analyzer for Chinese, Japanese and Korean.
org.apache.lucene.analysis.cn: Analyzer for Chinese.
org.apache.lucene.analysis.de: Analyzer for German.
org.apache.lucene.analysis.el: Analyzer for Greek.
org.apache.lucene.analysis.fr: Analyzer for French.
org.apache.lucene.analysis.ngram
org.apache.lucene.analysis.nl: Analyzer for Dutch.
org.apache.lucene.analysis.payloads: Provides various convenience classes for creating payloads on Tokens.
org.apache.lucene.analysis.ru: Analyzer for Russian.
org.apache.lucene.analysis.sinks: Implementations of the SinkTokenizer that might be useful.
org.apache.lucene.analysis.snowball: `TokenFilter` and `Analyzer` implementations that use Snowball stemmers.
org.apache.lucene.analysis.standard: A fast grammar-based tokenizer constructed with JFlex.
org.apache.lucene.analysis.th
org.apache.lucene.index.memory: High-performance single-document main memory Apache Lucene fulltext search index.
org.apache.lucene.search.highlight: The highlight package contains classes to provide "keyword in context" features typically used to highlight search terms in the text of results pages.
org.apache.lucene.wikipedia.analysis

Uses of Token in org.apache.lucene.analysis

Methods in org.apache.lucene.analysis that return Token
`Token` `SinkTokenizer.next()` Returns the next token out of the list of cached tokens
`Token` `TokenStream.next()` Returns the next token in the stream, or null at EOS.
`Token` `CachingTokenFilter.next()`
`Token` `TeeTokenFilter.next(Token result)`
`Token` `PorterStemFilter.next(Token result)`
`Token` `StopFilter.next(Token result)` Returns the next input Token whose termText() is not a stop word.
`Token` `LowerCaseFilter.next(Token result)`
`Token` `TokenStream.next(Token result)` Returns the next token in the stream, or null at EOS.
`Token` `ISOLatin1AccentFilter.next(Token result)`
`Token` `KeywordTokenizer.next(Token result)`
`Token` `LengthFilter.next(Token result)` Returns the next input Token whose termText() is the right len
`Token` `CharTokenizer.next(Token token)`

Methods in org.apache.lucene.analysis with parameters of type Token
`void` `SinkTokenizer.add(Token t)` Override this method to cache only certain tokens, or new tokens based on the old tokens.
`Token` `TeeTokenFilter.next(Token result)`
`Token` `PorterStemFilter.next(Token result)`
`Token` `StopFilter.next(Token result)` Returns the next input Token whose termText() is not a stop word.
`Token` `LowerCaseFilter.next(Token result)`
`Token` `TokenStream.next(Token result)` Returns the next token in the stream, or null at EOS.
`Token` `ISOLatin1AccentFilter.next(Token result)`
`Token` `KeywordTokenizer.next(Token result)`
`Token` `LengthFilter.next(Token result)` Returns the next input Token whose termText() is the right len
`Token` `CharTokenizer.next(Token token)`

Uses of Token in org.apache.lucene.analysis.br

Methods in org.apache.lucene.analysis.br that return Token
`Token` `BrazilianStemFilter.next()`

Uses of Token in org.apache.lucene.analysis.cjk

Methods in org.apache.lucene.analysis.cjk that return Token
`Token` `CJKTokenizer.next()` Returns the next token in the stream, or null at EOS.

Uses of Token in org.apache.lucene.analysis.cn

Methods in org.apache.lucene.analysis.cn that return Token
`Token` `ChineseFilter.next()`
`Token` `ChineseTokenizer.next()`

Uses of Token in org.apache.lucene.analysis.de

Methods in org.apache.lucene.analysis.de that return Token
`Token` `GermanStemFilter.next()`

Uses of Token in org.apache.lucene.analysis.el

Methods in org.apache.lucene.analysis.el that return Token
`Token` `GreekLowerCaseFilter.next()`

Uses of Token in org.apache.lucene.analysis.fr

Methods in org.apache.lucene.analysis.fr that return Token
`Token` `FrenchStemFilter.next()`
`Token` `ElisionFilter.next()` Returns the next input Token whith termText() without elisioned start

Uses of Token in org.apache.lucene.analysis.ngram

Methods in org.apache.lucene.analysis.ngram that return Token
`Token` `NGramTokenFilter.next()` Returns the next token in the stream, or null at EOS.
`Token` `NGramTokenizer.next()` Returns the next token in the stream, or null at EOS.
`Token` `EdgeNGramTokenizer.next()` Returns the next token in the stream, or null at EOS.
`Token` `EdgeNGramTokenFilter.next()` Returns the next token in the stream, or null at EOS.

Uses of Token in org.apache.lucene.analysis.nl

Methods in org.apache.lucene.analysis.nl that return Token
`Token` `DutchStemFilter.next()`

Uses of Token in org.apache.lucene.analysis.payloads

Methods in org.apache.lucene.analysis.payloads that return Token
`Token` `TokenOffsetPayloadTokenFilter.next(Token result)`
`Token` `NumericPayloadTokenFilter.next(Token result)`
`Token` `TypeAsPayloadTokenFilter.next(Token result)`

Methods in org.apache.lucene.analysis.payloads with parameters of type Token
`Token` `TokenOffsetPayloadTokenFilter.next(Token result)`
`Token` `NumericPayloadTokenFilter.next(Token result)`
`Token` `TypeAsPayloadTokenFilter.next(Token result)`

Uses of Token in org.apache.lucene.analysis.ru

Methods in org.apache.lucene.analysis.ru that return Token
`Token` `RussianLowerCaseFilter.next()`
`Token` `RussianStemFilter.next()`

Uses of Token in org.apache.lucene.analysis.sinks

Methods in org.apache.lucene.analysis.sinks with parameters of type Token
`void` `DateRecognizerSinkTokenizer.add(Token t)`
`void` `TokenRangeSinkTokenizer.add(Token t)`
`void` `TokenTypeSinkTokenizer.add(Token t)`

Uses of Token in org.apache.lucene.analysis.snowball

Methods in org.apache.lucene.analysis.snowball that return Token
`Token` `SnowballFilter.next()` Returns the next input Token, after being stemmed

Uses of Token in org.apache.lucene.analysis.standard

Methods in org.apache.lucene.analysis.standard that return Token
`Token` `StandardTokenizer.next(Token result)`
`Token` `StandardFilter.next(Token result)` Returns the next token in the stream, or null at EOS.

Methods in org.apache.lucene.analysis.standard with parameters of type Token
`Token` `StandardTokenizer.next(Token result)`
`Token` `StandardFilter.next(Token result)` Returns the next token in the stream, or null at EOS.

Uses of Token in org.apache.lucene.analysis.th

Methods in org.apache.lucene.analysis.th that return Token
`Token` `ThaiWordFilter.next()`

Uses of Token in org.apache.lucene.index.memory

Methods in org.apache.lucene.index.memory that return Token
`protected Token` `SynonymTokenFilter.createToken(String synonym, Token current)` Creates and returns a token for the given synonym of the current input token; Override for custom (stateless or stateful) behaviour, if desired.
`Token` `SynonymTokenFilter.next()` Returns the next token in the stream, or null at EOS.

Methods in org.apache.lucene.index.memory with parameters of type Token
`protected Token` `SynonymTokenFilter.createToken(String synonym, Token current)` Creates and returns a token for the given synonym of the current input token; Override for custom (stateless or stateful) behaviour, if desired.

Uses of Token in org.apache.lucene.search.highlight

Methods in org.apache.lucene.search.highlight that return Token
`Token` `TokenGroup.getToken(int index)`

Methods in org.apache.lucene.search.highlight with parameters of type Token
`float` `Scorer.getTokenScore(Token token)` Called for each token in the current fragment
`float` `QueryScorer.getTokenScore(Token token)`
`boolean` `SimpleFragmenter.isNewFragment(Token token)`
`boolean` `Fragmenter.isNewFragment(Token nextToken)` Test to see if this token from the stream should be held in a new TextFragment
`boolean` `NullFragmenter.isNewFragment(Token token)`

Uses of Token in org.apache.lucene.wikipedia.analysis

Methods in org.apache.lucene.wikipedia.analysis that return Token
`Token` `WikipediaTokenizer.next(Token result)`

Methods in org.apache.lucene.wikipedia.analysis with parameters of type Token
`Token` `WikipediaTokenizer.next(Token result)`

Copyright © 2000-2008 Apache Software Foundation. All Rights Reserved.