## Uses of Class
org.apache.lucene.analysis.Tokenizer

Packages that use Tokenizer
org.apache.lucene.analysis: API and code to convert text into indexable/searchable tokens.
org.apache.lucene.analysis.standard: A fast grammar-based tokenizer constructed with JFlex.

Uses of Tokenizer in org.apache.lucene.analysis

Subclasses of Tokenizer in org.apache.lucene.analysis
`class` `CharTokenizer` An abstract base class for simple, character-oriented tokenizers.
`class` `KeywordTokenizer` Emits the entire input as a single token.
`class` `LetterTokenizer` A LetterTokenizer is a tokenizer that divides text at non-letters.
`class` `LowerCaseTokenizer` LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together.
`class` `SinkTokenizer` A SinkTokenizer can be used to cache Tokens for use in an Analyzer
`class` `WhitespaceTokenizer` A WhitespaceTokenizer is a tokenizer that divides text at whitespace.

Uses of Tokenizer in org.apache.lucene.analysis.standard

Subclasses of Tokenizer in org.apache.lucene.analysis.standard
`class` `StandardTokenizer` A grammar-based tokenizer constructed with JFlex

Copyright © 2000-2008 Apache Software Foundation. All Rights Reserved.