## org.apache.lucene.analysis
Class SinkTokenizer

    java.lang.Object
      org.apache.lucene.analysis.TokenStream
          org.apache.lucene.analysis.Tokenizer
              org.apache.lucene.analysis.SinkTokenizer

Direct Known Subclasses:
  DateRecognizerSinkTokenizer, TokenRangeSinkTokenizer, TokenTypeSinkTokenizer

    public class SinkTokenizer

extends Tokenizer

A SinkTokenizer can be used to cache Tokens for use in an Analyzer

See Also:
  `TeeTokenFilter`

Field Summary
`protected Iterator` `iter`
`protected List` `lst`

Fields inherited from class org.apache.lucene.analysis.Tokenizer
`input`

Constructor Summary
`SinkTokenizer()`
`SinkTokenizer(int initCap)`
`SinkTokenizer(List input)`

Method Summary
`void` `add(Token t)` Override this method to cache only certain tokens, or new tokens based on the old tokens.
`void` `close()` By default, closes the input Reader.
`List` `getTokens()` Get the tokens in the internal List.
`Token` `next()` Returns the next token out of the list of cached tokens
`void` `reset()` Reset the internal data structures to the start at the front of the list of tokens.

Methods inherited from class org.apache.lucene.analysis.Tokenizer
`reset`

Methods inherited from class org.apache.lucene.analysis.TokenStream
`next`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### lst

    protected List lst

### iter

    protected Iterator iter

Constructor Detail

### SinkTokenizer

    public SinkTokenizer(List input)

### SinkTokenizer

    public SinkTokenizer()

### SinkTokenizer

    public SinkTokenizer(int initCap)

Method Detail

### getTokens

    public List getTokens()

  Get the tokens in the internal List.

  WARNING: Adding tokens to this list requires the `reset()` method to be called in order for them to be made available. Also, this Tokenizer does nothing to protect against `ConcurrentModificationException`s in the case of adds happening while `TokenStream.next(org.apache.lucene.analysis.Token)` is being called.

  Returns:
    A List of `Token`s

### next

    public Token next()
               throws IOException

  Returns the next token out of the list of cached tokens

  Overrides:
    `next` in class `TokenStream`
  Returns:
    The next `Token` in the Sink.
  Throws:
    `IOException`

### add

    public void add(Token t)

  Override this method to cache only certain tokens, or new tokens based on the old tokens.

  Parameters:
    `t` - The `Token` to add to the sink

### close

    public void close()
               throws IOException

  Description copied from class: `Tokenizer`
  By default, closes the input Reader.

  Overrides:
    `close` in class `Tokenizer`
  Throws:
    `IOException`

### reset

    public void reset()
               throws IOException

  Reset the internal data structures to the start at the front of the list of tokens. Should be called if tokens were added to the list after an invocation of `TokenStream.next(Token)`

  Overrides:
    `reset` in class `TokenStream`
  Throws:
    `IOException`

Copyright © 2000-2008 Apache Software Foundation. All Rights Reserved.