## org.apache.lucene.facet.taxonomy.lucene
Class LuceneTaxonomyWriter

    java.lang.Object
      org.apache.lucene.facet.taxonomy.lucene.LuceneTaxonomyWriter

All Implemented Interfaces:
  Closeable, TaxonomyWriter

    public class LuceneTaxonomyWriter

extends Object
implements TaxonomyWriter

`TaxonomyWriter` which uses a Lucene index to store the taxonomy information on disk, and keeps an additional in-memory cache of some or all categories.

By using a Lucene index to store the information on disk, rather than some specialized file format, we get for "free" Lucene's correctness (especially regarding multi-process concurrency), and the ability to write to any implementation of Directory (and not just the file system).

In addition to the permanently-stored Lucene index, efficiency dictates that we also keep an in-memory cache of recently seen or all categories, so that we do not need to go back to disk for every category addition to see which ordinal this category already has, if any. A `TaxonomyWriterCache` object determines the specific caching algorithm used.

This class offers some hooks for extending classes to control the `IndexWriter` instance that is used. See `openLuceneIndex(org.apache.lucene.store.Directory, org.apache.lucene.index.IndexWriterConfig.OpenMode)` and `closeLuceneIndex()` .

WARNING: This API is experimental and might change in incompatible ways in the next release.

Nested Class Summary
`static class` `LuceneTaxonomyWriter.DiskOrdinalMap` `LuceneTaxonomyWriter.OrdinalMap` maintained on file system
`static class` `LuceneTaxonomyWriter.MemoryOrdinalMap` `LuceneTaxonomyWriter.OrdinalMap` maintained in memory
`static interface` `LuceneTaxonomyWriter.OrdinalMap` Mapping from old ordinal to new ordinals, used when merging indexes wit separate taxonomies.

Field Summary
`protected org.apache.lucene.index.IndexWriter` `indexWriter`

Constructor Summary
`LuceneTaxonomyWriter(org.apache.lucene.store.Directory d)`
`LuceneTaxonomyWriter(org.apache.lucene.store.Directory directory, org.apache.lucene.index.IndexWriterConfig.OpenMode openMode)` Creates a new instance with a default cached as defined by `defaultTaxonomyWriterCache()`.
`LuceneTaxonomyWriter(org.apache.lucene.store.Directory directory, org.apache.lucene.index.IndexWriterConfig.OpenMode openMode, TaxonomyWriterCache cache)` Construct a Taxonomy writer.

Method Summary
`int` `addCategory(CategoryPath categoryPath)` addCategory() adds a category with a given path name to the taxonomy, and returns its ordinal.
`protected int` `addCategoryDocument(CategoryPath categoryPath, int length, int parent)`
`void` `addTaxonomies(org.apache.lucene.store.Directory[] taxonomies, LuceneTaxonomyWriter.OrdinalMap[] ordinalMaps)` Take all the categories of one or more given taxonomies, and add them to the main taxonomy (this), if they are not already there.
`void` `close()` Frees used resources as well as closes the underlying `IndexWriter`, which commits whatever changes made to it to the underlying `Directory`.
`protected void` `closeLuceneIndex()` A hook for extending classes to control closing the `IndexWriter` returned by `openLuceneIndex(org.apache.lucene.store.Directory, org.apache.lucene.index.IndexWriterConfig.OpenMode)`.
`protected void` `closeResources()` A hook for extending classes to close additional resources that were used.
`void` `commit()` Calling commit() ensures that all the categories written so far are visible to a reader that is opened (or reopened) after that call.
`void` `commit(Map<String,String> commitUserData)` Like commit(), but also store properties with the index.
`static TaxonomyWriterCache` `defaultTaxonomyWriterCache()` Defines the default `TaxonomyWriterCache` to use in constructors which do not specify one.
`protected int` `findCategory(CategoryPath categoryPath)` Look up the given category in the cache and/or the on-disk storage, returning the category's ordinal, or a negative number in case the category does not yet exist in the taxonomy.
`int` `getCacheMemoryUsage()` Returns the number of memory bytes used by the cache.
`int` `getParent(int ordinal)` getParent() returns the ordinal of the parent category of the category with the given ordinal.
`int` `getSize()` getSize() returns the number of categories in the taxonomy.
`protected void` `openLuceneIndex(org.apache.lucene.store.Directory directory, org.apache.lucene.index.IndexWriterConfig.OpenMode openMode)` A hook for extensions of this class to provide their own `IndexWriter` implementation or instance.
`protected org.apache.lucene.index.IndexReader` `openReader()` Open an `IndexReader` from the `indexWriter` member, by calling `IndexWriter.getReader()`.
`void` `prepareCommit()` prepare most of the work needed for a two-phase commit.
`void` `prepareCommit(Map<String,String> commitUserData)` Like above, and also prepares to store user data with the index.
`void` `setCacheMissesUntilFill(int i)` Set the number of cache misses before an attempt is made to read the entire taxonomy into the in-memory cache.
`void` `setDelimiter(char delimiter)` setDelimiter changes the character that the taxonomy uses in its internal storage as a delimiter between category components.
`static void` `unlock(org.apache.lucene.store.Directory directory)` Forcibly unlocks the taxonomy in the named directory.

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### indexWriter

    protected org.apache.lucene.index.IndexWriter indexWriter

Constructor Detail

### LuceneTaxonomyWriter

    public LuceneTaxonomyWriter(org.apache.lucene.store.Directory directory,
                                org.apache.lucene.index.IndexWriterConfig.OpenMode openMode,
                                TaxonomyWriterCache cache)
                         throws org.apache.lucene.index.CorruptIndexException,
                                org.apache.lucene.store.LockObtainFailedException,
                                IOException

  Construct a Taxonomy writer.

  Parameters:
    `directory` - The `Directory` in which to store the taxonomy. Note that the taxonomy is written directly to that directory (not to a subdirectory of it).
    `openMode` - Specifies how to open a taxonomy for writing: `APPEND` means open an existing index for append (failing if the index does not yet exist). `CREATE` means create a new index (first deleting the old one if it already existed). `APPEND_OR_CREATE` appends to an existing index if there is one, otherwise it creates a new index.
    `cache` - A `TaxonomyWriterCache` implementation which determines the in-memory caching policy. See for example `LruTaxonomyWriterCache` and `Cl2oTaxonomyWriterCache`. If null or missing, `defaultTaxonomyWriterCache()` is used.
  Throws:
    `org.apache.lucene.index.CorruptIndexException` - if the taxonomy is corrupted.
    `org.apache.lucene.store.LockObtainFailedException` - if the taxonomy is locked by another writer. If it is known that no other concurrent writer is active, the lock might have been left around by an old dead process, and should be removed using `unlock(Directory)`.
    `IOException` - if another error occurred.

### LuceneTaxonomyWriter

    public LuceneTaxonomyWriter(org.apache.lucene.store.Directory directory,
                                org.apache.lucene.index.IndexWriterConfig.OpenMode openMode)
                         throws org.apache.lucene.index.CorruptIndexException,
                                org.apache.lucene.store.LockObtainFailedException,
                                IOException

  Creates a new instance with a default cached as defined by `defaultTaxonomyWriterCache()`.

  Throws:
    `org.apache.lucene.index.CorruptIndexException`
    `org.apache.lucene.store.LockObtainFailedException`
    `IOException`

### LuceneTaxonomyWriter

    public LuceneTaxonomyWriter(org.apache.lucene.store.Directory d)
                         throws org.apache.lucene.index.CorruptIndexException,
                                org.apache.lucene.store.LockObtainFailedException,
                                IOException

Throws:
  `org.apache.lucene.index.CorruptIndexException`
  `org.apache.lucene.store.LockObtainFailedException`
  `IOException`

Method Detail

### setDelimiter

    public void setDelimiter(char delimiter)

  setDelimiter changes the character that the taxonomy uses in its internal storage as a delimiter between category components. Do not use this method unless you really know what you are doing. It has nothing to do with whatever character the application may be using to represent categories for its own use.

  If you do use this method, make sure you call it before any other methods that actually queries the taxonomy. Moreover, make sure you always pass the same delimiter for all LuceneTaxonomyWriter and LuceneTaxonomyReader objects you create for the same directory.

### unlock

    public static void unlock(org.apache.lucene.store.Directory directory)
                       throws IOException

  Forcibly unlocks the taxonomy in the named directory.

  Caution: this should only be used by failure recovery code, when it is known that no other process nor thread is in fact currently accessing this taxonomy.

  This method is unnecessary if your `Directory` uses a `NativeFSLockFactory` instead of the default `SimpleFSLockFactory`. When the "native" lock is used, a lock does not stay behind forever when the process using it dies.

  Throws:
    `IOException`

### openLuceneIndex

    protected void openLuceneIndex(org.apache.lucene.store.Directory directory,
                                   org.apache.lucene.index.IndexWriterConfig.OpenMode openMode)
                            throws org.apache.lucene.index.CorruptIndexException,
                                   org.apache.lucene.store.LockObtainFailedException,
                                   IOException

  A hook for extensions of this class to provide their own `IndexWriter` implementation or instance. Extending classes can instantiate and configure the `IndexWriter` as they see fit, including setting a `MergeScheduler`, or `IndexDeletionPolicy`, different RAM size etc.
  NOTE: the instance this method returns will be closed upon calling to `close()`. If you wish to do something different, you should override `closeLuceneIndex()`.

  Parameters:
    `directory` - the `Directory` on top of wich an `IndexWriter` should be opened.
    `openMode` - see `IndexWriterConfig.OpenMode`
  Throws:
    `org.apache.lucene.index.CorruptIndexException`
    `org.apache.lucene.store.LockObtainFailedException`
    `IOException`

### openReader

    protected org.apache.lucene.index.IndexReader openReader()
                                                      throws IOException

  Open an `IndexReader` from the `indexWriter` member, by calling `IndexWriter.getReader()`. Extending classes can override this method to return their own `IndexReader`.

  Throws:
    `IOException`

### defaultTaxonomyWriterCache

    public static TaxonomyWriterCache defaultTaxonomyWriterCache()

  Defines the default `TaxonomyWriterCache` to use in constructors which do not specify one.

  The current default is `Cl2oTaxonomyWriterCache` constructed with the parameters (1024, 0.15f, 3), i.e., the entire taxonomy is cached in memory while building it.

### close

    public void close()
               throws org.apache.lucene.index.CorruptIndexException,
                      IOException

  Frees used resources as well as closes the underlying `IndexWriter`, which commits whatever changes made to it to the underlying `Directory`.

  Specified by:
    `close` in interface `Closeable`
  Throws:
    `org.apache.lucene.index.CorruptIndexException`
    `IOException`

### getCacheMemoryUsage

    public int getCacheMemoryUsage()

  Returns the number of memory bytes used by the cache.

  Returns:
    Number of cache bytes in memory, for CL2O only; zero otherwise.

### closeResources

    protected void closeResources()
                           throws IOException

  A hook for extending classes to close additional resources that were used. The default implementation closes the `IndexReader` as well as the `TaxonomyWriterCache` instances that were used.
  NOTE: if you override this method, you should include a `super.closeResources()` call in your implementation.

  Throws:
    `IOException`

### closeLuceneIndex

    protected void closeLuceneIndex()
                             throws org.apache.lucene.index.CorruptIndexException,
                                    IOException

  A hook for extending classes to control closing the `IndexWriter` returned by `openLuceneIndex(org.apache.lucene.store.Directory, org.apache.lucene.index.IndexWriterConfig.OpenMode)`.

  Throws:
    `org.apache.lucene.index.CorruptIndexException`
    `IOException`

### findCategory

    protected int findCategory(CategoryPath categoryPath)
                        throws IOException

  Look up the given category in the cache and/or the on-disk storage, returning the category's ordinal, or a negative number in case the category does not yet exist in the taxonomy.

  Throws:
    `IOException`

### addCategory

    public int addCategory(CategoryPath categoryPath)
                    throws IOException

  Description copied from interface: `TaxonomyWriter`
  addCategory() adds a category with a given path name to the taxonomy, and returns its ordinal. If the category was already present in the taxonomy, its existing ordinal is returned.

  Before adding a category, addCategory() makes sure that all its ancestor categories exist in the taxonomy as well. As result, the ordinal of a category is guaranteed to be smaller then the ordinal of any of its descendants.

  Specified by:
    `addCategory` in interface `TaxonomyWriter`
  Throws:
    `IOException`

### addCategoryDocument

    protected int addCategoryDocument(CategoryPath categoryPath,
                                      int length,
                                      int parent)
                               throws org.apache.lucene.index.CorruptIndexException,
                                      IOException

  Throws:
    `org.apache.lucene.index.CorruptIndexException`
    `IOException`

### commit

    public void commit()
                throws org.apache.lucene.index.CorruptIndexException,
                       IOException

  Calling commit() ensures that all the categories written so far are visible to a reader that is opened (or reopened) after that call. When the index is closed(), commit() is also implicitly done. See `TaxonomyWriter.commit()`

  Specified by:
    `commit` in interface `TaxonomyWriter`
  Throws:
    `org.apache.lucene.index.CorruptIndexException`
    `IOException`

### commit

    public void commit(Map<String,String> commitUserData)
                throws org.apache.lucene.index.CorruptIndexException,
                       IOException

  Like commit(), but also store properties with the index. These properties are retrievable by `LuceneTaxonomyReader.getCommitUserData()`. See `TaxonomyWriter.commit(Map)`.

  Specified by:
    `commit` in interface `TaxonomyWriter`
  Throws:
    `org.apache.lucene.index.CorruptIndexException`
    `IOException`

### prepareCommit

    public void prepareCommit()
                       throws org.apache.lucene.index.CorruptIndexException,
                              IOException

  prepare most of the work needed for a two-phase commit. See `IndexWriter.prepareCommit()`.

  Specified by:
    `prepareCommit` in interface `TaxonomyWriter`
  Throws:
    `org.apache.lucene.index.CorruptIndexException`
    `IOException`

### prepareCommit

    public void prepareCommit(Map<String,String> commitUserData)
                       throws org.apache.lucene.index.CorruptIndexException,
                              IOException

  Like above, and also prepares to store user data with the index. See `IndexWriter.prepareCommit(Map)`

  Specified by:
    `prepareCommit` in interface `TaxonomyWriter`
  Throws:
    `org.apache.lucene.index.CorruptIndexException`
    `IOException`

### getSize

    public int getSize()

  getSize() returns the number of categories in the taxonomy.

  Because categories are numbered consecutively starting with 0, it means the taxonomy contains ordinals 0 through getSize()-1.

  Note that the number returned by getSize() is often slightly higher than the number of categories inserted into the taxonomy; This is because when a category is added to the taxonomy, its ancestors are also added automatically (including the root, which always get ordinal 0).

  Specified by:
    `getSize` in interface `TaxonomyWriter`

### setCacheMissesUntilFill

    public void setCacheMissesUntilFill(int i)

  Set the number of cache misses before an attempt is made to read the entire taxonomy into the in-memory cache.

  LuceneTaxonomyWriter holds an in-memory cache of recently seen categories to speed up operation. On each cache-miss, the on-disk index needs to be consulted. When an existing taxonomy is opened, a lot of slow disk reads like that are needed until the cache is filled, so it is more efficient to read the entire taxonomy into memory at once. We do this complete read after a certain number (defined by this method) of cache misses.

  If the number is set to `0`, the entire taxonomy is read into the cache on first use, without fetching individual categories first.

  Note that if the memory cache of choice is limited in size, and cannot hold the entire content of the on-disk taxonomy, then it is never read in its entirety into the cache, regardless of the setting of this method.

### getParent

    public int getParent(int ordinal)
                  throws IOException

  Description copied from interface: `TaxonomyWriter`
  getParent() returns the ordinal of the parent category of the category with the given ordinal.

  When a category is specified as a path name, finding the path of its parent is as trivial as dropping the last component of the path. getParent() is functionally equivalent to calling getPath() on the given ordinal, dropping the last component of the path, and then calling getOrdinal() to get an ordinal back.

  If the given ordinal is the ROOT_ORDINAL, an INVALID_ORDINAL is returned. If the given ordinal is a top-level category, the ROOT_ORDINAL is returned. If an invalid ordinal is given (negative or beyond the last available ordinal), an ArrayIndexOutOfBoundsException is thrown. However, it is expected that getParent will only be called for ordinals which are already known to be in the taxonomy.

  TODO (Facet): instead of a getParent(ordinal) method, consider having a getCategory(categorypath, prefixlen) which is similar to addCategory except it doesn't add new categories; This method can be used to get the ordinals of all prefixes of the given category, and it can use exactly the same code and cache used by addCategory() so it means less code.

  Specified by:
    `getParent` in interface `TaxonomyWriter`
  Throws:
    `IOException`

### addTaxonomies

    public void addTaxonomies(org.apache.lucene.store.Directory[] taxonomies,
                              LuceneTaxonomyWriter.OrdinalMap[] ordinalMaps)
                       throws IOException

  Take all the categories of one or more given taxonomies, and add them to the main taxonomy (this), if they are not already there.

  Additionally, fill a mapping for each of the added taxonomies, mapping its ordinals to the ordinals in the enlarged main taxonomy. These mapping are saved into an array of OrdinalMap objects given by the user, one for each of the given taxonomies (not including "this", the main taxonomy). Often the first of these will be a MemoryOrdinalMap and the others will be a DiskOrdinalMap - see discussion in {OrdinalMap}.

  Note that the taxonomies to be added are given as Directory objects, not opened TaxonomyReader/TaxonomyWriter objects, so if any of them are currently managed by an open TaxonomyWriter, make sure to commit() (or close()) it first. The main taxonomy (this) is an open TaxonomyWriter, and does not need to be commit()ed before this call.

  Throws:
    `IOException`

Copyright © 2000-2011 Apache Software Foundation. All Rights Reserved.