<?xml version="1.0" encoding="UTF-8"?>
<book>
    <bookinfo>
        <graphic fileref="logo.jpg"/>
        <productname>Open Source Native XML Database</productname>
        <title>Configuring Database Indexes</title>
        <orgname>The eXist Project</orgname>
    </bookinfo>
    <ci:include xmlns:ci="http://apache.org/cocoon/include/1.0" src="sidebar.xml"/>
    <chapter>
        <title>Configuring Database Indexes</title>
        <section>
            <title>Overview</title>
            <para>In this section, we discuss the types of database indexes used by eXist, as well
                as how they are created, configured and maintained. It assumes readers have a basic
                understanding of XML and XQuery.</para>
            <para>Database indexes are used extensively by eXist to facilitate efficient querying of
                the database. This is accomplished both by system-generated and user-configured
                database indexes. The current version of eXist uses three types of indexes:</para>
            <orderedlist>
                <listitem>
                    <para>
                        <emphasis>Structural Indexes</emphasis>: These index the nodal structure,
                        elements (tags), and attributes of documents in a collection.</para>
                </listitem>
                <listitem>
                    <para>
                        <emphasis>Fulltext Indexes</emphasis>: These map text tokens to text nodes
                        and attributes of documents in a collection.</para>
                </listitem>
                <listitem>
                    <para>
                        <emphasis>Range Indexes</emphasis>: These index specific text nodes and
                        attributes in a collection based on user-configured index paths and selected
                        data types.</para>
                </listitem>
            </orderedlist>
            <para>These indexes are discussed in detail in the following section.</para>
        </section>
        <section>
            <title>Index Types</title>
            <section>
                <title>Structural index</title>
                <para>This index keeps track of the elements (tags), attributes, and nodal structure
                    for all XML documents in a collection. It is created and maintained
                    automatically in eXist, and can neither be reconfigured nor disabled by the
                    user. The structural index is also required for executing nearly all XPath and
                    XQuery expressions in eXist (with the exception of wildcard-only expressions
                    such as "<command>//*</command>"). This index is stored in the database file
                        <filename>elements.dbx</filename>. </para>
                <para>Technically, the structural index maps every element and attribute
                        <emphasis>qname</emphasis> (or <emphasis>qualified name</emphasis>) in a
                    document collection to a list of <sgmltag>documentId, nodeId</sgmltag> pairs.
                    This mapping is used by the query engine to resolve queries for a given XPath
                    expression.</para>
                <para>For example, given the following query:</para>
                <synopsis>//book/section</synopsis>
                <para>eXist uses two index lookups: the first for the "book" node, and the second
                    for the "section" node. It then computes the <emphasis>structural
                    join</emphasis> between these node sets to determine which "section" elements
                    are in fact children of "book" elements.</para>
            </section>
            <section>
                <title>Fulltext Index</title>
                <para>This index maps text tokens (or blocks of structured text) to the document
                    text and attribute nodes in which they occur. It is also created and maintained
                    automatically, and stored in <filename>words.dbx</filename>. You can, however,
                    select specific documents and collections you would like eXist to index by
                    creating a configuration file. A fulltext index can also be configured to select
                    specific parts of a document. For instance, it is possible to include or exclude
                    certain elements or attributes from being indexed. Without any such
                    configuration, eXist defaults to index the entire text. (For more information on
                    user-based configuration, see <emphasis>Configuring Indexes</emphasis> below). </para>
                <para>The fulltext Index is also required for eXist's fulltext search extensions. In
                    particular, you can use the following eXist-specific operators and functions
                    that apply a fulltext index:</para>
                <itemizedlist>
                    <listitem>
                        <para>Operators: <command>&amp;=</command> and
                        <command>|=</command></para>
                    </listitem>
                    <listitem>
                        <para>Main Functions: <function>match-all()</function>,
                                <function>match-any()</function> and
                        <function>near()</function></para>
                    </listitem>
                </itemizedlist>

                <para>Check the <ulink url="xquery.xml">XQuery Documentation</ulink> for more
                    information.</para>

                <para>It is important to note that, if you have disabled fulltext indexing for
                    certain elements, these operators and functions will also be effectively
                    disabled, and will not return matches. Furthermore, eXist has NO fallback to
                    "brute-force" searching. As a result, eXist will not return results for queries
                    that normally would have results provided fulltext indexing was enabled. Note
                    also that this is in direct contrast to the operation of range indexing, which
                    does fallback to full searching of the document if no range index applies (see
                    below).</para>
            </section>

            <section>
                <title>Range index</title>
                <para>This index is "type-specific" - meaning it is based on the data type of
                    specific node values in the document. These indexes provide a shortcut for the
                    database to directly select nodes based on these type values. Unlike structural
                    and fulltext Indexes, range indexes can be created and configured directly by
                    the user, and in this sense, they are similar to indexes used by relational
                    databases. eXist therefore does not come installed with configured range
                    indexes. However, configured indexes are created when loading a document, and
                    are automatically maintained during subsequent updates to the document or a part
                    of it. </para>
                <para>Range indexes are used when matching or comparing nodes by way of standard
                    XPath operators and functions. Whenever these functions or operators are used,
                    eXist looks for, and implements, any user-defined range index that applies.
                    Unlike fulltext indexing, eXist will return the correct query results even if no
                    range index applies - defaulting to a "brute-force" inspection of the DOM if
                    necessary.</para>

                <para>To see how range indexes work, consider the following fragment:</para>

                <example>
                    <title>Example: List Entry</title>
                    <programlisting>
                        <markup><![CDATA[<items>
                            <item n="1">
                               <name>Tall Bookcase</name>
                               <price>299.99</price>
                            </item>
                            <item n="2">
                               <name>Short Bookcase</name>
                               <price>199.99</price>
                            </item>
                        </items>]]></markup>
                        </programlisting>
                </example>

                <para>With this short inventory, the <sgmltag>price</sgmltag> elements have dollar
                    values expressed as a floating-point numbers, (e.g. "299.99"), which have an
                        <ulink url="http://www.w3.org/TR/xmlschema-0/">XML Schema Definition</ulink>
                    (XSD) data type of <command>xs:double</command>. Using this builtin type to
                    define a range index, we can improve the efficiency of searches for price
                    values. (Instructions on how to configure range indexes using configuration
                    files are provided under the <emphasis>Configuring Indexes</emphasis> section
                    below.) During indexing, eXist will apply this data type selection by attempting
                    to cast all <sgmltag>price</sgmltag> values as double floating point numbers,
                    and add appropriate values to the index. Values that cannot be cast as double
                    floating point numbers are therefore ignored. This range index will then be used
                    by any expression that compares <sgmltag>price</sgmltag> to a numeric value -
                    for instance:</para>

                <synopsis>//item[price &gt; 100.0]</synopsis>

                <para>For non-string data types, the range index also provides the query engine a
                    more efficient method of data conversion. Instead of retrieving the value of
                    each selected element and casting it as a <command>xs:double</command> type, the
                    engine can evaluate the expression by using the range index as a form of lookup
                    index.</para>

                <para>The benefits of a range index can apply to string values as well. When working
                    without a range index, eXist employs the fulltext index to scan for the correct
                    nodes - but this index then requires eXist to scan the resulting nodes to filter
                    wrong matches. Using a range index, this extra scan is not required. In addtion,
                    the range index can be used for equality comparisons;
                    "<option>&lt;</option>" (less-than) and "<option>&gt;</option>"
                    (greater-than) comparisons; regular expression searches using the fn:matches()
                    function, or standard string search functions like
                    <function>fn:starts-with()</function> and <function>fn:contains()</function>.</para>

                <para>To illustrate this functionality, let's return to the previous example. If you
                    define a range index of type <command>xs:string</command> for element
                        <sgmltag>name</sgmltag>, a query on this element to select tall bookcases
                    using <function>fn:matches()</function> will be supported by the following
                    index:</para>

                <synopsis>//item[fn:matches(name, '[Tt]all\s[Bb]')]</synopsis>

                <para>Another advantage of using the range index for strings is that it can be
                    defined for elements with mixed content. For example, suppose you have the
                    following:</para>

                <example>
                    <title>Mixed Content Element</title>
                    <programlisting>
                        <markup><![CDATA[
                            <mixed>
                               <span>un</span>
                               <span>even</span>
                            </mixed>]]></markup>
                    </programlisting>
                </example>

                <para>In this case, you can query the database using an index-assisted regular
                    expression that applies across the entire <sgmltag>mixed</sgmltag> node - for
                    instance:</para>

                <synopsis>//item[fn:matches(mixed, 'un.*')]</synopsis>

                <para>In general, three conditions must be met in order to optimize a search using a
                    range index:</para>

                <orderedlist>
                    <listitem>
                        <para>
                            <emphasis>The range index must be defined on <emphasis>all</emphasis>
                                items in the input sequence.</emphasis>
                        </para>
                        <para>For example, suppose you have two collections in the database: C1 and
                            C2. If you have a range index defined for collection C1, but your query
                            happens to operate on both C1 and C2, then the range index would NOT be
                            used. The eXist "Query Optimizer" selects an optimization strategy based
                            on the entire input sequence of the query. Since, in this example, since
                            only nodes in C1 have a range index, no range index optimization would
                            be applied.</para>
                    </listitem>
                    <listitem>
                        <para>
                            <emphasis>The index data type (first argument type) must match the test
                                data type (second argument type).</emphasis>
                        </para>
                        <para>In other words, with range indexes, there is no promotion of data
                            types (i.e. no data type precedes or replaces another data type). For
                            example, if you defined an index of type <command>xs:double</command> on
                                <sgmltag>price</sgmltag>, a query that compares this element's value
                            with a string literal would not use a range index, for instance:</para>

                        <synopsis>//item[price = '1000.0']</synopsis>

                        <para>In order to apply the range index, you would need to cast the value as
                            a type <command>xs:double</command>, i.e.:</para>

                        <synopsis>//item[price = xs:double($price)] (where $price is any test value)</synopsis>

                        <para>Similarly, when we compare <command>xs:double</command> values with
                                <command>xs:integer</command> values, as in, for instance:</para>

                        <synopsis>//item[price = 1000]</synopsis>

                        <para>the range index would again not be used since the
                            <sgmltag>price</sgmltag> data type differs from the test value type,
                            although this conflict might not seem as obvious as it is with string
                            values.</para>
                    </listitem>
                    <listitem>
                        <para>
                            <emphasis>The right-hand argument has no dependencies on the current
                                context item.</emphasis>
                        </para>
                        <para>That is, the test or conditional value must not depend on the value
                            against which it is being tested. For example, range indexes will not be
                            applied given the following expression:</para>
                        <synopsis>//item[price = self]</synopsis>
                    </listitem>
                </orderedlist>
            </section>
        </section>
        <section>
            <title>Maintaining Indexes and Re-indexing</title>
            <table>
                <title/>
                <tgroup cols="2">
                    <tbody>
                        <tr>
                            <td>
                                <para>The current eXist index system automatically maintains and
                                    updates indexes defined by the user. You therefore do not need
                                    to update an index when you update a database document or
                                    collection. eXist will even update indexes following partial
                                    document updates via <emphasis>XUpdate</emphasis> or
                                        <emphasis>XQuery Update</emphasis> expressions.</para>
                                <para>The only exception to eXist's automatic update occurs when you
                                        <emphasis>add a new index definition to an existing database
                                        collection</emphasis>. In this case, the new index settings
                                    will ONLY apply to new data added to this collection, or any of
                                    its sub-collections, and NOT to previously existing data. To
                                    apply the new settings to the entire collection, you need to
                                    trigger a "manual reindex" of the collection being updated. You
                                    can re-index collections using the Java Admin Client (shown on
                                    the right). From the Admin menu, select
                                        <guimenu>File</guimenu>&#187;<guimenuitem>Reindex
                                        Collection</guimenuitem></para>
                            </td>
                            <td>
                                <screenshot><graphic fileref="resources/client4.png"/></screenshot>
                            </td>
                        </tr>
                    </tbody>
                </tgroup>
            </table>
        </section>
        <section>
            <title>Configuring Indexes</title>

            <para>For later versions of eXist, it is recommended that users configure fulltext and
                range indexes using collection-specific configuration files. These files are stored
                as standard XML documents in the system collection:
                <filename>/db/system/config</filename>, which can be accessed using the Admin
                interface or Java Client. In addition to defining settings for indexing collections,
                the configuration document specifies collection-specific other settings such as
                triggers or default permissions.</para>

            <para>The contents of the system collection (<filename>/db/system/config</filename>)
                mirror the hierarchical structure of the main collection. Configurations are
                therefore "inherited" by descendants in the hierarchy, (i.e. the configuration
                settings for the child collection are added to or override those set for the
                parent). It is furthermore possible for each collection in the hierarchy to have its
                own <emphasis>index creation policy</emphasis> defined by a configuration file. If
                no collection-specific configuration file is created for any document, the global
                settings in the main configuration file, <filename>conf.xml</filename>, will apply
                by default. That being said, the <filename>conf.xml</filename> file should only
                define the default global <emphasis>index creation policy</emphasis>.</para>

            <para>To configure indexes for a given collection - for example:
                <filename>/db/foo</filename> - you must create a new <filename>.xconf</filename>
                configuration file and store it in the system collection (e.g.
                    <filename>/db/system/config/db/foo</filename>). You can choose any name for this
                document so long as it has the <filename>.xconf</filename> extension. Note that
                since subcollections will inherit the configuration policy of their parent
                collections, you are not required to specify a configuration for every collection.</para>

            <note>
                <para>You can store only ONE <filename>.xconf</filename> configuration document per
                    collection in the system collection <filename>/db/system/config</filename>. For
                    example, the collection <filename>/db/system/config/foo</filename> would contain
                    one configuration file and/or other subcollections.</para>
            </note>

            <important>
                <para>To specify a global index configuration that applies to all collections, you
                    must create a file called <filename>collection.xconf</filename> and store it in
                    the system collection: <filename>/db/system/config/db/</filename>. (Note that
                        <filename>collection.xconf</filename> is a reserved filename.)</para>
            </important>

            <section>
                <title>Configuration Structure and Syntax</title>

                <para>Index configuration files are standard XML documents that have their elements
                    and attributes defined by the eXist namespace:</para>

                <synopsis>http://exist-db.org/collection-config/1.0</synopsis>

                <para>The following example shows a simple configuration document:</para>

                <example>
                    <title>Simple Configuration Document</title>
                    <programlisting><markup><![CDATA[<collection xmlns="http://exist-db.org/collection-config/1.0">
    <index xmlns:x="http://www.foo.com">
       <fulltext>
           <include path="//item/name"/>
       </fulltext>
       <create path="//item/price" type="xs:double"/>
    </index>
</collection>]]></markup></programlisting>
                </example>

                <para>All configuration documents have the <sgmltag>collection</sgmltag> root
                    element. These documents also have an <sgmltag>index</sgmltag> element directly
                    below the root element, which encloses the index configuration. Only ONE
                        <sgmltag>index</sgmltag> element is permitted in a document. </para>

                <para>In the <sgmltag>index</sgmltag> element are elements that define the fulltext
                    index and range index settings. The fulltext index is defined by the
                        <sgmltag>fulltext</sgmltag> element - along with <sgmltag>include</sgmltag>
                    and <sgmltag>exclude</sgmltag> elements that provide specific index paths. Range
                    indexes are defined by <sgmltag>create</sgmltag> elements. Note that an index
                    element contains only ONE <sgmltag>fulltext</sgmltag> element, but may contain
                    any number of <sgmltag>create</sgmltag> elements as range indexes.</para>

                <para>To illustrate how basic configuration document is created, let's suppose we
                    have a collection of XML documents with the following structure:</para>
                <example>
                    <title>Sample XML Document</title>
                    <programlisting><![CDATA[<?xml version="1.0" encoding="UTF-8"?>
<items xmlns:x="http://www.foo.com">
    <item n="1"">
        <name>Red Bicycle</name>
        <price specialprice="false">645.50</price>
        <stock>15</stock>
        <x:rating>8.7</x:rating>
    </item>
</items>]]></programlisting>
                </example>
                <para>A configuration document for this collection might look like the following:</para>
                <example>
                    <title>Configuration Document (<filename>collection.xconf</filename>)</title>
                    <programlisting><markup><![CDATA[<collection xmlns="http://exist-db.org/collection-config/1.0">
	<index xmlns:x="http://www.foo.com">
		<fulltext default="none" attributes="false" alphanum="false">
			<include path="//item/name"/>
		</fulltext>
		<create path="//item/@n" type="xs:integer"/>
		<create path="//item/name" type="xs:string"/>
		<create path="//item/stock" type="xs:integer"/>
		<create path="//item/price" type="xs:double"/>
		<create path="//item/prices/@specialprice" type="xs:boolean"/>
                  <create path="//item/x:rating" type="xs:double"/>
	</index>
</collection>]]></markup></programlisting>
                </example>

                <para>With this example, the fulltext default attribute is set to "none", which
                    disables the default fulltext indexing for all document elements. The exception
                    to this are the name elements using the <sgmltag>include</sgmltag> element.</para>

                <para>Notice as well that each <sgmltag>create</sgmltag> element has a
                    <option>path</option> attribute that defines the nodes to which the
                    configuration applies, and which are expressed as <emphasis>index
                    paths</emphasis>. Notice also that each <sgmltag>include</sgmltag> has a
                        <option>path</option> attribute, but not a type attribute. Since range
                    indexes are "type-specific", a node will be ignored if its value cannot be cast
                    to the specified type. For instance, if, in the above example, a price element
                    contains the string "unknown" instead of a double value, the indexer will simply
                    ignore this node. However, the value would still be found through a string
                    comparison. Note that a path component may use a namespace prefix, for which a
                    mapping has to be defined in the enclosing <option>index</option> tag. For
                    example, the document node <sgmltag>x:rating</sgmltag> in the above sample XML
                    uses the namespace <option>x</option>.</para>

                <para>Unlike the fulltext index, the range index applies ONLY to the element
                    specified in the index path, and NOT to descendant nodes of that element.
                    Returning to a previous example, consider the following markup:</para>

                <example>
                    <title>Mixed Content Element</title>
                    <programlisting>
                        <markup><![CDATA[<mixed>
    <span>un</span>
    <span>even</span>
</mixed>]]></markup>
                    </programlisting>
                </example>

                <para>which has the index definition:</para>

                <synopsis>&lt;create path=&quot;//item/mixed&quot; type=&quot;xs:string&quot;/&gt;</synopsis>

                <para>In this case, the string added to the corresponding range index during
                    indexing would be "uneven". In contrast, the fulltext index would (by default)
                    add the strings "un" and "even".</para>

                <para>However, you can configure the fulltext index to treat mixed content in the same way as the
                    range index. For this, we use a special type of &lt;include&gt;:</para>
                
                <example>
                    <title>Configure the Fulltext Index for a Mixed Content Element</title>
                    <programlisting>
                        <markup><![CDATA[<fulltext default="none" attributes="false" alphanum="false">
    <include path="//mixed" content="mixed"/>
</fulltext>]]></markup>
                    </programlisting>
                </example>
                
                <para>The concatenated text nodes of element mixed and all its descendants will be passed to the indexer
                    as one single string. The indexer thus sees and indexes "uneven" as a single token. Accordingly, using 
                    this definition, the expression:</para>
                
                <synopsis>//mixed[. &amp;= 'uneven']</synopsis>
                
                <para>will return the mixed node. You can still explicitely query for child and descendant nodes (unless
                you excluded them from the index).</para> 
                    
                <para>Although the syntax of index paths appears similar to XPath syntax, it is not
                    the same. Index path syntax is in fact much simpler than XPath, and uses the
                    following components to construct paths:</para>
                <itemizedlist>
                    <listitem>
                        <para>Elements are specified by their <emphasis>qname</emphasis></para>
                    </listitem>
                    <listitem>
                        <para>Attributes are specified by <option>@[ATTRIBUTE]</option></para>
                    </listitem>
                    <listitem>
                        <para>Child nodes are selected using the forward-slash
                        (<option>/</option>)</para>
                    </listitem>
                    <listitem>
                        <para>All descendant nodes in a tree are selected using the double
                            forward-slash (<option>//</option>)</para>
                    </listitem>
                </itemizedlist>

                <important>
                    <para>Please note that index paths are NOT the same as XPath expressions, even
                        though the syntax is similar. The main reason for this difference is that
                        index paths must be evaluated during the indexing phase. During this phase,
                        eXist engine has only minimal structural information about the document, and
                        so the use of XPath expressions is not possible. With improvements made to
                        the search method for future versions of eXist, however, the full use of
                        XPath in configuring indexes may become possible.</para>
                </important>
            </section>
            <section>
                <title>Document Elements and Attributes</title>
                <para>The following table summarizes the function and syntax of elements used in a
                    configuration document:</para>

                <variablelist>
                    <varlistentry>
                        <term>
                            <sgmltag>collection</sgmltag>
                        </term>
                        <listitem>
                            <para>Description: Configuration document root element.</para>
                            <para>Namespace:
                                <option>http://exist-db.org/collection-config/1.0</option></para>
                        </listitem>
                    </varlistentry>
                    <varlistentry>
                        <term>
                            <sgmltag>index</sgmltag>
                        </term>
                        <listitem>
                            <para>Description: Container for the index configuration.</para>
                            <para>Namespace: Optional collection namespace for prefixed index path
                                components. (e.g. <option>xmlns:x="http://www.foo.com"</option> for
                                the <option>x</option> namespace prefix).</para>
                        </listitem>
                    </varlistentry>
                    <varlistentry>
                        <term>
                            <sgmltag>fulltext</sgmltag>
                        </term>
                        <listitem>
                            <para>Description: Specifies the default fulltext indexing settings, and
                                contains <sgmltag>include</sgmltag> and <sgmltag>exclude</sgmltag>
                                elements that specify which document nodes to include or exclude
                                during indexing. </para>
                            <para>Attributes:</para>
                            <itemizedlist>
                                <listitem>
                                    <para><option>default</option>: Sets the default index range for
                                        text nodes. All fulltext indexes must have this attribute
                                        defined.</para>
                                    <itemizedlist>
                                        <title>Values</title>
                                        <listitem>
                                            <para><option>all</option>: All test nodes are indexed
                                                (default). To exclude specific nodes, use the
                                                  <sgmltag>exclude</sgmltag> element (see
                                            below).</para>
                                        </listitem>
                                        <listitem>
                                            <para><option>none</option>: No text nodes are indexed.
                                                To include specific nodes, use the
                                                <sgmltag>include</sgmltag> element (see
                                            below).</para>
                                        </listitem>
                                    </itemizedlist>
                                </listitem>
                                <listitem>
                                    <para><option>attributes</option>: Default setting to include
                                        (or exclude) attribute values during the document indexing.
                                        Compare this setting with the <option>default</option>
                                        setting for text nodes.</para>
                                    <itemizedlist>
                                        <title>Values</title>
                                        <listitem>
                                            <para><option>true</option>: Attributes are indexed
                                                (default).</para>
                                        </listitem>
                                        <listitem>
                                            <para><option>false</option>: Attributes are not
                                                indexed.</para>
                                        </listitem>
                                    </itemizedlist>
                                </listitem>
                                <listitem>
                                    <para><option>alphanum</option>: Default setting to include (or
                                        exclude) alpha-numeric values during the document indexing.
                                        These sequences include: numbers, dates, and any
                                        alpha-numeric sequence that is not a simple word string.</para>
                                    <itemizedlist>
                                        <title>Values</title>
                                        <listitem>
                                            <para><option>true</option>: Alpha-numeric strings are
                                                indexed (default).</para>
                                        </listitem>
                                        <listitem>
                                            <para><option>false</option>: Alpha-numeric strings are
                                                not indexed.</para>
                                        </listitem>
                                    </itemizedlist>
                                </listitem>
                            </itemizedlist>
                        </listitem>
                    </varlistentry>
                    <varlistentry>
                        <term>
                            <sgmltag>include</sgmltag>
                        </term>
                        <listitem>
                            <para>Description: Specifies a document node to include in fulltext
                                indexing. </para>
                            <para>Attributes: <option>path</option> (Specifies the document node
                                path).</para>
                            <para>The following includes the values of all <option>id</option>
                                attributes for all <sgmltag>article</sgmltag> elements.</para>
                            <synopsis format="linespecific"><sgmltag>include path="//article/@id" /</sgmltag></synopsis>
                            <para>The optional attribute <option>content</option> can be used to create an additional
                                index on the mixed content of a node (see description above):</para>
                            <synopsis><sgmltag>include path="//mixed" content="mixed"/</sgmltag></synopsis>
                        </listitem>
                    </varlistentry>
                    <varlistentry>
                        <term>
                            <sgmltag>exclude</sgmltag>
                        </term>
                        <listitem>
                            <para>Description: Specifies a document node to exclude in fulltext
                                indexing. </para>
                            <para>Attributes: <option>path</option> (Specifies the document node
                                path).</para>
                            <para>The following excludes the values of all <sgmltag>url</sgmltag>
                                elements and their descendants.</para>
                            <synopsis format="linespecific"><sgmltag>exclude path="//url" /</sgmltag></synopsis>
                        </listitem>
                    </varlistentry>
                    <varlistentry>
                        <term>
                            <sgmltag>create</sgmltag>
                        </term>
                        <listitem>
                            <para>Description: Defines a range index of selected elements or
                                attributes.</para>
                            <para>Attributes:</para>
                            <itemizedlist>
                                <listitem>
                                    <para><option>path</option>: Specifies an index path that
                                        defines the nodes to which the configuration applies. A
                                        range index is created for all elements or attributes that
                                        match this path.</para>
                                </listitem>
                                <listitem>
                                    <para><option>type</option>: Defines the range index data type
                                        (XSD). It is expected future versions of eXist will expand
                                        on the following supported data types.</para>
                                    <itemizedlist>
                                        <title>Values</title>
                                        <listitem>
                                            <para>
                                                <option>xs:string</option>
                                            </para>
                                        </listitem>
                                        <listitem>
                                            <para>
                                                <option>xs:dateTime</option>
                                            </para>
                                        </listitem>
                                        <listitem>
                                            <para><option>xs:integer</option> (NOTE: integers are
                                                limited to the range of values that can be
                                                represented by a Java signed long).</para>
                                        </listitem>
                                        <listitem>
                                            <para>
                                                <option>xs:double</option>
                                            </para>
                                        </listitem>
                                        <listitem>
                                            <para>
                                                <option>xs:float</option>
                                            </para>
                                        </listitem>
                                        <listitem>
                                            <para>
                                                <option>xs:boolean</option>
                                            </para>
                                        </listitem>
                                    </itemizedlist>
                                </listitem>
                            </itemizedlist>
                        </listitem>
                    </varlistentry>
                </variablelist>
            </section>
        </section>
        <section>
            <title>Debugging the Range Index</title>
            <para>It is sometimes a bit difficult to see if a range index is correctly defined or not. The simplest way to get
                some information on index usage is to set the priority for eXist's standard logger to <option>TRACE</option>.
                For example, change the <sgmltag>root</sgmltag> category in <filename>log4j.xml</filename> as follows:</para>
            <example>
                <title>Configure log4j to Display Trace Output</title>
                <programlisting>
                    <markup><![CDATA[<root>
    <priority value="trace"/>
    <appender-ref ref="console"/>
</root>]]></markup>
                </programlisting>
            </example>
            <para>This enables trace and sends all log output to the console instead of the log files. For expressions that can
                benefit from a range index, you should now see messages like "Checking if range index can be used ..." or
                "Using range index for key...".</para>
            
            <para>Another possibility to see what's in your index is to use the util:index-keys function:</para>
            
            <example>
                <title>Query to List Index Contents</title>
                <programlisting><![CDATA[declare namespace f="http://exist-db.org/xquery/test";
declare function f:term-callback($term as xs:string, $data as xs:int+) as element() {
   <entry>
     <term>{$term}</term>
     <frequency>{$data[1]}</frequency>
     <documents>{$data[2]}</documents>
     <position>{$data[3]}</position>     
   </entry>
};
util:index-keys(//city/name, "T", util:function("f:term-callback", 2), 1000)]]></programlisting>
            </example>
            <para>This query will show you all keys (starting with the letter 'T') indexed for the element selected by the 
                path expression <option>//city/name</option> together with some information about this key.</para>
        </section>
    </chapter>
</book>
