<?xml version="1.0" encoding="UTF-8"?>
<book xmlns:ci="http://apache.org/cocoon/include/1.0">
    <bookinfo>
        <graphic fileref="logo.jpg"/>
        <productname>Open Source Native XML Database</productname>
        <title>Developer's Guide</title>
        <author>
            <firstname>Wolfgang M.</firstname>
            <surname>Meier</surname>
            <affiliation>
                <address format="linespecific">
                    <email>wolfgang at exist-db.org</email>
                </address>
            </affiliation>
        </author>
    </bookinfo>
    <ci:include src="sidebar.xml"/>
    <chapter id="ch01">
        <title>Developer's Guide</title>
        <section id="overview">
            <title>Overview</title>
            <para>This section provides a quick but broad introduction to the APIs and interfaces
                provided by eXist. We begin with an overview of how to configure eXist for XQuery to
                begin creating and executing XQuery scripts for web applications. For this, we look
                at how eXist uses either the <application>XQueryServlet</application> servlet or
                    <application>XQueryGenerator</application> (Cocoon) to generate output from
                XQuery scripts. In section 3, we look at the basic REST-style API and its available
                HTTP request operations. Following that, in sections 4 and 5, we address Java
                programmers, and focus on the XML:DB API - a standard Java API used to access native
                XML database services - and its extensions. In sections 6 and 7, we discuss other
                ways to integrate eXist with Cocoon, including the XMLDBTransformer, and XSP
                Logicsheets. Sections 8 and 9 examine the network APIs for XML-RPC and its methods -
                this includes the use of XUpdate. SOAP interface is discussed as an alternative to
                XML-RPC in section 10. Finally, in section 11, we include an important appendix of
                libraries required to implement specific APIs and interfaces.</para>
        </section>

        <section id="sectXQuery">
            <title>Writing Web Applications using XQuery</title>
            <para>Not only is XQuery a powerful query language, it is also a functional programming
                language with strong typing features, and can therefore be used to implement the
                entire processing logic of a web application. Because of this functionality, much of
                the Java code of eXist web applications has gradually been replaced by XQuery
                scripts. As a result, eXist provides library modules for getting request parameters,
                getting/setting session attributes, encoding URLs and so on. Developers who have
                worked with other functional languages such as <emphasis>Lisp</emphasis> or
                    <emphasis>Scheme</emphasis> will find XQuery very easy to learn.</para>

            <section>
                <title>XQueryServlet and XQueryGenerator (Cocoon)</title>
                <para>eXist generates HTML web pages from XQuery files in two ways: the
                    XQueryServlet and XQueryGenerator. With both the XQueryServlet and
                    XQueryGenerator the compiled XQuery script is stored in a cache for future use.
                    For this, eXist compiles XQuery into a tree of expression objects, which can be
                    repeatedly executed. This code will only be recompiled if the source file has
                    changed.</para>

                <variablelist>
                    <varlistentry>
                        <term>XQueryServlet</term>
                        <listitem>
                            <para>This servlet responds to URL-patterns (e.g.
                                <filename>*.xql</filename> and <filename>*.xqy</filename>) as
                                defined in the <filename>web.xml</filename> configuration file of
                                the application. The servlet will interpret this pattern as pointing
                                to a valid XQuery file. The XQuery file is then loaded, compiled and
                                executed, and the results are then sent to the client.</para>
                            <para>To use the servlet, you must define the URL-patterns for your web
                                application by adding the following to the
                                <filename>WEB-INF/web.xml</filename> configuration file:</para>
                            <example>
                                <title>Configuration for the Servlet</title>
                                <programlisting><![CDATA[
<web-app>
    <display-name>eXist Server</display-name>
    <description>eXist Server Setup</description>
    
    <servlet>
        <servlet-name>org.exist.http.servlets.XQueryServlet</servlet-name>
        <servlet-class>org.exist.http.servlets.XQueryServlet</servlet-class>

        <init-param>
            <param-name>uri</param-name>
            <param-value>xmldb:exist:///db</param-value>
        </init-param>
    </servlet>
    
    <servlet-mapping>
	  <servlet-name>org.exist.http.servlets.XQueryServlet</servlet-name>
	  <url-pattern>*.xql</url-pattern>
    </servlet-mapping>
</web-app>
]]></programlisting>
                            </example>
                            <para>This will configure the servlet to respond to any URL-pattern
                                ending with the <option>.xql</option> file extension as specified in
                                    <sgmltag>servlet-mapping</sgmltag>. Note that the
                                <option>.xq</option> is specifically NOT used for the
                                    <sgmltag>url-pattern</sgmltag> definition so as not to interfere
                                with Cocoon examples, which exclusively use this file extension.
                                Also note that the <option>uri</option> parameter in
                                    <sgmltag>init-param</sgmltag> specifies the
                                <filename>XML:DB</filename> root collection used by the servlet. To
                                configure this parameter to have the servlet access a remote
                                database, follow instructions provided in the <ulink
                                    url="deployment.xml">deployment docs</ulink>.</para>
                        </listitem>
                    </varlistentry>
                    <varlistentry>
                        <term>XQueryGenerator (Cocoon)</term>
                        <listitem>
                            <para>As with the servlet, the Cocoon generator reads and executes
                                XQuery scripts. However, unlike the servlet, the generator passes
                                the results to a Cocoon pipeline for further processing.
                                Furthermore, the XQueryGenerator has to be configured in the Cocoon
                                sitemap (<filename>sitemap.xmap</filename>). The sitemap registers
                                the generator and configures a pipeline to map resources for
                                different web applications. For more information on configuring and
                                using sitemaps, consult the documentation provided by <ulink
                                    url="http://cocoon.apache.org">Cocoon</ulink>.The following is a
                                basic sitemap:</para>
                            <para/>
                            <example>
                                <title>Cocoon Sitemap</title>
                                <programlisting><![CDATA[
<map:sitemap xmlns:map="http://apache.org/cocoon/sitemap/1.0">
    <map:components>
        <map:generators default="file">
            <map:generator name="xquery" 
                logger="sitemap.generator.xquery"
                src="org.exist.cocoon.XQueryGenerator"/>
        </map:generators>
        <map:readers default="resource"/>
        <map:serializers default="html"/>
        <map:selectors default="browser"/>
        <map:matchers default="wildcard"/>
        <map:transformers default="xslt">
        </map:transformers>
	</map:components>
    <map:pipelines>
        <map:pipeline>
            <map:match pattern="*.xq">
                <map:generate src="{1}.xq" type="xquery"/>
                <map:serialize encoding="UTF-8" type="html"/>
            </map:match>
        </map:pipeline>
    </map:pipelines>
</map:sitemap>]]></programlisting>
                            </example>
                            <para>According to the above pipeline definition, any path ending with
                                the <option>.xq</option> extension is matched and processed by the
                                pipeline. The pipeline generates results using the XQueryGenerator
                                defined as type <option> xquery</option> in
                                <sgmltag>map:components</sgmltag>.</para>
                        </listitem>
                    </varlistentry>
                </variablelist>
            </section>

            <section>
                <title>Example XQuery Script</title>
                <para>A script example provided with eXist offers a simple number guessing game that
                    illustrates many of the features of XQuery. The source code for this game is
                    found in <ulink url="xquery/guess.xql">webapp/xquery/guess.xql</ulink>, and can
                    be viewed locally at <ulink url="http://localhost:8080/exist/examples.xml"
                        >http://localhost:8080/exist/examples.xml</ulink>. As the file extension
                    indicates, this particular script is processed by the XQueryServlet. The full
                    script is as follows:</para>
                <example>
                    <title>Guess a Number</title>
                    <programlisting><![CDATA[
xquery version "1.0";

declare namespace request="http://exist-db.org/xquery/request";
declare namespace math="java:java.lang.Math";

declare function local:random($max as xs:integer) 
as empty()
{
    let $r := ceiling(math:random() * $max) cast as xs:integer
    return
        request:set-session-attribute("random", $r)
};

declare function local:guess($guess as xs:integer,
$rand as xs:integer) as element()
{
    if ($guess lt $rand) then
        <p>Your number is too small!</p>
    else if ($guess gt $rand) then
        <p>Your number is too large!</p>
    else (
        local:random(100),
        <p>Congratulations! You guessed the right number.
        Try again!</p> 
    )
};

declare function local:main() as node()?
{
    request:create-session(),
    let $rand := request:get-session-attribute("random"),
        $guess := request:request-parameter("guess", ())
    return
        if ($rand) then local:guess($guess, $rand)
        else local:random(100)
};

<html>
    <head><title>Number Guessing</title></head>
    <body>
        <form action="{request:encode-url(request:request-uri())}">
            <table border="0">
                <tr>
                    <th colspan="2">
                        Guess a number
                    </th>
                </tr>
                <tr>
                    <td>Number:</td>
                    <td><input type="text" name="guess"
                        size="3"/></td>
                </tr>
                <tr>
                    <td colspan="2" align="left">
                        <input type="submit"/>
                    </td>
                </tr>
            </table> 
        </form>
        { local:main() }
    </body>
</html>
]]></programlisting>
                </example>
                <para>In this example, a random number is generated using the local function
                        <methodname>local:random</methodname>, which uses the Java binding to call
                    the static method <methodname>Math.random</methodname>. (For information on the
                    Java binding with eXist, check the <ulink url="xquery.xml">XQuery Docs</ulink>.)
                    The generated number is stored as an HTTP session variable (attribute). The user
                    is then asked to guess its value. If the user submits a guess, it is read from
                    the HTTP request parameter and compared to the stored number. If the numbers
                    match, a new random is generated and the game starts again. </para>

                <para>Notice that you do not have to import the request module for handling HTTP
                    parameter, but can simply declare its namespace (i.e.
                        <filename>http://exist-db.org/xquery/request</filename>) in the document.
                    The module, if it is available, is loaded automatically by the XQuery engine.
                    How do the functions in this module access the HTTP request and session objects?
                    The XQueryServlet and the XQueryGenerator both export a number of global
                    variables to the XQuery script: <option>$request</option>,
                    <option>$response</option> and <option>$session</option>. These variables store
                    the corresponding HTTP objects as passed by the servlet engine (or Cocoon),
                    which are accessed by the functions in the library module.</para>

                <para>Both XQueryServlet and the XQueryGenerator provide initialization parameters
                    to set the username and password used for requests. However, the code will also
                    check if the current HTTP session contains the session attributes
                    <option>user</option> and <option>password</option>. If so, the session settings
                    will overwrite any previous settings. For more information on changing user
                    identities, see the <ulink url="xquery/login.xql">Session
                Example</ulink>.</para>
            </section>
        </section>
        <section>
            <title>REST-Style Web API</title>
            <para>eXist provides a REST-style (or <emphasis>RESTful</emphasis>) API through HTTP,
                which provides the simplest and quickest way to access the database. To implement
                this API, all one needs is an HTTP client, which is provided by nearly all
                programming languages and environments. However, not all of the database features
                are available using this approach.</para>

            <para> When running eXist as a stand-alone server - i.e. when the database has been
                started using the shell-script <filename>bin/server.sh</filename> (Unix) or batch
                file <filename>bin/server.bat</filename> (Windows/DOS) - HTTP access is supported
                through a simple, built-in web server. This web server however has limited
                capabilities restricted to the basic operations defined by eXist's REST API (e.g.
                    <command>GET</command>, <command>POST</command>, <command>PUT</command> and ,
                    <command>DELETE</command>).</para>

            <para>When running in a servlet-context, this same server functionality is provided by
                the <application>EXistServlet</application>. In the standard eXist distribution,
                this servlet is configured to have a listen address at:</para>

            <synopsis>http://localhost:8080/exist/rest/</synopsis>

            <para>Both the stand-alone server and the servlet rely on Java class
                    <classname>org.exist.http.RESTServer</classname> to do the actual work.</para>

            <para>The server treats all HTTP request paths as paths to a database collection, i.e.
                all resources are read from the database instead of the file system. Relative paths
                are therefore resolved relative to the database root collection. For example, if you
                enter the following URL into your web-browser:</para>

            <synopsis>http://localhost:8080/exist/rest/db/shakespeare/plays/hamlet.xml</synopsis>

            <para>the server will receive an HTTP GET request for the resource
                <filename>hamlet.xml</filename> in the collection
                <filename>/db/shakespeare/plays</filename> in the database. The server will look for
                this collection, and check if the resource is available, and if so, retrieve its
                contents and send them back to the client. If the document does not exist, an
                    <emphasis>HTTP 404 (Not Found)</emphasis> status response will be returned.</para>

            <para>To keep the interface simple, the basic database operations are directly mapped to
                HTTP request methods wherever possible. The following request methods are supported:</para>

            <variablelist>
                <varlistentry>
                    <term>GET</term>
                    <listitem>
                        <para>Retrieves a representation of the resource or collection from the
                            database. XQuery and XPath queries may also be specified using GET's
                            optional parameters applied to the selected resource.</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>PUT</term>
                    <listitem>
                        <para>Uploads a resource onto the database. If required, collections are
                            automatically created, and existing resources are overwritten.</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>DELETE</term>
                    <listitem>
                        <para>Removes a resource (document or collection) from the database.</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>POST</term>
                    <listitem>
                        <para>Submits data in the form of an XML fragment in the content of the
                            request which specifies the action to take. The fragment can be either
                            an XUpdate document or a query request. Query requests are used to pass
                            complex XQuery expressions too large to be URL-encoded.</para>
                    </listitem>
                </varlistentry>
            </variablelist>

            <section>
                <title>HTTP Authentication</title>

                <para>The REST server and servlet support basic HTTP authentication, and only valid
                    users can access the database. If no username and password are specified, the
                    server assumes a "guest" user identity, which has limited capabilities. If the
                    username submitted is not known, or an incorrect password is submitted, an error
                    page (Status code 403 - Forbidden) is returned.</para>
            </section>

            <section>
                <title>GET Requests</title>
                <para>If the server receives an HTTP GET request, it first tries to locate known
                    parameters. If no parameters are given or known, it will try to locate the
                    collection or document specified in the URI database path, and return a
                    representation of this resource the client. Note that when the located resource
                    is XML, the returned <emphasis>content-type</emphasis> attribute value will be
                        <option>text/xml</option>, and for binary resources
                        <option>application/octet-stream</option>.</para>

                <para>If the path resolves to a database collection, the retrieved results are
                    returned as an XML fragment. An example fragment is shown below:</para>
                <example>
                    <title>XML Results for GET Request for a Collection</title>
                    <programlisting>
                        <markup><![CDATA[
<exist:result xmlns:exist="http://exist.sourceforge.net/NS/exist">
    <exist:collection name="/db/xinclude" owner="guest" group="guest" 
        permissions="rwur-ur-u">
        <exist:resource name="disclaimer.xml" owner="guest" group="guest" 
            permissions="rwur-ur--"/>
        <exist:resource name="sidebar.xml" owner="guest" group="guest" 
            permissions="rwur-ur--"/>
        <exist:resource name="xinclude.xml" owner="guest" group="guest" 
            permissions="rwur-ur--"/>
    </exist:collection>
</exist:result>]]></markup>
                        </programlisting>
                </example>
                <para>If an <option>xml-stylesheet</option> processing instruction is found in an
                    XML document being requested, the database will try to apply the stylesheet
                    before returning the document. Note that in this case, any relative path in a
                    hypertext link will be resolved relative to the location of the source document.
                    For example, if the document <filename>hamlet.xml</filename>, which is stored in
                    collection <filename>/db/shakespeare/plays</filename> contains the XSLT
                    processing instruction:</para>

                <synopsis>&lt;?xml-stylesheet type="text/xml" href="shakes.xsl"?&gt;</synopsis>

                <para>then the database will try to load the stylesheet from
                        <filename>/db/shakespeare/plays/shakes.xsl</filename> and apply it to the
                    document.</para>

                <para>Optionally, GET accepts the following request parameters, which must be
                    URL-encoded:</para>
                <variablelist>
                    <varlistentry>
                        <term>_xsl=<emphasis>XSL Stylesheet</emphasis></term>
                        <listitem>
                            <para>Applies an XSL stylesheet to the requested resource. If the
                                    <parameter moreinfo="none">_xsl</parameter> parameter contains
                                an external URI, the corresponding external resource is retrieved.
                                Otherwise, the path is treated as relative to the database root
                                collection and the stylesheet is loaded from the database. This
                                option will override any XSL stylesheet processing instructions
                                found in the source XML file.</para>
                            <para>Setting <parameter moreinfo="none">_xsl</parameter> to
                                <option>no</option> disables any stylesheet processing. This is
                                useful for retrieving the unprocessed XML from documents that have a
                                stylesheet declaration.</para>
                        </listitem>
                    </varlistentry>
                    <varlistentry>
                        <term>_query=<emphasis>XPath/XQuery Expression</emphasis></term>
                        <listitem>
                            <para>Executes a query specified by the request. The collection or
                                resource referenced in the request path is added to the set of
                                statically known documents for the query. </para>
                        </listitem>
                    </varlistentry>
                    <varlistentry>
                        <term>_indent=<emphasis>yes | no</emphasis></term>
                        <listitem>
                            <para>Returns indented pretty-print XML.</para>
                        </listitem>
                    </varlistentry>
                    <varlistentry>
                        <term>_encoding=<emphasis>Character Encoding Type</emphasis></term>
                        <listitem>
                            <para>Sets the character encoding for the resultant XML.</para>
                        </listitem>
                    </varlistentry>
                    <varlistentry>
                        <term>_howmany=<emphasis>Number of Items</emphasis></term>
                        <listitem>
                            <para>Specifies the number of items to return from the resultant
                                sequence.</para>
                        </listitem>
                    </varlistentry>
                    <varlistentry>
                        <term>_start=<emphasis>Starting Position in Sequence</emphasis></term>
                        <listitem>
                            <para>Specifies the index position of the first item in the result
                                sequence to be returned</para>
                        </listitem>
                    </varlistentry>
                    <varlistentry>
                        <term>_wrap=<emphasis>yes | no</emphasis></term>
                        <listitem>
                            <para>Specifies whether the returned query results are to be wrapped
                                into a surrounding <sgmltag>exist:result</sgmltag> element. The
                                default value is <option>yes</option>.</para>
                        </listitem>
                    </varlistentry>
                </variablelist>
                <para>EXAMPLE: The following URI will find all <sgmltag>SPEECH</sgmltag> elements in
                    the collection <filename>/db/shakespeare</filename> with "Juliet" as the
                        <sgmltag>SPEAKER</sgmltag>. As specified, it will return ten items from the
                    result sequence, starting at position 1:</para>
                <synopsis>http://localhost:8080/exist/rest/db/shakespeare?_query=//SPEECH[SPEAKER=%22JULIET%22]&amp;_start=1&amp;_howmany=10</synopsis>
            </section>
            <section>
                <title>PUT Requests</title>
                <para>Documents can be stored or updated using an HTTP PUT request. The request URI
                    points to the location where the document will be stored. As defined by the HTTP
                    specifications, an existing document at the specified path will be updated, i.e.
                    removed, before storing the new resource. As well, any collections defined in
                    the path that do not exist will be created automatically.</para>
                <para>For example, the following Python script stores a document (the name of which
                    is specified on the command-line) in the database collection
                    <filename>/db/test</filename>, which will be created if this collection does not
                    exist. Note that the HTTP header field <option>content-type</option> is
                    specified as <option>text/xml</option>, since otherwise the document is stored
                    as a binary resource.</para>

                <example>
                    <title>PUT Example using Python (See: samples/http/put.py)</title>
                    <programlisting><![CDATA[
import httplib
import sys
from string import rfind

collection = sys.argv[1]
file = sys.argv[2]

f = open(file, 'r')
print "reading file %s ..." % file
xml = f.read()
f.close()

p = rfind(file, '/')
if p > -1:
    doc = file[p+1:]
else:
    doc = file
print doc
print "storing document to collection %s ..." % collection
con = httplib.HTTP('localhost:8080')
con.putrequest('PUT', '/exist/rest/%s/%s' % (collection, doc))
con.putheader('Content-Type', 'text/xml')
clen = len(xml)
con.putheader('Content-Length', `clen`)
con.endheaders()
con.send(xml)

errcode, errmsg, headers = con.getreply()

if errcode != 200:
    f = con.getfile()
    print 'An error occurred: %s' % errmsg
    f.close()
else:
    print "Ok."
]]></programlisting>
                </example>
            </section>
            <section>
                <title>DELETE Requests</title>
                <para>DELETE removes a collection or resource from the database. For this, the
                    server first checks if the request path points to an existing database
                    collection or resource, and once found, removes it.</para>
            </section>
            <section>
                <title>POST Requests</title>
                <para>POST requests require an XML fragment in the content of the request, which
                    specifies the action to take.</para>
                <para>If the root node of the fragment uses the XUpdate namespace
                        (<filename>http://www.xmldb.org/xupdate</filename>), the fragment is sent to
                    the XUpdateProcessor to be processed. Otherwise, the root node will have the
                    namespace for eXist requests
                    (<filename>http://exist.sourceforge.net/NS/exist</filename>), in which case the
                    fragment is interpreted as an <emphasis>extended query request</emphasis>.
                    Extended query requests can be used to post complex XQuery scripts that are too
                    large to be encoded in a GET request.</para>
                <para>The structure of the POST XML request is as follows:</para>
                <example>
                    <title>Extended Query Request</title>
                    <programlisting><![CDATA[
<query xmlns="http://exist.sourceforge.net/NS/exist"
    start="[first item to be returned]" 
    max="[maximum number of items to be returned]">
    <text>[XQuery expression]</text>
    <properties>
        <property name="[name1]" value="[value1]"/>
    </properties>
</query>
]]></programlisting>
                </example>
                <para>The root element <option>query</option> identifies the fragment as an extended
                    query request, and the XQuery expression for this request is enclosed in the
                        <option>text</option> element. Optional output properties, such as
                    pretty-print, may be passed in the <option>properties</option> element. An
                    example of POST for Perl is provided below:</para>
                <example>
                    <title>POST Example using Perl (See: samples/http/search.pl)</title>
                    <programlisting><![CDATA[
require LWP::UserAgent;

$URL = 'http://localhost:8080/exist/rest/db/';
$QUERY = <<END;
<?xml version="1.0" encoding="UTF-8"?>
<query xmlns="http://exist.sourceforge.net/NS/exist"
    start="1" max="20">
    <text>
        for \$speech in //SPEECH[LINE &= 'corrupt*']
        order by \$speech/SPEAKER[1]
        return
            <hit>{\$speech}</hit>
    </text>
    <properties>
        <property name="indent" value="yes"/>
    </properties>
</query>
END

$ua = LWP::UserAgent->new();
$req = HTTP::Request->new(POST => $URL);
$req->content_type('text/xml');
$req->content($QUERY);

$res = $ua->request($req);
if($res->is_success) {
    print $res->content . "\n";
} else {
    print "Error:\n\n" . $res->status_line . "\n";
}
]]></programlisting>
                </example>
                <note>
                    <para>Please note that you may have to enclose the XQuery expression in a CDATA
                        section (i.e. &lt;![CDATA[ ... ]]&gt;) to avoid parsing errors (this
                        is not shown above).</para>
                </note>
                <para>The returned query results are enclosed in the <sgmltag>exist:result</sgmltag>
                    element, which are shown below for the above example:</para>
                <example>
                    <title>Returned Results for POST Request</title>
                    <screen><![CDATA[
<exist:result xmlns:exist="http://exist.sourceforge.net/NS/exist" hits="2628" start="1" count="10">
<SPEECH xmlns:exist="http://exist.sourceforge.net/NS/exist">
    <SPEAKER>BERNARDO</SPEAKER>
    <LINE>Who's there?</LINE>
</SPEECH>
... more items follow ...
</exist:result>
]]></screen>
                </example>
            </section>
        </section>

        <section>
            <title>Writing Java Applications with the XML:DB API</title>

            <para>The preferred way to work with eXist when developing Java applications is to use
                the XML:DB API. This API provides a common interface to native or XML-enabled
                databases and supports the development of portable, reusable applications. eXist's
                implementation of the XML:DB standards follows the Xindice implementation, and
                conforms to the latest working drafts put forth by the <ulink
                    url="http://xmldb-org.sourceforge.net/xapi/">XML:DB Initiative</ulink>. For more
                information, refer to the <ulink url="api/index.html">Javadocs for this API</ulink>.</para>

            <para>The basic components employed by the XML:DB API are <emphasis>drivers</emphasis>,
                    <emphasis>collections</emphasis>, <emphasis>resources</emphasis> and
                    <emphasis>services</emphasis>.</para>

            <para><emphasis>Drivers</emphasis> are implementations of the database interface that
                encapsulate the database access logic for specific XML database products. They are
                provided by the product vendor and must be registered with the database manager.</para>

            <para>A <emphasis>collection</emphasis> is a hierarchical container for
                    <emphasis>resources</emphasis> and further sub-collections. Currently two
                different resources are defined by the API: <classname>XMLResource</classname> and
                    <classname>BinaryResource</classname>. An <classname>XMLResource</classname>
                represents an XML document or a document fragment, selected by a previously executed
                XPath query.</para>
            <para>Finally, <emphasis>services</emphasis> are requested for special tasks such as
                querying a collection with XPath, or managing a collection.</para>
            <note>
                <para>There are several XML:DB examples provided in eXist's
                    <filename>samples</filename> directory . To start an example, use the <filename
                        moreinfo="none">start.jar</filename> jar file and pass the name of the
                    example class as the first parameter, for instance:</para>
                <synopsis format="linespecific">java -jar start.jar 
					org.exist.examples.xmldb.Retrieve [- other options]</synopsis>
            </note>
            <para>Programming with the XML:DB API is straightforward. You will find some code
                examples in the <filename>samples/org/exist/examples/xmldb</filename> directory. In
                the following simple example, a document can be retrieved from the eXist server and
                printed to standard output.</para>
            <example>
                <title>Retrieving a Document with XML:DB</title>
                <programlisting format="linespecific"><![CDATA[
import org.xmldb.api.base.*;
import org.xmldb.api.modules.*;
import org.xmldb.api.*;
import javax.xml.transform.OutputKeys;

public class RetrieveExample {
    protected static String URI = "xmldb:exist://localhost:8080/exist/xmlrpc";

    public static void main(String args[]) throws Exception {
        String driver = "org.exist.xmldb.DatabaseImpl";
        
        // initialize database driver
        Class cl = Class.forName(driver);
        Database database = (Database) cl.newInstance();
        DatabaseManager.registerDatabase(database);

        // get the collection
        Collection col = DatabaseManager.getCollection(URI + args[0]);
        col.setProperty(OutputKeys.INDENT, "no");
        XMLResource res = (XMLResource)col.getResource(args[1]);
        if(res == null)
            System.out.println("document not found!");
        else
            System.out.println(res.getContent());   }
}
        ]]></programlisting>
            </example>
            <para>With this example, the database driver class for eXist
                    (<classname>org.exist.xmldb.DatabaseImpl</classname>) is first registered with
                the <classname>DatabaseManager</classname>. Next we obtain a
                <classname>Collection</classname> object from the database manager by calling the
                static method <methodname>DatabaseManger.getCollection()</methodname>. The method
                expects a fully qualified URI for its parameter value, which identifies the desired
                collection. The format of this URI should look like the following:</para>
            <synopsis format="linespecific">xmldb:[DATABASE-ID]://[HOST-ADDRESS]/db/collection</synopsis>
            <para>Because more than one database driver can be registered with the database manager,
                the first part of the URI (<option>xmldb:exist</option>) is required to determine
                which driver class to use. The <emphasis>database-id</emphasis> is used by the
                database manager to select the correct driver from its list of available drivers. To
                use eXist, this ID should always be "exist" (unless you have set up multiple
                database instances; additional instances may have other names).</para>
            <para>The final part of the URI identifies the collection path, and optionally the host
                address of the database server on the network. Internally, eXist uses two different
                driver implementations: The first talks to a remote database engine using XML-RPC
                calls, the second has direct access to a local instance of eXist. The root
                collection is always identified by <option>/db</option>. For example, the URI</para>
            <synopsis format="linespecific">xmldb:exist://localhost:8080/exist/xmlrpc/db/shakespeare/plays</synopsis>
            <para>references the Shakespeare collection on a remote server running the XML-RPC
                interface as a servlet at <option>localhost:8080/exist/xmlrpc</option>. If we leave
                out the host address, the XML:DB driver will try to connect to a locally attached
                database instance, e.g.:</para>
            <synopsis format="linespecific">xmldb:exist:///db/shakespeare/plays</synopsis>
            <para>In this case, we have to tell the XML:DB driver that it should create a new
                database instance if none has been started. This is done by setting the
                    <option>create-database</option> property of class
                <classname>Database</classname> to "true" (more information on embedded use of eXist
                can be found in the <ulink url="deployment.xml">deployment guide</ulink>.</para>
            <para>The <methodname>setProperty</methodname> calls are used to set database-specific
                parameters. In this case, pretty-printing of XML output is turned on for the
                collection. eXist uses the property keys defined in the standard Java package
                    <filename>javax.xml.transform</filename>. Thus, in Java you can simply use class
                    <classname>OutputKeys</classname> to get the correct keys.</para>
            <para>Calling <methodname>col.getResource()</methodname> finally retrieves the document,
                which is returned as an <classname>XMLResource</classname>. All resources have a
                method <methodname>getContent()</methodname>, which returns the resource's content,
                depending on it's type. In this case we retrieve the content as type
                    <classname>String</classname>.</para>
            <para>To query the repository, we may either use the standard
                    <classname>XPathQueryService</classname> or eXist's
                <classname>XQueryService</classname> class. The XML:DB API defines different kinds
                of services, which may or may not be provided by the database. The
                    <methodname>getService</methodname> method of class
                <classname>Collection</classname> calls a service if it is available. The method
                expects the service name as the first parameter, and the version (as a string) as
                the second, which is used to distinguish between different versions of the service
                defined by the XML:DB API.</para>
            <para>The following is an example of using the XML:DB API to execute a database query:</para>

            <example>
                <title>Querying the Database (XML:DB API)</title>
                <programlisting format="linespecific"><![CDATA[import org.xmldb.api.base.*;
import org.xmldb.api.modules.*;
import org.xmldb.api.*;

public class QueryExample {
    public static void main(String args[]) throws Exception {
        String driver = "org.exist.xmldb.DatabaseImpl";
        Class cl = Class.forName(driver);			
        Database database = (Database)cl.newInstance();
        DatabaseManager.registerDatabase(database);
        
        Collection col = 
            DatabaseManager.getCollection(
                "xmldb:exist://localhost:8080/exist/xmlrpc/db"
            );
        XPathQueryService service =
            (XPathQueryService) col.getService("XPathQueryService", "1.0");
        service.setProperty("indent", "yes");
                
        ResourceSet result = service.query(args[0]);
        ResourceIterator i = result.getIterator();
        while(i.hasMoreResources()) {
            Resource r = i.nextResource();
            System.out.println((String)r.getContent());
        }
    }
}
        ]]></programlisting>
            </example>
            <para>To execute the query, method <methodname>service.query(xpath)</methodname> is
                called. This method returns a <classname>ResourceSet</classname>, containing the
                Resources found by the query. <methodname>ResourceSet.getIterator()</methodname>
                gives us an iterator over these resources. Every Resource contains a single document
                fragment or value selected by the XPath expression.</para>
            <para>Internally, eXist does not distinguish between XPath and XQuery expressions.
                    <classname>XQueryService</classname> thus maps to the same implementation class
                as <classname>XPathQueryService</classname>. However, it provides a few additional
                methods. Most important, when talking to an embedded database,
                    <classname>XQueryService</classname> allows for the XQuery expression to be
                compiled as an internal representation, which can then be reused. With compilation,
                the previous example code would look as follows:</para>
            <example>
                <title>Compiling a Query (XML:DB API)</title>
                <programlisting format="linespecific"><![CDATA[import org.xmldb.api.base.*;
import org.xmldb.api.modules.*;
import org.xmldb.api.*;
import org.exist.xmldb.XQueryService;

public class QueryExample {
    public static void main(String args[]) throws Exception {
        String driver = "org.exist.xmldb.DatabaseImpl";
        Class cl = Class.forName(driver);			
        Database database = (Database)cl.newInstance();
        database.setProperty("create-database", "true");
        DatabaseManager.registerDatabase(database);
        
        Collection col = 
            DatabaseManager.getCollection("xmldb:exist:///db");
        XQueryService service =
            (XQueryService) col.getService("XQueryService", "1.0");
        service.setProperty("indent", "yes");
        
        CompiledExpression compiled = service.compile(args[0]);
        ResourceSet result = service.execute(compiled);
        ResourceIterator i = result.getIterator();
        while(i.hasMoreResources()) {
            Resource r = i.nextResource();
            System.out.println((String)r.getContent());
        }
    }
}
        ]]></programlisting>
            </example>
            <para>The XML-RPC server automatically caches compiled expressions, and so calling
                    <methodname>compile</methodname> through the remote driver produces no effect if
                the expression is already cached.</para>
            <para>Next, we would like to store a new document into the repository. This is done by
                creating a new <classname>XMLResource</classname>, assigning it the content of the
                new document, and calling the <methodname>storeResource</methodname> method of class
                    <classname>Collection</classname>. First, a new Resource is created by method
                    <methodname>Collection.createResource()</methodname>, and expects two
                parameters: the id and type of resource being created. If the id-parameter is null,
                a unique resource-id will be automatically generated .</para>
            <para>In some cases, the collection may not yet exist, and so we must create it. To
                create a new collection, call the <methodname>createCollection</methodname> method
                of the <classname>CollectionManagementService</classname> service. In the following
                example, we simply start at the root-collection object to get the
                    <classname>CollectionManagementService</classname> service.</para>
            <example>
                <title>Adding a File (XML:DB API)</title>
                <programlisting format="linespecific"><![CDATA[public class StoreExample {
    public final static String URI = "xmldb:exist://localhost:8080/exist/xmlrpc";

    public static void main(String args[]) throws Exception {
        if(args.length < 2) {
            System.out.println("usage: StoreExample collection-path document");
            System.exit(1);
        }

        String collection = args[0], file = args[1];

        // initialize driver
        String driver = "org.exist.xmldb.DatabaseImpl";
        Class cl = Class.forName(driver);
        Database database = (Database)cl.newInstance();
        DatabaseManager.registerDatabase(database);

        // try to get collection
        Collection col =
            DatabaseManager.getCollection(URI + collection);
        if(col == null) {
            // collection does not exist: get root collection and create
            // for simplicity, we assume that the new collection is a
            // direct child of the root collection, e.g. /db/test.
            // the example will fail otherwise.
            Collection root = DatabaseManager.getCollection(URI + "/db");
            CollectionManagementService mgtService = (CollectionManagementService)
                root.getService("CollectionManagementService", "1.0");
            col = mgtService.createCollection(collection.substring("/db".length()));
        }
        // create new XMLResource; an id will be assigned to the new resource
        XMLResource document = (XMLResource)col.createResource(null, "XMLResource");
        File f = new File(file);
        if(!f.canRead()) {
            System.out.println("cannot read file " + file);
            return;
        }
        document.setContent(f);
        System.out.print("storing document " + document.getId() + "...");
        col.storeResource(document);
        System.out.println("ok.");
    }
}]]></programlisting>
            </example>
            <para>Please note that the <classname>XMLResource.setContent()</classname> method takes
                a Java object as its parameter. The eXist driver checks if the object is a File.
                Otherwise, the object is transformed into a String by calling the object's
                    <methodname>toString()</methodname> method. Passing a File has one big
                advantage: If the database is running in the embedded mode, the file will be
                directly passed to the indexer. Thus, the file's content does not have to be loaded
                into the main memory. This is handy if your files are very large.</para>
        </section>
        <section>
            <title>Extensions to XML:DB</title>
            <section>
                <title>Additional Services</title>
                <para>eXist provides several services in addition to those defined by the XML:DB
                    specification:</para>
                <para>The <ulink url="api/org/exist/xmldb/UserManagementService.html"
                        >UserManagementService</ulink> service contains methods to manage users and
                    handle permissions. These methods resemble common Unix commands such as
                        <methodname>chown</methodname> or <methodname>chmod</methodname>. As with
                    other services, <classname>UserManagementService</classname> can be retrieved
                    from a collection object, as in:</para>
                <synopsis format="linespecific">UserManagementService service = 
                    (UserManagementService)collection.getService("UserManagementService", "1.0");</synopsis>
                <para>Another service called <ulink
                        url="api/org/exist/xmldb/DatabaseInstanceManager.html"
                        >DatabaseInstanceManager</ulink>, provides a single method to shut down the
                    database instance accessed by the driver. You have to be a member of the
                        <option>dba</option> user group to use this method or an exception will be
                    thrown. See the <ulink url="deployment.xml#embedded">Deployment Guide</ulink>
                    for an example.</para>
                <para>Finally, interface <ulink url="api/org/exist/xmldb/IndexQueryService.html"
                        >IndexQueryService</ulink> supports access to the terms and elements
                    contained in eXist's internal index. Method getIndexedElements() returns a list
                    of element occurrences for the current collection. For each occurring element,
                    the element's name and a frequency count is returned.</para>
                <para>Method scanIndexTerms() allows for a retrieval of the list of occurring words
                    for the current collection. This might be useful, for example, to provide users
                    a list of searchable terms together with their frequency.</para>
            </section>
            <section id="multidb">
                <title>Multiple Database Instances</title>
                <para>As explained above, passing a local XML:DB URI to the
                        <classname>DatabaseManager</classname> means that the driver will try to
                    start or access an embedded database instance. You can configure more than one
                    database instance by setting the location of the central configuration file. The
                    configuration file is set through the <option>configuration</option> property of
                    the <classname>DatabaseImpl</classname> driver class. If you would like to use
                    different drivers for different database instances, specify a name for the
                    created instance through the <option>database-id</option> property. You may
                    later use this name in the URI to refer to a database instance. The following
                    fragment sets up two instances:</para>
                <example>
                    <title>Multiple Database Instances</title>
                    <programlisting format="linespecific"><![CDATA[
// initialize driver
String driver = "org.exist.xmldb.DatabaseImpl";
Class cl = Class.forName(driver);			
Database database1 = (Database)cl.newInstance();
database1.setProperty("create-database", "true");
database1.setProperty("configuration", "/home/exist/test/conf.xml");
database1.setProperty("database-id", "test");
DatabaseManager.registerDatabase(database1);

Database database2 = (Database)cl.newInstance();
database2.setProperty("create-database", "true");
database2.setProperty("configuration", "/home/exist/production/conf.xml");
database2.setProperty("database-id", "exist");
DatabaseManager.registerDatabase(database1);
]]></programlisting>
                </example>
                <para>With the above example, the URI</para>
                <synopsis format="linespecific">xmldb:test:///db</synopsis>
                <para>selects the test database instance. Both instances should have their own data
                    and log directory as specified in the configuration files.</para>
            </section>
        </section>
        <section>
            <title>XMLDBTransformer for Cocoon</title>
            <para>eXist offers several ways to access the database from Cocoon-based applications.
                This includes access via the XMLDB pseudo-protocol, through XSP pages, and through
                the XMLDBTransformer. The XMLDBTransformer provides a simple way to query the
                database, and works in a similar way to other transformers supplied with Cocoon.
                Consult the <ulink url="cocoon.apache.org/">Cocoon documentation</ulink> for more on
                using Transformers and about their basic concepts.</para>
            <para>As with other transformers, the XMLDBTransformer listens for a limited set of tags
                that belong to the namespace <filename moreinfo="none"
                    >http://exist-db/transformer/1.0</filename>. These are
                <sgmltag>collection</sgmltag>, <sgmltag>for-each</sgmltag>,
                <sgmltag>select-node</sgmltag>, <sgmltag>current-node</sgmltag>. To examine how they
                are used, let's consider the following example (Note that the complete version of
                this example can be found at <filename moreinfo="none"
                >webapp/examples/simple2.xml</filename>):</para>
            <example>
                <title>XMLDBTransformer Example</title>
                <screen format="linespecific"><![CDATA[
<xdb:collection xmlns:xdb="http://exist-db.org/transformer/1.0"
	uri="xdb:exist:///db">
	<!-- iterate through all rdf:Description elements containing the
	     term "computer" -->
	<xdb:for-each query="//rdf:Description[dc:title &amp;= 'computer']"
		from="0" to="9" sort-by="/dc:title">
		<!-- output a book element for each entry -->
		<book>
			<!-- extract the title. There's only one title, so we use
			     select-node -->
			<title><xdb:select-node query="dc:title/text()"/></title>
			<!-- extract the creators. There's probably more than one,
			     so we use a nested for-each -->
			<xdb:for-each query="dc:creator/text()">
				<creator><xdb:current-node/></creator>
			</xdb:for-each>
		</book>
	</xdb:for-each>
</xdb:collection>
]]></screen>
            </example>
            <para>As we can see above, before you can start to query the database, you must specify
                a collection in the <sgmltag>collection</sgmltag> element, which accepts a standard
                XMLDB URI in its <option>uri</option> attribute. To process a query, you may either
                use the <sgmltag>for-each</sgmltag>, or the <sgmltag>select-node</sgmltag> tag. The
                difference is the following:</para>
            <itemizedlist>
                <listitem>
                    <para><sgmltag>for-each</sgmltag> iterates over all the results it finds for the
                        given query. The body of the statement will therefore be evaluated for each
                        query result.</para>
                </listitem>
                <listitem>
                    <para><sgmltag>select-node</sgmltag> selects the first result it finds. If the
                        result set contains more than one match, the remaining matches will be
                        ignored.</para>
                </listitem>
            </itemizedlist>
            <para>The <sgmltag>current-node</sgmltag> element is used to return the current node
                being processed in a for-each iteration to the output document. You can restrict the
                number of for-each iterations by specifying the bounds set by the
                <option>from</option> and <option>to</option> attributes. The
                <option>sort-by</option> attribute is still experimental: the query results will be
                sorted by an XPath expression. For each of the results, the XPath expression is
                evaluated and the resulting string value is used to sort the query results in
                ascending order.</para>
            <para>As shown above, it is possible to nest multiple for-each or select-node tags. The
                nested tag will be evaluated relative to the current result node. In the example
                above, the main for-each statement selects all <sgmltag>rdf:Description</sgmltag>
                fragments whose title contains the term "computer". During each iteration, we
                further process the current result fragment by using nested
                <sgmltag>for-each</sgmltag> and <sgmltag>select-node</sgmltag> tags to select the
                title and creators.</para>
            <para>Notice that the same result could be achieved by an XSLT stylesheet. However, if
                the selected fragments are rather large, post-processing with XSLT can be much
                slower, since each fragment has to be serialized and then parsed by the XSLT
                processor.</para>
            <para>The results of the XMLDBTransformer query are enclosed in the element
                    <sgmltag>result-set</sgmltag>. Attributes for this tag include the number of
                hits for the query, the XPath query processed, the query time (in milliseconds), and
                the start and end position of the retrieved records in the result set. The output of
                the XMLDBTransformer for the above fragment is shown below:</para>
            <example>
                <title>XMLDBTransformer Output</title>
                <screen format="linespecific"><![CDATA[
<xdb:result-set count="72" xpath="//rdf:Description[dc:title &= 'computer']"
	query-time="370" from="0" to="9">
	<book xdb:document-id="zit.rdf" xdb:collection="/db/library"> 
		<title> A Centennial History of the American Society of Mechanical Engineers 1880-1980 </title> 
		<creator xdb:document-id="zit.rdf" xdb:collection="/db/library"> Sinclair, Bruce </creator>
	</book>
	<!-- more books here ... -->
</xdb:result-set>
]]></screen>
            </example>

        </section>
        <section>
            <title>XML:DB Logicsheet for Cocoon</title>
            <para>Cocoon offers a powerful mechanism called XSP (eXtensible Server Pages) to write
                dynamic XML-based web pages. Similar to JSP, XSP embeds Java code in the XML pages.
                However, embedding large sections of Java code in an XML document is usually
                considered poor programming form. To support the separation of content and
                programming logic, XSP allows us to put reusable code into "logicsheets", which
                correspond to the tag libraries found in JSP. A logicsheet helps to minimize the
                amount of Java code used inside an XSP page.</para>
            <para>Version 0.8 of eXist includes a logicsheet based on the XML:DB API, which defines
                tags for all important tasks. While it is possible to write all of the XML:DB
                related code by hand, these predefined tags make the XML file more readable and
                helps users without Java experience to understand the process involved.</para>
            <para>An overview of the available XSP tags is available with the <ulink
                    url="xmldb/xsl.html">stylesheet documentation</ulink> (generated using <ulink
                    url="www.xsldoc.org">xsldoc</ulink>). In the following simple XSP example, a
                document is retrieved and displayed:</para>
            <example>
                <title>Simple XSP Page (example1.xsp)</title>
                <programlisting format="linespecific"><![CDATA[
<xsp:page xmlns:xsp="http://apache.org/xsp"
          xmlns:xdb="http://exist-db.org/xmldb/1.0"
>
<document>
    <body>
        <section title="View document">
            
        <p>Retrieving document <xsp:expr>request.getParameter("doc")</xsp:expr></p>
        
        <xdb:collection uri="xdb:exist:///db/shakespeare/plays">
            <xml-source>
                <xdb:get-document encoding="ISO-8859-1" as="xml">
                     <xdb:name>request.getParameter("doc")</xdb:name>
                </xdb:get-document>
            </xml-source>
        </xdb:collection>
        </section>
    </body>
</document>
</xsp:page>
]]></programlisting>
            </example>
            <para>The Cocoon version included with eXist is already configured to recognize the
                    <filename>xmldb</filename> namespace and associate it with the XML:DB
                logicsheet. The logicsheet is defined in <filename moreinfo="none"
                    >src/org/exist/xmldb.xsl</filename>. To use the logicsheet from our page we just
                declare the xmldb namespace (i.e. xmlns:xdb="http://exist-db.org/xmldb/1.0").</para>
            <para>The above sample code retrieves a document from the collection <filename
                    moreinfo="none">/db/shakespeare/plays</filename>. The name of the document is
                passed in the HTTP request parameter <parameter>doc</parameter>.</para>
            <para>To post-process the retrieved XML data, we set the attribute
                <parameter>as</parameter> to "xml". This indicates that the resource should be fed
                into the current Cocoon processing stream. To include the data as a string value,
                you may specify <parameter moreinfo="none">as="string"</parameter>. As a result, all
                XML markup characters will be escaped.</para>
            <para>Please note that the parameters of the logicsheet tags may be specified either as
                an attribute of an element or as a child element. If you specify a parameter as a
                child element, its content will be interpreted as a Java expression. Literal values
                should be set via an attribute. For example, the <parameter moreinfo="none"
                >xpath</parameter> parameter is specified as a Java expression, it is thus embedded
                in an <sgmltag>xdb:xpath</sgmltag> element.</para>
            <para>Finally, in order to tell Cocoon how to process this page, we have to add a new
                    <sgmltag>map:match</sgmltag>pattern to the sitemap - for example:</para>
            <example>
                <title>Cocoon Sitemap Snippet (XSP)</title>
                <programlisting format="linespecific"><![CDATA[
<map:match pattern="test.xsp">
    <map:generate type="serverpages" src="test.xsp"/>
    <map:transform src="stylesheets/doc2html-2.xsl"/>
    <map:serialize type="xhtml"/>
</map:match>
]]></programlisting>
            </example>
            <para>The next example shows how to query the database:</para>
            <example>
                <title>Querying the Database (example2.xsp)</title>
                <programlisting format="linespecific"><![CDATA[
<xsp:page xmlns:xsp="http://apache.org/xsp"
          xmlns:xdb="http://exist-db.org/xmldb/1.0"
>
    <html>
        <body>
            <h1>Find books by title</h1>
            <xdb:collection uri="xdb:exist:///db">
                <xdb:execute>
                    <xdb:xpath>
                        "document()//rdf:Description[dc:title" +
                        "&amp;='" + request.getParameter("title") + "']"
                    </xdb:xpath>
                    <p>Found <xdb:get-hit-count/> hits.</p>
                    
                    <xdb:results>
                        <pre>
                            <xdb:get-xml as="string"/>
                        </pre>
                    </xdb:results>
                </xdb:execute>
            </xdb:collection>
        </body>
    </html>
</xsp:page>
]]></programlisting>
            </example>
            <para>This XSP example page takes the HTTP request parameter
                <parameter>title</parameter> as its input and creates an XPath expression that finds
                all <sgmltag>df:Description</sgmltag> elements having a <sgmltag>dc:title</sgmltag>
                element containing the keywords entered by the user. As required by the XML:DB API,
                any action has to be enclosed in an <sgmltag>xdb:collection</sgmltag> element. The
                query is specified in the <sgmltag>xdb:xpath</sgmltag> element using a Java
                expression, which inserts the value of the request parameter
                <parameter>title</parameter> into the XPath query string.</para>
            <para>The <sgmltag>xdb:results</sgmltag> element will iterate through the generated
                result set, inserting each resource into the page by calling
                <sgmltag>xdb:get-xml</sgmltag>. In this case, <sgmltag>xdb:get-xml</sgmltag> inserts
                the resource contents as a string, which means that all XML markup is
            escaped.</para>
        </section>

        <section>
            <title>Using the XML-RPC API</title>
            <para>XML-RPC (XML Remote Procedural Call) provides a simple way to call remote
                procedures from a wide variety of programming languages. eXist's XML-RPC API makes
                it easy to access eXist from other applications, CGI scripts, PHP, JSP and more. For
                more information on XML-RPC see <ulink url="http://www.xmlrpc.org"
                >www.xmlrpc.org</ulink>. For the Java server, eXist uses the XML-RPC library created
                by Hannes Wallnoefer which recently has moved to Apache (see: <ulink
                    url="http://xml.apache.org/xmlrpc">http://xml.apache.org/xmlrpc</ulink>). Perl
                examples use the RPC::XML package, which should be available at every CPAN mirror
                (see <ulink url="http://www.cpan.org">CPAN</ulink>).</para>
            <para>The following is a small example, which shows how to talk to eXist from Java using
                the Apache XML-RPC library. This example can be found in <filename moreinfo="none"
                    >samples/org/exist/examples/xmldb/Retrieve.java</filename>.</para>
            <example>
                <title>Retrieving a document from eXist</title>
                <programlisting format="linespecific"> <![CDATA[
public class Retrieve {

protected final static String uri = 
    "http://localhost:8080/exist/xmlrpc";

protected static void usage() {
    System.out.println( "usage: org.exist.examples.xmlrpc.Retrieve " +
        "path-to-document" );
    System.exit( 0 );
}

public static void main( String args[] ) throws Exception {
    if ( args.length < 1 ) {
        usage();
    }
    XmlRpc.setEncoding("UTF-8");
    XmlRpcClient xmlrpc = new XmlRpcClient( uri );
    Hashtable options = new Hashtable();
    options.put("indent", "yes");
    options.put("encoding", "UTF-8");
    options.put("expand-xincludes", "yes");
    options.put("highlight-matches", "elements");
    
    Vector params = new Vector();
    params.addElement( args[0] ); 
    params.addElement( options );
    String xml = (String)
        xmlrpc.execute( "getDocumentAsString", params );
    System.out.println( xml );
}
}
]]></programlisting>
            </example>
            <para>As shown above, the <methodname>execute</methodname> method of
                    <classname>XmlRpcClient</classname> expects as its parameters a method (passed
                as a string) to call on the server and a Vector of parameters to pass to this
                executed method. In this example, the method
                <methodname>getDocumentAsString</methodname> is called as the first parameter, and a
                Vector <parameter>params</parameter>. Various output properties can also be set
                through the hashtable argument (see the method description below). Since all
                parameters are passed in a Vector, they are necessarily Java objects.</para>

            <para>XML-RPC messages (requests and responses sent between the server and client) are
                themselves XML documents. In some cases, these documents may use a character
                encoding which is in conflict with the encoding of the document we would like to
                receive. It is thus important to set the <emphasis>transport</emphasis> encoding to
                    <option>UTF-8</option> as shown in the above example. However, conflicts may
                persist depending on which client library is used. To avoid such conflicts, eXist
                provides alternative declarations for selected methods, which expect string
                parameters as byte arrays. The XML-RPC library will send them as binary data (using
                Base64 encoding for transport). With this approach, document encodings are preserved
                regardless of the character encoding used by the XML-RPC transport layer.</para>
            <note>
                <para>Please note that the XML-RPC API uses <option>int</option> to encode booleans.
                    This is because some clients do not correctly pass boolean parameters.</para>
            </note>
            <para>Querying is as easy using XML-RPC. The following example:</para>
            <example>
                <title>Sending a Query to eXist (XML-RPC)</title>
                <programlisting format="linespecific">
<![CDATA[#!/usr/bin/perl
use RPC::XML;
use RPC::XML::Client;

$query = <<END;
for \$speech in //SPEECH[LINE &= 'tear*']
order by \$speech/SPEAKER[1]
return
    \$speech
END

$URL = "http://guest:guest\@localhost:8080/exist/xmlrpc";
print "connecting to $URL...\n";
$client = new RPC::XML::Client $URL;

# Output options
$options = RPC::XML::struct->new(
    'indent' => 'yes', 
    'encoding' => 'UTF-8',
    'highlight-matches' => 'none');

$req = RPC::XML::request->new("query", $query, 20, 1, $options);
$response = $client->send_request($req);
if($response->is_fault) {
    die "An error occurred: " . $response->string . "\n";
}
print $response->value;
]]></programlisting>
            </example>
            <para>You will find the source code of this example in
                    <filename>samples/xmlrpc/search2.pl</filename>. It uses the simple query method,
                which executes the query and returns a document containing the specified number of
                results. However, the result set is not cached on the server. </para>
            <para>The following example calls the <methodname>executeQuery</methodname> method,
                which returns a unique session id. In this case, the actual results are cached on
                the server and can be retrieved using the <methodname>retrieve</methodname> method. </para>
            <example>
                <title>Another Query Examplet (XML-RPC)</title>
                <programlisting format="linespecific"><![CDATA[use RPC::XML;
#!/usr/bin/perl

use RPC::XML;
use RPC::XML::Client;

# Execute an XQuery through XML-RPC. The query is passed
# to the "executeQuery" method, which returns a handle to
# the created result set. The handle can then be used to
# retrieve results.

$query = <<END;
for \$speech in //SPEECH[LINE &= 'corrupt*']
order by \$speech/SPEAKER[1]
return
    \$speech
END

$URL = "http://guest:guest\@localhost:8080/exist/xmlrpc";
print "connecting to $URL...\n";
$client = new RPC::XML::Client $URL;

# Execute the query. The method call returns a handle
# to the created result set.
$req = RPC::XML::request->new("executeQuery", 
    RPC::XML::base64->new($query), 
	"UTF-8");
$resp = process($req);
$result_id = $resp->value;

# Get the number of hits in the result set
$req = RPC::XML::request->new("getHits", $result_id);
$resp = process($req);
$hits = $resp->value;
print "Found $hits hits.\n";

# Output options
$options = RPC::XML::struct->new(
    'indent' => 'no', 
    'encoding' => 'UTF-8');
# Retrieve query results 1 to 10
for($i = 1; $i < 10 && $i < $hits; $i++) {
    $req = RPC::XML::request->new("retrieve", $result_id, $i, $options);
    $resp = process($req);
    print $resp->value . "\n";
}

# Send the request and check for errors
sub process {
    my($request) = @_;
    $response = $client->send_request($request);
    if($response->is_fault) {
        die "An error occurred: " . $response->string . "\n";
    }
    return $response;
}
]]></programlisting>
            </example>
        </section>
        <section>
            <title>XML-RPC: Available Methods</title>
            <para>This section gives you an overview of the methods implemented by the eXist XML-RPC
                server. Only the most common methods are presented here. For a complete list see the
                Java interface <ulink url="api/org/exist/xmlrpc/RpcAPI.html">RpcAPI.java</ulink>.
                Note that the method signatures are presented below using Java data types. Also note
                that some methods like <methodname>getDocument()</methodname> and
                    <methodname>retrieve()</methodname> accept a struct to specify optional output
                properties.</para>

            <para>In general, the following optional fields for methods are supported:</para>

            <variablelist>

                <varlistentry>
                    <term>indent</term>
                    <listitem>
                        <para>Returns indented pretty-print XML. [<option>yes | no</option>]</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>encoding</term>
                    <listitem>
                        <para>Specifies the character encoding used for the output. If the method
                            returns a string, only the XML declaration will be modified
                        accordingly.</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>omit-xml-declaration</term>
                    <listitem>
                        <para>Add XML declaration to the head of the document. [<option>yes |
                            no</option>]</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>expand-xincludes</term>
                    <listitem>
                        <para>Expand XInclude elements. [<option>yes | no</option>]</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>process-xsl-pi</term>
                    <listitem>
                        <para>Specifying "yes": XSL processing instructions in the document will be
                            processed and the corresponding stylesheet applied to the output.
                                [<option>yes | no</option>]</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>highlight-matches</term>
                    <listitem>
                        <para>Database adds special tags to highlight the strings in the text that
                            have triggered a fulltext match. Set to "<option>elements</option>" to
                            highlight matches in element values, "<option>attributes</option>" for
                            attribute values or "<option>both</option>" for both elements and
                            attributes.</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>stylesheet</term>
                    <listitem>
                        <para>Use this parameter to specify an XSL stylesheet which should be
                            applied to the output. If the parameter contains a relative path, the
                            stylesheet will be loaded from the database.</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>stylesheet-param.key1 ... stylesheet-param.key2</term>
                    <listitem>
                        <para>If a stylesheet has been specified with <option>stylesheet</option>,
                            you can also pass it parameters. Stylesheet parameters are recognized if
                            they start with the prefix <option>stylesheet-param.</option>, followed
                            by the name of the parameter. The leading
                            "<option>stylesheet-param.</option>" string will be removed before the
                            parameter is passed to the stylesheet.</para>
                    </listitem>
                </varlistentry>
            </variablelist>

            <section>
                <title>Retrieving documents</title>
                <itemizedlist>
                    <listitem>
                        <synopsis format="linespecific">byte[] getDocument(String name, Hashtable parameters)</synopsis>
                        <synopsis>String getDocumentAsString(String name, Hashtable parameters)</synopsis>
                        <para>Retrieves a document from the database.</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>name</term>
                                <listitem>
                                    <para>Path of the document to be retrieved (e.g.
                                            <filename>/db/shakespeare/plays/r_and_j.xml</filename>).</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>parameters</term>
                                <listitem>
                                    <para>A struct containing <option>key=value</option> pairs for
                                        configuring the output.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                    <listitem>
                        <synopsis format="linespecific">Hashtable getDocumentData(String name, Hashtable parameters)</synopsis>
                        <synopsis>Hashtable getNextChunk(String handle, Int offset)</synopsis>
                        <para>To retrieve a document from the database, but limit the number of
                            bytes transmitted in one chunk to avoid memory shortage on the server,
                            use the following:</para>
                        <para><methodname>getDocumentData()</methodname> returns a struct containing
                            the following fields: <parameter>data</parameter>,
                            <parameter>handle</parameter>, <parameter>offset</parameter>.
                                <parameter>data</parameter> contains the document's data (as
                                <option>byte[]</option>) or the first chunk of data if the document
                            size exceeds the predefined internal limit.
                            <parameter>handle</parameter> and <parameter>offset</parameter> can be
                            passed to <methodname>getNextChunk()</methodname> to retrieve the
                            remaining data chunks.</para>
                        <para>If <parameter>offset</parameter> is 0, no more chunks are available
                            and all of the data is already contained in the
                            <parameter>data</parameter> field. Otherwise, further chunks can be
                            retrieved by passing the handle and the offset (as returned by the last
                            call) to <methodname>getNextChunk()</methodname>. Once the last chunk is
                            read, <parameter>offset</parameter> will be 0 and the handle becomes
                            invalid.</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>name</term>
                                <listitem>
                                    <para>Path of the document to be retrieved (e.g.
                                            <filename>/db/shakespeare/plays/r_and_j.xml</filename>).</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>parameters</term>
                                <listitem>
                                    <para>A struct containing <option>key=value</option> pairs to
                                        configure the output.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>handle</term>
                                <listitem>
                                    <para>The handle returned by the call to
                                            <methodname>getDocumentData()</methodname>. This
                                        identifies a temporary file on the server to be read.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>offset</term>
                                <listitem>
                                    <para>The data offset in the document at which the next chunk in
                                        the sequence will be read.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                </itemizedlist>
            </section>
            <section>
                <title>Storing Documents</title>
                <itemizedlist>
                    <listitem>
                        <synopsis format="linespecific">boolean parse(byte[] xml, String docName, int overwrite)</synopsis>
                        <synopsis format="linespecific">boolean parse(byte[] xml, String docName)</synopsis>
                        <para>Inserts a new document into the database or replace an existing one:</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>xml</term>
                                <listitem>
                                    <para>XML content of this document as a UTF-8 encoded byte
                                        array.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>docName</term>
                                <listitem>
                                    <para>Path to the database location where the new document is to
                                        be stored.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>overwrite</term>
                                <listitem>
                                    <para>Set this value to &gt; 0 to automatically replace an
                                        existing document at the same location.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                    <listitem>
                        <synopsis>String upload(byte[] chunk, int length)</synopsis>
                        <synopsis>String upload(String file, byte[] chunk, int length)</synopsis>
                        <synopsis>boolean parseLocal(String localFile, String docName, boolean replace)</synopsis>
                        <para>Uploads an entire document on to the database before parsing it.</para>
                        <para>While the parse method receives the document as a large single chunk,
                            the upload method allows you to upload the whole document to the server
                            before parsing. This way, <emphasis>out-of-memory</emphasis> exceptions
                            can be avoided, since the document is not entirely kept in the main
                            memory. To identify the file on the server, upload returns an identifier
                            string. After uploading all chunks, you can call
                            <methodname>parseLocal</methodname> and pass it this identifier string
                            as the first argument.</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>file</term>
                                <listitem>
                                    <para>The name of the file to which the uploaded chunk is
                                        appended. This is the name of a temporary file on the
                                        server. Use the two-argument version of upload for the first
                                        chunk. The method creates a temporary file and returns its
                                        name. On subsequent calls to this chunk, pass this
                                    name.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>chunk</term>
                                <listitem>
                                    <para>A byte array containing the data to be appended.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>length</term>
                                <listitem>
                                    <para>Defines the number of bytes to be read from chunk.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>localFile</term>
                                <listitem>
                                    <para>The name of the local file on the server that is to be
                                        stored in the database. This should be the same as the name
                                        returned by <methodname>upload</methodname>.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>docName</term>
                                <listitem>
                                    <para>The full path specifying the location where the document
                                        should be stored in the database.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>replace</term>
                                <listitem>
                                    <para>Set this to <option>true</option> if an existing document
                                        with the same name should be automatically
                                    overwritten.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                </itemizedlist>
            </section>
            <section>
                <title>Creating a Collection</title>
                <itemizedlist>
                    <listitem>
                        <synopsis format="linespecific">boolean createCollection(String name)</synopsis>
                        <para>Creates a new collection</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>name</term>
                                <listitem>
                                    <para>Path to the new collection.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                </itemizedlist>
            </section>
            <section>
                <title>Removing Documents or Collections</title>
                <itemizedlist>
                    <listitem>
                        <synopsis format="linespecific">boolean remove(String docName)</synopsis>
                        <para>Removes a document from the database.</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>docName</term>
                                <listitem>
                                    <para>The full path to the database document.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                    <listitem>
                        <synopsis format="linespecific">boolean removeCollection( String collection)</synopsis>
                        <para>Removes a collection from the database (including all of its documents
                            and sub-collections).</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>collection</term>
                                <listitem>
                                    <para>The full path to the collection.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                </itemizedlist>
            </section>
            <section>
                <title>Querying</title>
                <itemizedlist>
                    <listitem>
                        <synopsis format="linespecific">int executeQuery(String xquery)</synopsis>
                        <synopsis format="linespecific">int executeQuery(byte[] xquery)</synopsis>
                        <synopsis format="linespecific">int executeQuery(byte[] xquery, String encoding)</synopsis>
                        <para>Executes an XQuery and returns a reference identifier to the generated
                            result set. This reference can be used later to retrieve results.</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>xquery</term>
                                <listitem>
                                    <para>A valid XQuery expression.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>encoding</term>
                                <listitem>
                                    <para>The character encoding used for the query string.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                    <listitem>
                        <synopsis format="linespecific">Hashtable querySummary(int result-Id)</synopsis>
                        <para>Returns a summary of query results for the result-set referenced by
                                <parameter>result-Id</parameter>.</para>
                        <para>The <parameter>result-Id</parameter> value is taken from a previous
                            call to <methodname>executeQuery</methodname> (See above). The
                                <methodname>querySummary</methodname> method returns a struct with
                            the following fields: <parameter>queryTime</parameter>,
                            <parameter>hits</parameter>, <parameter>documents</parameter>,
                                <parameter>doctype</parameter>.</para>
                        <para><parameter>queryTime</parameter> and <parameter>hits</parameter> are
                            integer values that describe the processing time in milliseconds for the
                            query execution and the number of hits in the result-set respectively.
                            The field <parameter>documents</parameter> is an array of an array (i.e.
                                <option>Object[][3]</option>) that represents a table in which each
                            row identifies one document. The first field in each row contains the
                                <parameter>document-id</parameter> (integer value). The second has
                            the document's name as a string value. The third contains the number of
                            hits found in this document (integer value).</para>
                        <para>The<parameter>doctype</parameter> field is also an array of an array
                            (Object[][2]) that contains the <option>doctype</option> public
                            identifier and the number of hits found for this
                            <option>doctype</option> in each row.</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>resultId</term>
                                <listitem>
                                    <para>Reference to a result-set as returned by a previous call
                                        to <methodname>executeQuery</methodname>.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                    <listitem>
                        <synopsis format="linespecific">byte[] retrieve(int resultId, int pos, Hashtable parameters)</synopsis>
                        <para>Retrieves a single result-fragment from the result-set referenced by
                                <parameter>resultId</parameter>. The result-fragment is identified
                            by its position in the result-set, which is passed in the parameter
                                <parameter>pos</parameter>.</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>resultId</term>
                                <listitem>
                                    <para>Reference to a result-set as returned by a previous call
                                        to <methodname>executeQuery</methodname>.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>pos</term>
                                <listitem>
                                    <para>The position of the item in the result-sequence, starting
                                        at 0.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>parameters</term>
                                <listitem>
                                    <para>A struct containing <option>key=value</option> pairs to
                                        configure the output.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                    <listitem>
                        <synopsis format="linespecific">int getHits(int resultId)</synopsis>
                        <para>Get the number of hits in the result-set identified by
                                <parameter>resultId</parameter>.</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>resultId</term>
                                <listitem>
                                    <para>Reference to a result-set as returned by a previous call
                                        to <methodname>executeQuery</methodname>.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                    <listitem>
                        <synopsis format="linespecific">String query(byte[] xquery, int howmany, int start, Hashtable parameters)</synopsis>
                        <para>Executes an XQuery expression and returns a specified subset of the
                            results. This method will directly return a subset of the
                            result-sequence, starting at <parameter>start</parameter>, as a new XML
                            document. The number of results returned is determined by parameter
                                <parameter>howmany</parameter>. The result-set will be deleted on
                            the server, so later calls to this method will again execute the query.</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>xquery</term>
                                <listitem>
                                    <para>An XQuery expression.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>start</term>
                                <listitem>
                                    <para>The position of the first item to be retrieved from the
                                        result-sequence.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>howmany</term>
                                <listitem>
                                    <para>The maximum number of items to retrieve.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>parameters</term>
                                <listitem>
                                    <para>A struct containing <option>key=value</option> pairs to
                                        configure the output.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                    <listitem>
                        <synopsis>void releaseQueryResult(int resultId)</synopsis>
                        <para>Forces the result-set identified by its result id to be released on
                            the server.</para>
                    </listitem>
                </itemizedlist>
            </section>
            <section>
                <title>Retrieving Information on Collections and Documents</title>
                <itemizedlist>
                    <listitem>
                        <synopsis format="linespecific">Hashtable describeCollection(String collection)</synopsis>
                        <para>Returns a struct describing a specified collection.</para>
                        <para>The returned struct has the following fields:
                            <parameter>name</parameter> (the collection path),
                            <parameter>owner</parameter> (identifies the collection owner),
                                <parameter>group</parameter> (identifies the group that owns the
                            collection), <parameter>created</parameter> (the creation date of the
                            collection expressed as a long value),
                            <parameter>permissions</parameter> (the active permissions that apply to
                            the collection as an integer value).</para>

                        <para><parameter>collections</parameter> is an array listing the names of
                            available sub-collections in this collection.</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>collection</term>
                                <listitem>
                                    <para>The full path to the collection.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                    <listitem>
                        <synopsis format="linespecific">Hashtable describeResource(String resource)</synopsis>
                        <para>Returns a struct describing a specified resource.</para>
                        <para>The returned struct has the following fields:
                            <parameter>name</parameter> (the collection path),
                            <parameter>owner</parameter> (identifies the collection owner),
                                <parameter>group</parameter> (identifies the group that owns the
                            collection), <parameter>created</parameter> (the creation date of the
                            collection expressed as a long value),
                            <parameter>permissions</parameter> (the active permissions that apply to
                            the collection as an integer value), <parameter>type</parameter> (either
                                <option>XMLResource</option> for XML documents or
                                <option>BinaryResource</option> for binary files),
                                <parameter>content-length</parameter> (the estimated size of the
                            resource in bytes). The <parameter>content-length</parameter> is based
                            on the number of pages occupied by the resource in the DOM storage. For
                            binary resources, the value will always be 0.</para>
                    </listitem>
                    <listitem>
                        <synopsis format="linespecific">Hashtable getCollectionDesc(String collection)</synopsis>
                        <para>Returns a struct describing a collection.</para>
                        <para>The returned struct has the following fields:
                            <parameter>name</parameter> (the collection path),
                            <parameter>owner</parameter> (identifies the collection owner),
                                <parameter>group</parameter> (identifies the group that owns the
                            collection), <parameter>created</parameter> (the creation date of the
                            collection expressed as a long value),
                            <parameter>permissions</parameter> (the active permissions that apply to
                            the collection as an integer value).</para>

                        <para><parameter>collections</parameter> is an array listing the names of
                            available sub-collections in this collection.</para>

                        <para><parameter>documents</parameter> is an array listing information on
                            all of the documents in this collection. Each item in the array is a
                            struct with the following fields: name, owner, group, permissions, type.
                            The type field contains a string describing the type of the resource:
                            either <option>XMLResource</option>or <option>BinaryResource</option>. </para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>collection</term>
                                <listitem>
                                    <para>The full path to the collection.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                </itemizedlist>
            </section>
            <section>
                <title>XUpdate</title>
                <itemizedlist>
                    <listitem>
                        <synopsis>int xupdate(String collectionName, byte[] xupdate)</synopsis>
                        <synopsis>int xupdateResource(String documentName, byte[] xupdate)</synopsis>
                        <para>Applies a set of XUpdate modifications to a collection or document.</para>
                        <variablelist>
                            <varlistentry>
                                <term>collectionName</term>
                                <listitem>
                                    <para>The full path to the collection to which the XUpdate
                                        modifications should be applied.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>documentName</term>
                                <listitem>
                                    <para>The full path to the document to which the XUpdate
                                        modifications should be applied.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>xupdate</term>
                                <listitem>
                                    <para>The XUpdate document containing the modifications. This
                                        should be send as an <option>UTF-8</option> encoded binary
                                        array.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                </itemizedlist>
            </section>
            <section>
                <title>Managing Users and Permissions</title>
                <itemizedlist>
                    <listitem>
                        <synopsis>boolean setUser(String name, String passwd, Vector groups)</synopsis>
                        <synopsis>boolean setUser(String name, String passwd, Vector groups, String home)</synopsis>
                        <para>Modifies or creates a database user.</para>
                        <variablelist>
                            <title>Parameters:</title>
                            <varlistentry>
                                <term>name</term>
                                <listitem>
                                    <para>Username value.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>passwd</term>
                                <listitem>
                                    <para>The MD5 encoded password for the user. Sending a readable
                                        password through the network could pose a security problem,
                                        and thus passwords are only sent and stored with MD5
                                        encoding.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>groups</term>
                                <listitem>
                                    <para>A vector of groups assigned to the user. The first group
                                        in the vector will become the user's primary group.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>home</term>
                                <listitem>
                                    <para>An optional setting for the user's home collection path.
                                        The collection will be created if it does not exist, and
                                        provides the user with full access.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                    <listitem>
                        <synopsis>boolean setPermissions(String resource, String permissions)</synopsis>
                        <synopsis>boolean setPermissions(String resource, int permissions)</synopsis>
                        <synopsis>boolean setPermissions(String resource, String owner, String ownerGroup, String permissions)</synopsis>
                        <synopsis>boolean setPermissions(String resource, String owner, String ownerGroup, int permissions)</synopsis>
                        <para>Sets the permissions assigned to a given collection or document.</para>
                        <variablelist>
                            <varlistentry>
                                <term>resource</term>
                                <listitem>
                                    <para>The full path to the collection or document on which the
                                        specified permissions will be set. The method first checks
                                        if the specified path points to a collection or
                                    resource.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>owner</term>
                                <listitem>
                                    <para>The name of the user owning this resource.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>ownerGroup</term>
                                <listitem>
                                    <para>The name of the group owning this resource.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>permissions</term>
                                <listitem>
                                    <para>The permissions assigned to the resource, which can be
                                        specified either as an integer value constructed using the
                                            <ulink url="api/org/exist/security/Permission"
                                            >Permission</ulink> class, or using a modification
                                        string. The bit encoding of the integer value corresponds to
                                        Unix conventions. The modification string has the following
                                        syntax:</para>
                                    <synopsis>[user|group|other]=[+|-][read|write|update][, ...]</synopsis>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                    <listitem>
                        <synopsis>Hashtable getPermissions(String resource)</synopsis>
                        <para>Returns the active permissions for the specified document or
                            collection.</para>
                        <para>The returned struct has the following fields:
                            <parameter>name</parameter> (the collection path),
                            <parameter>owner</parameter> (identifies the collection owner),
                                <parameter>group</parameter> (identifies the group that owns the
                            collection), <parameter>created</parameter> (the creation date of the
                            collection expressed as a long value),
                            <parameter>permissions</parameter> (the active permissions that apply to
                            the collection as an integer value).</para>
                    </listitem>
                    <listitem>
                        <synopsis>boolean removeUser(String name)</synopsis>
                        <para>Removes the identified user.</para>
                    </listitem>
                    <listitem>
                        <synopsis>Hashtable getUser(String name)</synopsis>
                        <para>Returns a struct describing the user identified by its name.</para>
                        <para>The returned struct has the following fields:
                            <parameter>name</parameter> (the collection path),
                            <parameter>home</parameter> (identifies the user's home directory),
                                <parameter>groups</parameter> (an array specifying all groups to
                            which the user belongs).</para>
                    </listitem>
                    <listitem>
                        <synopsis>Vector getUsers()</synopsis>
                        <para>Returns a list of all users currently known to the system.</para>
                        <para>Each user in the list is described by the same struct returned by the
                                <methodname>getUser()</methodname> method.</para>
                    </listitem>
                    <listitem>
                        <synopsis>Vector getGroups()</synopsis>
                        <para>Returns a list of all group names (as string values) currently
                            defined.</para>
                    </listitem>
                </itemizedlist>
            </section>
            <section>
                <title>Access to the Index Contents</title>
                <para>The following methods provide access to eXist's internal index structure.</para>
                <itemizedlist>
                    <listitem>
                        <synopsis>Vector getIndexedElements(String collectionName, boolean inclusive)</synopsis>
                        <para>Returns a list (i.e. array[][4]) of all indexed element names for the
                            specified collection.</para>
                        <para>For each element, an array of four items is returned:</para>
                        <orderedlist>
                            <listitem>
                                <para>name of the element</para>
                            </listitem>
                            <listitem>
                                <para>optional namespace URI</para>
                            </listitem>
                            <listitem>
                                <para>optional namespace prefix</para>
                            </listitem>
                            <listitem>
                                <para>number of occurrences of this element as an integer
                                value</para>
                            </listitem>
                        </orderedlist>
                        <variablelist>
                            <varlistentry>
                                <term>collectionName</term>
                                <listitem>
                                    <para>The full path to the collection.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>inclusive</term>
                                <listitem>
                                    <para>If set to <option>true</option>, the subcollections of the
                                        specified collection will be included into the
                                    result.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                    <listitem>
                        <synopsis>Vector scanIndexTerms(String collectionName, String start, String end, boolean inclusive)</synopsis>
                        <para>Return a list (array[][2]) of all index terms contained in the
                            specified collection.</para>
                        <para>For each term, an array with two items is returned:</para>
                        <orderedlist>
                            <listitem>
                                <para>the term itself</para>
                            </listitem>
                            <listitem>
                                <para>number occurrences of the term in the specified
                                collection</para>
                            </listitem>
                        </orderedlist>
                        <variablelist>
                            <varlistentry>
                                <term>collectionName</term>
                                <listitem>
                                    <para>The full path to the collection.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>start</term>
                                <listitem>
                                    <para>The start position for the returned range expressed as a
                                        string value. Returned index terms are positioned after the
                                        start position in ascending, alphabetical order.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>end</term>
                                <listitem>
                                    <para>The end position for the returned range expressed as a
                                        string value. Returned index terms are positioned before the
                                        end position in ascending, alphabetical order.</para>
                                </listitem>
                            </varlistentry>
                            <varlistentry>
                                <term>inclusive</term>
                                <listitem>
                                    <para>If set to<option>true</option>, subcollections of the
                                        specified collection will be included into the
                                    result.</para>
                                </listitem>
                            </varlistentry>
                        </variablelist>
                    </listitem>
                </itemizedlist>
            </section>
            <section>
                <title>Other Methods</title>
                <itemizedlist>
                    <listitem>
                        <synopsis>boolean shutdown()</synopsis>
                        <para>Shuts down the database engine. All dirty pages are written to
                        disk.</para>
                    </listitem>
                    <listitem>
                        <synopsis>boolean sync()</synopsis>
                        <para>Causes the database to write all dirty pages to disk.</para>
                    </listitem>
                </itemizedlist>
            </section>
        </section>
        <section>
            <title>SOAP</title>
            <para>Beginning with version 0.8, eXist provides a SOAP interface as an alternative to
                XML-RPC. Programming with SOAP is slightly more convenient than XML-RPC. While you
                have to write XML-RPC method calls by hand, most SOAP tools will automatically
                create the low-level code from a given WSDL service description. Also fewer methods
                are needed to exploit the same functionality. On the other hand, SOAP toolkits tend
                to be complex.</para>
            <para>eXist uses the Axis SOAP toolkit from Apache, which runs as a servlet. The Tomcat
                webserver shipped with eXist has been configured to start Axis automatically, and
                will listen on port 8080: <filename>http://localhost:8080/exist/services</filename>.
                Note however that SOAP is not available in the stand-alone server.</para>
            <para>The interface has been tested using various clients, including Perl (SOAP::Lite)
                and the Microsoft .NET framework. The client stubs needed to access the SOAP
                interface from Java have been automatically generated by Axis and are included in
                the distribution.</para>
            <para>eXist provides two web services: one that contains methods to query the server and
                retrieve documents, and a second for storing and removing documents and collections.
                The first will by default listen on:</para>
            <synopsis format="linespecific">http://localhost:8080/exist/services/Query</synopsis>
            <para>while the second is available on:</para>
            <synopsis format="linespecific">http://localhost:8080/exist/services/Admin</synopsis>
            <para>Both services are described in the Java docs regarding their interfaces. Visit:
                    <ulink url="api/org/exist/soap/Query.html">org.exist.soap.Query</ulink> and
                    <ulink url="api/org/exist/soap/Admin.html">org.exist.soap.Admin</ulink> for more
                information.</para>
            <para>The following SOAP example (available at:
                    <filename>samples/org/exist/examples/soap/GetDocument.java</filename>)
                demonstrates how to retrieve a document from the database:</para>
            <example>
                <title>Retrieving a document (SOAP)</title>
                <programlisting format="linespecific"><![CDATA[
package org.exist.examples.soap;

import org.exist.soap.Query;
import org.exist.soap.QueryService;
import org.exist.soap.QueryServiceLocator;

public class GetDocument {

    public static void main( String[] args ) throws Exception {
        QueryService service = new QueryServiceLocator();
        Query query = service.getQuery();
		String session = query.connect("guest", "guest");
        
		byte[] data = query.getResourceData(session, 
			"/db/shakespeare/plays/hamlet.xml",
			true, false, false);
		System.out.println(new String(data, "UTF-8"));
		query.disconnect(session);
    }
}
        ]]></programlisting>
            </example>
            <para>In this example, the <classname>Query</classname> client stub class has been
                automatically generated by the WSDL service description, and has methods for each of
                the operations defined in WSDL. You will find the web service description file
                    <filename moreinfo="none">query.wsdl</filename> in directory <filename
                    moreinfo="none">src/org/exist/soap</filename>. You may also get the WSDL
                directly from the server by pointing your web browser to <ulink
                    url="http://localhost:8080/exist/services/Query?WSDL">
                    http://localhost:8080/exist/services/Query?WSDL</ulink>.</para>
            <para>To use the services provided, the client first has to establish a connection with
                the database. This is done by calling <methodname>connect()</methodname> with a
                valid user id and password. <methodname>connect()</methodname> returns a session id,
                which can then be passed to any subsequent method calls.</para>
            <para>To retrieve a resource we simply call
                <methodname>Query.getResource()</methodname>. And to release the current session,
                the method <methodname>Query.disconnect()</methodname> is called. Otherwise the
                session will remain valid for at least 60 minutes.</para>
        </section>
        <section>
            <title>Appendix: Required Libraries</title>
            <para>eXist consists of three jar-files:</para>
            <variablelist>
                <varlistentry>
                    <term>exist.jar</term>
                    <listitem>
                        <para>The core classes of eXist.</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>start.jar</term>
                    <listitem>
                        <para>The bootstrap loader used to startup the database and client
                            applications. This library loads all other required jars.</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>exist-optional.jar</term>
                    <listitem>
                        <para>Optional components for eXist including Cocoon supported classes, SOAP
                            interfaces, Ant tasks, and the HTTP request module for XQuery. This jar
                            is only required if eXist is running in a Cocoon framework or when using
                            Ant.</para>
                    </listitem>
                </varlistentry>
            </variablelist>
            <para>The <filename>lib</filename> directory contains three subdirectories:</para>
            <orderedlist>
                <listitem>
                    <para>
                        <filename>core</filename>
                    </para>
                </listitem>
                <listitem>
                    <para>
                        <filename>optional</filename>
                    </para>
                </listitem>
                <listitem>
                    <para>
                        <filename>endorsed</filename>
                    </para>
                </listitem>
            </orderedlist>
            <para>Since these jar-files are required by the database core, they reside in the
                    <filename moreinfo="none">lib/core</filename> directory. However, you will not
                need all of them if you intend to use eXist as an embedded database in your own
                application. At an absolute minimum, you must include the following jars:</para>
            <variablelist>
                <varlistentry>
                    <term>xmldb.jar</term>
                    <listitem>
                        <para>Defines the common interfaces for the XML:DB API.</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>antlr-x.x.x.jar</term>
                    <listitem>
                        <para>The ANTLR parser generator used by the XQuery engine.</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>log4j.jar</term>
                    <listitem>
                        <para>Provides the logging facility.</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>commons-pool-x.x.jar</term>
                    <listitem>
                        <para>Provides various object pool implementations.</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>xml-commons-resolver-x.x.jar</term>
                    <listitem>
                        <para>A library for resolving XML external entities from the catalogue.
                            files.</para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>xmlrpc-x.x-patched.jar</term>
                    <listitem>
                        <para>XMLRPC protocol support. This library has been patched to handle the
                            full unicode character range. NOTE: You should include this library even
                            if you don't intend to connect to a remote database instance. The XML:DB
                            driver references this library and most Java machines will show a
                            runtime error if it is missing.</para>
                    </listitem>
                </varlistentry>
            </variablelist>
            <para>The other jars in <filename>lib/core</filename> are support libraries for the
                command-line client (i.e. <filename>excalibur-cli-x.x.jar</filename>,
                    <filename>libreadline-java.jar</filename>, and
                <filename>jEdit-syntax.jar</filename>). These are not required to run your own
                application.</para>
            <para>The jar files in <filename>lib/optional</filename> are only required for Cocoon
                (most of them are distributed with Cocoon) and Axis-SOAP.</para>
            <para>The <filename moreinfo="none">lib/endorsed</filename> directory furthermore plays
                a special role: the 1.4.x Java releases come with their own XML support libraries,
                including Xalan for XSLT processing, an XML parser, and the standard Java interfaces
                for SAX and DOM. Unfortunately, we have found that some features of eXist in
                combination with Cocoon will not work properly with the wrong version of Xalan (in
                particular, XSP pages occasionally fail to compile). To ensure that the correct
                versions are available, we have included these versions of Xerces and Xalan, plus
                the standard interfaces used by both of them.</para>
            <para>You can use Java's endorsed library loading mechanism to ensure that the correct
                XML support libraries are loaded. Specifying the
                    <option>-Djava.endorsed.dirs=lib/endorsed</option> system property on the Java
                command line will force the JVM to prefer any library it finds in the endorsed
                directory over its own system libraries. Copying the jars into <filename
                    moreinfo="none">$JAVA_HOME/jre/lib/endorsed</filename> will do the same thing.
                Note that the batch and shell scripts included with eXist all set the
                    <option>java.endorsed.dirs</option> system property to point to <filename
                    moreinfo="none">lib/endorsed</filename>.</para>
            <para/>
        </section>
    </chapter>
</book>
