XROOTD(1)

NAME
xrootd - The eXtended ROOT file server daemon

SYNOPSIS
xrootd [options] [exported directories]

DESCRIPTION
This manual page documents briefly the xrootd program.

xrootd is a ROOT remote file server daemon.

STARTING XROOTD
The simplest possible setup is one server and one or more clients:

                           +-- kan001 -----------+
    +-----------+        +--------+  +---------+ |
    | Client    |------->| Server |  | Storage | |
    | TXNetFile |        | xrootd |--|  /data  |-+
    +-----------+        +--------+  +---------+

As with the original rootd(1) server, this is relatively easy to setup. You can start it as:

    kan001> xrootd

Unlike the rootdserver(1), this doesn't immediately go into background, but in fact produces some amount of logging information.

Using some xrootd client application, you should be able to access a file on the server using the same syntax as with TNetFile/rootd, for example:

    root://kan001//tmp/mydatafile.root

From a client-side application such as a ROOT interactive session, you should be able to open the file and perform operations on it:

    root [1] myfile = TFile::Open("root://kan001//tmp/mydatafile.root");
    root [2] myfile->ls();
    XTNetFile**             root://kan001//tmp/mydatafile.root
     XTNetFile*             root://kan001//tmp/mydatafile.root
      KEY: TF1      fooHist;1       gaus
      KEY: TF1      barHist;1       gaus
    root [3] myfile->Close();
    root [4] .q

ACCESSING AREAS OTHER THAN /tmp
One important difference from rootd(1) is that xrootd by default only allows one to access files in /tmp. This is for security reasons so that a running xrootd does not give remote clients access to read (or write to!) everything visible as xrootd.

A client accessing an area which is not explicitly exported by xrootd e.g., root://kan001//data/mydatafile.root will get an error.

You must specify explicitly any directories you wish to export via xrootd. This can be done on the command line when starting xrootd, for example:

    kan001> xrootd /data

will export the /data directory.

MULTIPLE SERVERS
Often it is the case that the data that an application wants to access is spread over multiple data servers.

The client application thus needs to know (or determine from some external source) on which server any given data file is located. It can then access those files directly with URL's like:

    root://kan001//data/aaaFile.root
    root://kan002//data/bbbFile.root

While this may work for small systems with few users, it has some disadvantages:

    If users know statically where files are located it doesn't easily permit load balancing by moving files between servers or by replicating them to spread the load.

    If an external catalog is read by the client in order to determine which server has the file, any changes to that catalog must be synchronized with what actually happens to the files on disk.

    If the file isn't actually on disk, but is retrieved from somewhere else when the client asks for it (e.g. mass storage, discussed in later examples), there is no easy way to manage the disk space to insure that it is used efficiently and uniformly.

The xrootd system provides a method for solving these problems via its open load balancer component. This provides a method for providing transparent access data servers.

                                       +-- kan001 ----------------+
                                       |  +--------+  +---------+ |
                                     +--->| Server |  | Storage | |
                         + kanrdr +  | |  |  olbd  |  |  /data  | |
    +-----------+    +--------+   |  | +--| xrootd |--|         |-+
    | Client    |--->| Server |------+    +--------+  +---------+
    | TXNetFile |    | xrootd |   |  |
    +-----------+    |  olbd  |---+  | +-- kan002 ----------------+
                     +--------+      | |  +--------+  +---------+ |
                                     +--->| Server |  | Storage | |
                                     | |  |  olbd  |  |  /data  | |
                                     | +--| xrootd |--|         |-+
                                     |    +--------+  +---------+
                                     |
                                     ...

In addition to the dataservers an additional machine (the redirector - kanrdr above) is added. It does not need to have any disk space itself as its sole role is to redirect the client dynamically to an actual data server machine which can serve the requested file. Thus client applications open all files via the redirector kanrdr:

    root://kanrdr//data/aaaFile.root
    root://kanrdr//data/bbbFile.root

The redirection to the appropriate dataserver will take place automatically (with the proper server configuration, discussed below) and hence the user running the application doesn't need to know anything about the actual layout of the data over the data servers. Note also that after redirection the client is reading directly from the data server to which it was redirected. It does need the redirector any longer (at least until it needs to open a different file).

To accomplish this a second daemon, the olbd, must be started on both the redirector and each of the dataservers as show in the figure above. Configuration of an xrootd/olbd system

OPTIONS

-l logfile
    The primary advantage of using this option as opposed to simply redirecting the output to a file is that there is also a feature which does automatic log file rotation. At midnight each night xrootd will move the current logfile to logfile.yyyymmdd and then continue writing to logfile.

-p port
    As with rootd(1), xrootd will started automatically on the IANA assigned port number 1094. If you need to start it on another port, that can be done using this option. Clients will need to specify this port when accessing the server, using an URL like root://kan001:2525//tmp/mydatafile.root. In general it is probably simplest to run xrootd on the standard port 1094.

-c configfile
    xrootd can be configured via a config file as well as via command line options. There are in fact many more options available via the config file and hence this is the recommended way of starting xrootd is to use this option. The config file configfile can have configuration directives as described below.

-r  Flag this server as a redirector.

CONFIGURATION FILES
A configuration file for xrootd looks like

    # Start the server on port 2525
    xrd.port 2525
    # Export files in /data only
    xrootd.export /data

in order to achieve (for example) what was done with command line options in the sections above. Note that lines beginning with # are taken to be comments and ignored. For the moment we just note that all config lines are of the form:

    configclass.option [optval1] [optval2] [...]

It is also important to note that spelling errors in either configclass or option will result in the entire line being (silently) ignored.

Read-only data access
An important thing to note about xrootd access to files is that by default it allows any accessed file to be opened in update mode (and/or new files to be created). For many situations this is undesirable: the goal is simply to make data files available for others to read. The easiest thing to do in that case is simply to disallow all write access. To do this you need to add two lines to your config file:

    xrootd.fslib /opt/xrootd/lib/libXrdOfs.so
    oss.readonly

in your configuration file. The first of these will cause the shared library for the optional ofs/oss file layer to be loaded (the exact path the lib area will depend on how you installed xrootd). This shared library provides a number of filesystem extensions. The second line disallows all write/update access using one of these options.

More fine-grained configurations are possible, but globally disallowing writes like this should cover a common use case.

Multiple servers
As an example we take a redirector (called kanrdr) and two data servers (called kan001 and kan002) with files aaaFile.root and bbbFile.root as above.

For the redirector we use a config file:

    #
    # redirector.cf
    #
    # xrootd
    xrootd.fslib /opt/xrootd/lib/libXrdOfs.so
    xrootd.export /data
    odc.manager kanrdr 3121
    odc.trace redirect
    # olbd
    olb.port 3121
    olb.allow host kanrdr.slac.stanford.edu
    olb.allow host kan001.slac.stanford.edu
    olb.allow host kan002.slac.stanford.edu

and for the data servers we use:

    #
    # dataserver.cf
    #
    # xrootd
    xrootd.fslib /opt/xrootd/lib/libXrdOfs.so
    xrootd.export /data
    oss.readonly
    odc.manager kanrdr 3121
    # olbd
    olb.port 3121
    olb.subscribe kanrdr 3121

As in the earlier examples, the actual path the to the libXrdOfs.so may depend on your installation. (And you should obviously substitute your machine names for kanrdr/kan001/kan002.)

It is then possible to start the xrootd and olbd servers on each of the machines.

On dataserver kan001:

    xrootd    -l kan001.xrdlog -c dataserver.cf &
    olbd -s   -l kan001.olblog -c dataserver.cf &

On dataserver kan002:

    xrootd    -l kan002.xrdlog -c dataserver.cf &
    olbd -s   -l kan002.olblog -c dataserver.cf &

And on the redirector kanrdr:

    xrootd -r -l kanrdr.xrdlog -c redirector.cf &
    olbd -m   -l kanrdr.olblog -c redirector.cf &

This should give you a working system capable of providing (readonly) access to data files on two servers transparently masqueraded behind a single redirector machine.

Testing the example load balanced system
Once the system is running we can try to open one of the files, for example:

    root [1] myfile = TFile::Open("root://kanrdr//data/aaafile.root");
    root [2] myfile->ls();
    XTNetFile**             root://kanrdr//data/aaafile.root
     XTNetFile*             root://kanrdr//data/aaafile.root
      KEY: TF1      fooHist;1       gaus
      KEY: TF1      barHist;1       gaus

Your client will be redirected and connect itself to kan001, the actual location of the requested file.

SEE ALSO
rootd(1), proofd(1), system.rootdaemonrc(1)

See also http://xrootd.slac.stanford.edu for more on the extended ROOT server.

For more information on the ROOT system, please refer to http://root.cern.ch

ORIGINAL AUTHORS
The ROOT team (see web page above):
    Rene Brun and Fons Rademakers

COPYRIGHT
ROOT Software Terms and Conditions

The authors hereby grant permission to use, copy, and distribute this software and its documentation for any purpose, provided that existing copyright notices are retained in all copies and that this notice is included verbatim in any distributions. Additionally, the authors grant permission to modify this software and its documentation for any purpose, provided that such modifications are not distributed without the explicit consent of the authors and that existing copyright notices are retained in all copies. Users of the software are asked to feed back problems, benefits, and/or suggestions about the software to the ROOT Development Team (rootdev@root.cern.ch). Support for this software - fixing of bugs, incorporation of new features - is done on a best effort basis. All bug fixes and enhancements will be made available under the same terms and conditions as the original software,

IN NO EVENT SHALL THE AUTHORS OR DISTRIBUTORS BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OF THIS SOFTWARE, ITS DOCUMENTATION, OR ANY DERIVATIVES THEREOF, EVEN IF THE AUTHORS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

THE AUTHORS AND DISTRIBUTORS SPECIFICALLY DISCLAIM ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, AND THE AUTHORS AND DISTRIBUTORS HAVE NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.

AUTHOR
This manual page was originally written by Christian Holm Christensen <cholm@nbi.dk>, for the Debian GNU/Linux system and ROOT version 3.