COMPLEX(5)                 Grid Engine File Formats                 COMPLEX(5)



[1mNAME[0m
       complex - Grid Engine complexes configuration file format

[1mDESCRIPTION[0m
       [4mComplex[24m  reflects  the format of the Grid Engine complex configuration.
       The definition of complex attributes provides all pertinent information
       concerning the resource attributes a user may request for a Grid Engine
       job via the [4mqsub[24m(1) [1m-l [22moption  and  for  the  interpretation  of  these
       parameters within the Grid Engine system.

       The  Grid  Engine complex object defines all entries which are used for
       configuring the global, the host, and queue object. The  system  has  a
       set  of  pre defined entries, which are assigned to a host or queue per
       default.  In a addition can the user define new entries and assign them
       to one or multiple objects. Each load value has to have its correspond-
       ing complex entry object, which defines the  type  and  the  relational
       operator for it.

   [1mdefining resource attributes[0m
       The complex configuration should not be accessed directly.  In order to
       add or modify complex entries, the [4mqconf[24m(1) options -Mc and -mc  should
       be  used  instead.   While the -Mc option takes a [4mcomplex[24m configuration
       file as an argument and overrides the current  configuration,  the  -mc
       option bring up an editor filled in with the current [4mcomplex[24m configura-
       tion.

       The provided list contains all definitions of  resource  attributes  in
       the  system. Adding a new entry means to provide: name, shortcut, type,
       relop, requestable, consumable, default, and urgency.  The  fields  are
       described  below.  Changing  one is easily done by updated the field to
       change and removing an entry by deleting its definition.  An  attribute
       can  only  be  removed,  when  it  is not referenced in a host or queue
       object anymore. Also does the system have a  set  of  default  resource
       attributes which are always attached to a host or queue. They cannot be
       deleted nor can the type of such an attribute be changed.


   [1mworking with resource attributes[0m
       Before a user can request a resource attribute it has to be attached to
       the  global, host, or cqueue object. The resource attribute exists only
       for the objects, it got attached to( if it is attached  to  the  global
       object(qconf  -me  global),  it  exits sytem wide, host object: only on
       that host (qconf -me NAME): cqueue object: only on that  cqueue  (qconf
       -mq NAME).

       When  the user attached a resource attribute to an object, one also has
       to assign a value to it; the resource  limit.  Another  way  to  get  a
       resource  attribute value is done by configuring a load sensor for that
       attribute.


   [1mDefault queue resource attributes[0m
       In its default form it contains a selection of parameters in the  queue
       configuration  as  defined  in  [4mqueue_conf[24m(5).  The queue configuration
       parameters being requestable for a job by the user in principal are:

              qname
              hostname
              notify
              calendar
              min_cpu_interval
              tmpdir
              seq_no
              s_rt
              h_rt
              s_cpu
              h_cpu
              s_data
              h_data
              s_stack
              h_stack
              s_core
              h_core
              s_rss
              h_rss

   [1mDefault host resource attributes[0m
       The standard set of host related attributes consists of two categories.
       he  first  category  is built by several queue configuration attributes
       which are particularly suitable to be managed on a  host  basis.  These
       attributes are:

              slots
              s_vmem
              h_vmem
              s_fsize
              h_fsize
       (please refer to [4mqueue_conf[24m(5) for details).

       [1mNote: [22mDefining these attributes in the host complex is no contradiction
       to having them also in the queue configuration. It  allows  maintaining
       the  corresponding  resources on a host level and at the same time on a
       queue level. Total virtual free memory (h_vmem) can be  managed  for  a
       host,  for  example, and a subset of the total amount can be associated
       with a queue on that host.

       The second attribute category in the  standard  host  complex  are  the
       default  load  values  Every  [4msge_execd[24m(8) periodically reports load to
       [4msge_qmaster[24m(8).  The reported load values are either the standard  Grid
       Engine load values such as the CPU load average (see [4muptime[24m(1)) or load
       values defined by the Grid Engine administration (see  the  [1mload_sensor[0m
       parameter  in the cluster configuration [4msge_conf[24m(5) and the [4mGrid[24m [4mEngine[0m
       [4mInstallation[24m [4mand[24m [4mAdministration[24m [4mGuide[24m for details).   The  characteris-
       tics  definition  for  the  standard load values is part of the default
       host complex, while administrator defined load values require extension
       of     the     host    complex.    Please    refer    to    the    file
       <sge_root>/doc/load_parameters.asc  for  detailed  information  on  the
       standard set of load values.

   [1mOverriding attributes[0m
       One  attribute  can  be assigned to the global object, host object, and
       queue object at the same time. On the host level it might get its value
       from  the  user  defined resource limit and a load sensor. In case that
       the attribute is a consumable, we have  in  addition  to  the  resource
       limit  and its load report on host level also the internal usage, which
       the system keeps track of. The mergin is done as follows:

       In general an attribute can be overridden on a lower level
          - global by hosts and queues
          - hosts by queues and load values or resource  limits  on  the  same
       level.

       We  have  one  limitation  for overriding attributes based on its rela-
       tional operator:

       !=, == operators can only be overridden on the same level, but not on a
       lower level. The user defined value always overrides the load value.

       >=,  >,  <=,  < operators can only be overridden, when the new value is
       more restrictive than the old one.

       In the case of a consumable on host level, which has also a  load  sen-
       sor,  the  system  checks  for  the  current usage, and if the internal
       accounting is more restrictive than the load sensor report, the  inter-
       nal value is kept; if the load sensour report is more restrictive, that
       one is kept.


[1mFORMAT[0m
       The principal format of a [4mcomplex[24m configuration is that of a  tabulated
       list.  Each  line starting with a '#' character is a comment line. Each
       line despite comment lines define one element of the complex. A element
       definition line consists of the following 8 column entries per line (in
       the order of appearance):

   [1mname[0m
       The name of the complex element to be used to  request  this  attribute
       for  a  job in the [4mqsub[24m(1) [1m-l [22moption. An attribute [1mname [22mmay appear only
       once across all complexes, i.e. the  complex  attribute  definition  is
       unique.

   [1mshortcut[0m
       A  shortcut  for  [1mname [22mwhich may also be used to request this attribute
       for a job in the [4mqsub[24m(1) [1m-l [22moption. An attribute  [1mshortcut  [22mmay  appear
       only  once  across  all  complexes,  so  as to avoid the possibility of
       ambiguous complex attribute references.

   [1mtype[0m
       This setting determines how the corresponding values are to be  treated
       Grid  Engine internally in case of comparisons or in case of load scal-
       ing for the load complex entries:

       +o  With [1mINT [22monly raw integers are allowed.

       +o  With [1mDOUBLE [22mfloating point numbers in double precision (decimal  and
          scientific notation) can be specified.

       +o  With  [1mTIME [22mtime specifiers are allowed. Refer to [4mqueue_conf[24m(5) for a
          format description.

       +o  With  [1mMEMORY  [22mmemory  size  specifiers   are   allowed.   Refer   to
          [4mqueue_conf[24m(5) for a format description.

       +o  With  [1mBOOL  [22mthe  strings  TRUE and FALSE are allowed. When used in a
          load formula (refer to [4msched_conf[24m(5) ) TRUE  and  FALSE  get  mapped
          into '1' and '0'.

       +o  With  [1mSTRING  [22mall strings are allowed and [4mstrcmp[24m(3) is used for com-
          parisons.

       +o  [1mCSTRING [22mis like [1mSTRING [22mexcept comparisons are case insensitive.

       +o  [1mRESTRING [22mis like [1mSTRING [22mwith regular expression matching.  The  syn-
          tax follows the following rules:

           - "*"    : matches any character and any number of chars
                      (0 or more).
           - "?"    : matches any one character. It must match exactly 1
                      character.
           - "."    : is the character ".". It has no special meaning
           - "\"    : escape character. "\\" = "\", "\*" = "*",
                      "\?" = "?"
           - "[...]": matches one of the list of characters between the brackets.
                      Character ranges may be specified using the a-z notation.
                      The caret symbol (^) is [1mnot [22minterpreted as a logical
                      not; it is intrepreted literally.
           - "|"    : logical "or". Can only be used on the highest
                      level and [1mcannot [22mbe escaped.

          Examples:
           -l arch="*x24*|sol*"  :
                results in "arch=lx24-x86" OR "arch=lx24-amd64"
                   OR "arch=sol-sparc" OR "arch=sol-sparc64"
                   OR "arch=sol-x86" OR ...
           -l arch="sol-x??"  :
                results in "arch=sol-x86" OR "arch=sol-x64" OR ...
           -l arch="lx2[246]-x86"  :
                results in "arch=lx22-x86" OR "arch=lx24-x86"
                   OR "arch=lx26-x86"
           -l arch="lx2[4-6]-x86"  :
                results in "arch=lx24-x86" OR "arch=lx25-x86"
                   OR "arch=lx26-x86"
           -l arch="lx2[24-6]-x86"  :
                results in "arch=lx22-x86" OR "arch=lx24-x86"
                   OR "arch=lx25-x86" OR "arch=lx26-x86"
           -l arch="lx24-x86|sol-sparc"  :
                results in "arch=lx24-x86" OR "arch=sol-sparc"
           -l arch="lx24-[^x]86"  :
                results in "arch=lx24-^86" OR "arch=lx24-x86"
           -l arch="lx2[4|6]-x86"  :
                results in "arch=lx2[4" OR "arch=6"


       +o  [1mHOST [22mis like [1mCSTRING [22mbut the string must be a valid hostname.

   [1mrelop[0m
       The  [1mrelation  operator.   [22mThe relation operator is used when the value
       requested by the user for this parameter is compared against the corre-
       sponding  value  configured for the considered queues. If the result of
       the comparison is false, the job cannot run  in  this  queue.  Possible
       relation  operators  are  "==", "<", ">", "<=" and ">=". The only valid
       operator for string type attributes is "==".

   [1mrequestable[0m
       The entry can be used in a [4mqsub[24m(1) resource request if  this  field  is
       set  to  'y' or 'yes'.  If set to 'n' or 'no' this entry cannot be used
       by a user in order to request a queue or a class  of  queues.   If  the
       entry  is set to 'forced' or 'f' the attribute has to be requested by a
       job or it is rejected.

       To enable resource request enforcement the existence  of  the  resource
       has  to  be defined. This can be done on a cluster global, per host and
       per queue basis. The definition of resource availability  is  performed
       with the complex_values entry in [4mhost_conf[24m(5) and [4mqueue_conf[24m(5).

   [1mconsumable[0m
       The  [1mconsumable  [22mparameter can be set to either 'yes' ('y' abbreviated)
       or 'no' ('n'). It can be set to 'yes' only for numeric attributes (INT,
       DOUBLE, MEMORY, TIME - see [1mtype [22mabove). If set to 'yes' the consumption
       of the corresponding resource can be managed by  Grid  Engine  internal
       bookkeeping.  In  this case Grid Engine accounts for the consumption of
       this resource for all running jobs and ensures that jobs are only  dis-
       patched if the Grid Engine internal bookkeeping indicates enough avail-
       able consumable resources. Consumables are an efficient means to manage
       limited resources such a available memory, free space on a file system,
       network bandwidth or floating software licenses.

       Consumables can be combined with default or user defined  load  parame-
       ters  (see  [4msge_conf[24m(5)  and  [4mhost_conf[24m(5)),  i.e.  load  values can be
       reported for consumable attributes or the consumable flag  can  be  set
       for  load  attributes.  The  Grid Engine consumable resource management
       takes both the load (measuring availability of the  resource)  and  the
       internal  bookkeeping  into  account  in this case, and makes sure that
       neither of both exceeds a given limit.

       To enable consumable resource management the basic  availability  of  a
       resource  has  to be defined. This can be done on a cluster global, per
       host and per queue basis while  these  categories  may  supersede  each
       other  in  the  given order (i.e. a host can restrict availability of a
       cluster resource and a queue can restrict host and cluster  resources).
       The  definition  of  resource  availability  is performed with the [1mcom-[0m
       [1mplex_values [22mentry in [4mhost_conf[24m(5) and [4mqueue_conf[24m(5).  The  [1mcomplex_val-[0m
       [1mues [22mdefinition of the "global" host specifies cluster global consumable
       settings. To each consumable complex attribute in a [1mcomplex_values [22mlist
       a value is assigned which denotes the maximum available amount for that
       resource. The internal bookkeeping will subtract from  this  total  the
       assumed  resource  consumption by all running jobs as expressed through
       the jobs' resource requests.

       [1mNote: [22mJobs can be forced to request a  resource  and  thus  to  specify
       their  assumed  consumption  via  the  'force' value of the [1mrequestable[0m
       parameter (see above).

       [1mNote also: [22mA default resource consumption value can be  pre-defined  by
       the administrator for consumable attributes not explicitly requested by
       the job (see the [1mdefault [22mparameter below). This is meaningful  only  if
       requesting the attribute is not enforced as explained above.

       See  the [4mGrid[24m [4mEngine[24m [4mInstallation[24m [4mand[24m [4mAdministration[24m [4mGuide[24m for examples
       on the usage of the consumable resources facility.

   [1mdefault[0m
       Meaningful only  for  consumable  complex  attributes  (see  [1mconsumable[0m
       parameter  above).  Grid  Engine assumes the resource amount denoted in
       the [1mdefault [22mparameter implicitly to be  consumed  by  jobs  being  dis-
       patched  to  a  host  or  queue managing the consumable attribute. Jobs
       explicitly requesting the attribute via the [4m-l[24m option to [4mqsub[24m(1)  over-
       ride this default value.

   [1murgency[0m
       The  urgency  value allows influencing job priorities on a per resource
       base. The urgency value effects  the  addend  for  each  resource  when
       determining  the  resource  request  related  urgency contribution. For
       numeric type resource requests the addend is the product of the urgency
       value,  the  jobs  assumed  slot allocation and the per slot request as
       specified via [4m-l[24m option to  [4mqsub[24m(1).   For  string  type  requests  the
       resources  urgency value is directly used as addend. Urgency values are
       of type real. See under [4msge_priority[24m(5) for an overview on job  priori-
       ties.

[1mSEE ALSO[0m
       [4msge_intro[24m(1),     [4mqconf[24m(1),     [4mqsub[24m(1),    [4muptime[24m(1),    [4mhost_conf[24m(5),
       [4mqueue_conf[24m(5), [4msge_execd[24m(8), [4msge_qmaster[24m(8), [4msge_schedd[24m(8),
       [4mGrid[24m [4mEngine[24m [4mInstallation[24m [4mand[24m [4mAdministration[24m [4mGuide.[0m

[1mCOPYRIGHT[0m
       See [4msge_intro[24m(1) for a full statement of rights and permissions.



GE 6.0                   $Date: 2007/01/26 11:00:53 $               COMPLEX(5)
