COMPLEX(5)                 Grid Engine File Formats                 COMPLEX(5)



NNAAMMEE
       complex - Grid Engine complexes configuration file format

DDEESSCCRRIIPPTTIIOONN
       _C_o_m_p_l_e_x  reflects  the format of the Grid Engine complex configuration.
       The definition of complex attributes provides all pertinent information
       concerning the resource attributes a user may request for a Grid Engine
       job via the _q_s_u_b(1) --ll option  and  for  the  interpretation  of  these
       parameters within the Grid Engine system.

       The  Grid  Engine complex object defines all entries which are used for
       configuring the global, the host, and queue object. The  system  has  a
       set  of  pre  defined entries, which are assigned to a host or queue by
       default.  In a addition can the user define new entries and assign them
       to one or multiple objects. Each load value has to have its correspond-
       ing complex entry object, which defines the  type  and  the  relational
       operator for it.

   ddeeffiinniinngg rreessoouurrccee aattttrriibbuutteess
       The complex configuration should not be accessed directly.  In order to
       add or modify complex entries, the _q_c_o_n_f(1) options -Mc and -mc  should
       be  used  instead.   While the -Mc option takes a _c_o_m_p_l_e_x configuration
       file as an argument and overrides the current  configuration,  the  -mc
       option bring up an editor filled in with the current _c_o_m_p_l_e_x configura-
       tion.

       The provided list contains all definitions of  resource  attributes  in
       the  system. Adding a new entry means to provide: name, shortcut, type,
       relop, requestable, consumable, default, and urgency.  The  fields  are
       described  below.  Changing one is easily done by updating the field to
       change and removing an entry by deleting its definition.  An  attribute
       can  only  be  removed,  when  it  is not referenced in a host or queue
       object anymore. Also does the system have a  set  of  default  resource
       attributes which are always attached to a host or queue. They cannot be
       deleted nor can the type of such an attribute be changed.


   wwoorrkkiinngg wwiitthh rreessoouurrccee aattttrriibbuutteess
       Before a user can request a resource attribute it has to be attached to
       the  global, host, or cqueue object. The resource attribute exists only
       for the objects, it got attached to( if it is attached  to  the  global
       object(qconf  -me  global),  it exits system wide, host object: only on
       that host (qconf -me NAME): cqueue object: only on that  cqueue  (qconf
       -mq NAME).

       When  the user attached a resource attribute to an object, one also has
       to assign a value to it; the resource  limit.  Another  way  to  get  a
       resource  attribute  value  is  by  configuring  a load sensor for that
       attribute.


   DDeeffaauulltt qquueeuuee rreessoouurrccee aattttrriibbuutteess
       In its default form it contains a selection of parameters in the  queue
       configuration  as  defined  in  _q_u_e_u_e___c_o_n_f(5).  The queue configuration
       parameters requestable for a job by the user are:

              qname
              hostname
              notify
              calendar
              min_cpu_interval
              tmpdir
              seq_no
              s_rt
              h_rt
              s_cpu
              h_cpu
              s_data
              h_data
              s_stack
              h_stack
              s_core
              h_core
              s_rss
              h_rss

   DDeeffaauulltt hhoosstt rreessoouurrccee aattttrriibbuutteess
       The standard set of host related attributes consists of two categories.
       The  first  category is built by several queue configuration attributes
       which are particularly suitable to be managed on a  host  basis.  These
       attributes are:

              slots
              s_vmem
              h_vmem
              s_fsize
              h_fsize
       (please refer to _q_u_e_u_e___c_o_n_f(5) for details).

       NNoottee:: Defining these attributes in the host complex is no contradiction
       to having them also in the queue configuration. It  allows  maintaining
       the  corresponding  resources on a host level and at the same time on a
       queue level. Total virtual free memory (h_vmem) can be  managed  for  a
       host,  for  example, and a subset of the total amount can be associated
       with a queue on that host.

       The second attribute category in the  standard  host  complex  are  the
       default  load  values  Every  _s_g_e___e_x_e_c_d(8) periodically reports load to
       _s_g_e___q_m_a_s_t_e_r(8).  The reported load values are either the standard  Grid
       Engine load values such as the CPU load average (see _u_p_t_i_m_e(1)) or load
       values defined by the Grid Engine administration (see  the  llooaadd__sseennssoorr
       parameter  in the cluster configuration _s_g_e___c_o_n_f(5) and the _G_r_i_d _E_n_g_i_n_e
       _I_n_s_t_a_l_l_a_t_i_o_n _a_n_d _A_d_m_i_n_i_s_t_r_a_t_i_o_n _G_u_i_d_e for details).   The  characteris-
       tics  definition  for  the  standard load values is part of the default
       host complex, while administrator defined load values require extension
       of the host complex. Please refer to the file $SGE_ROOT/doc/load_param-
       eters.asc for detailed information on the standard set of load  values.

   OOvveerrrriiddiinngg aattttrriibbuutteess
       One  attribute  can  be assigned to the global object, host object, and
       queue object at the same time. On the host level it might get its value
       from  the  user  defined resource limit and a load sensor. In case that
       the attribute is a consumable, we have  in  addition  to  the  resource
       limit  and its load report on host level also the internal usage, which
       the system keeps track of. The merger is done as follows:

       In general an attribute can be overridden on a lower level
          - global by hosts and queues
          - hosts by queues and load values or resource  limits  on  the  same
       level.

       We  have  one  limitation  for overriding attributes based on its rela-
       tional operator:

       !=, == operators can only be overridden on the same level, but not on a
       lower level. The user defined value always overrides the load value.

       >=,  >,  <=,  < operators can only be overridden, when the new value is
       more restrictive than the old one.

       In the case of a consumable on host level, which has also a  load  sen-
       sor,  the system checks for the current usage. If the internal account-
       ing is more restrictive than the load sensor report, the internal value
       is  kept;  if  the  load sensor report is more restrictive, that one is
       kept.

       Note: Grid Engine allows backslashes (\)  be  used  to  escape  newline
       (\newline)  characters. The backslash and the newline are replaced with
       a space (" ") character before any interpretation.


FFOORRMMAATT
       The principal format of a _c_o_m_p_l_e_x configuration is that of a  tabulated
       list.  Each  line starting with a '#' character is a comment line. Each
       line despite comment lines define one element of the complex. A element
       definition line consists of the following 8 column entries per line (in
       the order of appearance):

   nnaammee
       The name of the complex element to be used to  request  this  attribute
       for  a job in the _q_s_u_b(1) --ll option. A complex attribute name (see _c_o_m_-
       _p_l_e_x___n_a_m_e in _s_g_e___t_y_p_e_s(1)) may appear only once across  all  complexes,
       i.e. the complex attribute definition is unique.

   sshhoorrttccuutt
       A  shortcut  for  nnaammee which may also be used to request this attribute
       for a job in the _q_s_u_b(1) --ll option. An attribute  sshhoorrttccuutt  may  appear
       only  once  across  all  complexes,  so  as to avoid the possibility of
       ambiguous complex attribute references.

   ttyyppee
       This setting determines how the corresponding values are to be  treated
       Grid  Engine internally in case of comparisons or in case of load scal-
       ing for the load complex entries:

       +o  With IINNTT, only raw integers are allowed.

       +o  With DDOOUUBBLLEE, floating point numbers in double precision (decimal and
          scientific notation) can be specified.

       +o  With TTIIMMEE, time specifiers are allowed. Refer to _q_u_e_u_e___c_o_n_f(5) for a
          format description.

       +o  With  MMEEMMOORRYY,  memory  size  specifiers  are   allowed.   Refer   to
          _q_u_e_u_e___c_o_n_f(5) for a format description.

       +o  With  BBOOOOLL,  the  strings TRUE and FALSE are allowed. When used in a
          load formula (refer to _s_c_h_e_d___c_o_n_f(5) ) TRUE  and  FALSE  get  mapped
          into '1' and '0'.

       +o  With  SSTTRRIINNGG, all strings are allowed and is used for wildcard regu-
          lar boolean expression matching.  Please  see  _s_g_e___t_y_p_e_s(1)  manpage
          for eexxpprreessssiioonn definition.

          Examples:
           -l arch="*x24*|sol*"  :
                results in "arch=lx24-x86" OR "arch=lx24-amd64"
                   OR "arch=sol-sparc" OR "arch=sol-sparc64"
                   OR "arch=sol-x86" OR ...
           -l arch="sol-x??"  :
                results in "arch=sol-x86" OR "arch=sol-x64" OR ...
           -l arch="lx2[246]-x86"  :
                results in "arch=lx22-x86" OR "arch=lx24-x86"
                   OR "arch=lx26-x86"
           -l arch="lx2[4-6]-x86"  :
                results in "arch=lx24-x86" OR "arch=lx25-x86"
                   OR "arch=lx26-x86"
           -l arch="lx2[24-6]-x86"  :
                results in "arch=lx22-x86" OR "arch=lx24-x86"
                   OR "arch=lx25-x86" OR "arch=lx26-x86"
           -l arch="!lx24-x86&!sol-sparc"  :
                results in NEITHER "arch=lx24-x86" NOR "arch=sol-sparc"
           -l arch="lx2[4|6]-x86"  :
                results in "arch=lx2[4" OR "arch=6"


       +o  CCSSTTRRIINNGG is like SSTTRRIINNGG except comparisons are case insensitive.

       +o  RREESSTTRRIINNGG is like SSTTRRIINNGG and it will be deprecated in the future.

       +o  HHOOSSTT is like CCSSTTRRIINNGG but the expression must match a valid hostname.

   rreelloopp
       The rreellaattiioonn ooppeerraattoorr..  The relation operator is used  when  the  value
       requested by the user for this parameter is compared against the corre-
       sponding value configured for the considered queues. If the  result  of
       the  comparison  is  false,  the job cannot run in this queue. Possible
       relation operators are "==", "<", ">", "<=" and ">=".  The  only  valid
       operator for string type attributes is "==".

   rreeqquueessttaabbllee
       The  entry  can  be used in a _q_s_u_b(1) resource request if this field is
       set to 'y' or 'yes'.  If set to 'n' or 'no' this entry cannot  be  used
       by  a  user  in  order to request a queue or a class of queues.  If the
       entry is set to 'forced' or 'f' the attribute has to be requested by  a
       job or it is rejected.

       To  enable resource request enforcement the resource has to be defined.
       This can be done on a cluster global, per host and per queue basis. The
       definition  of resource availability is performed with the complex_val-
       ues entry in _h_o_s_t___c_o_n_f(5) and _q_u_e_u_e___c_o_n_f(5).

   ccoonnssuummaabbllee
       The ccoonnssuummaabbllee parameter can be set to either 'yes'  ('y'  abbreviated)
       or 'no' ('n'). It can be set to 'yes' only for numeric attributes (INT,
       DOUBLE, MEMORY, TIME - see ttyyppee above). If set to 'yes' the consumption
       of  the  corresponding  resource can be managed by Grid Engine internal
       bookkeeping. In this case Grid Engine accounts for the  consumption  of
       this  resource for all running jobs and ensures that jobs are only dis-
       patched if the Grid Engine internal bookkeeping indicates enough avail-
       able consumable resources. Consumables are an efficient means to manage
       limited resources such a available memory, free space on a file system,
       network bandwidth or floating software licenses.

       Consumables  can  be combined with default or user defined load parame-
       ters (see _s_g_e___c_o_n_f(5)  and  _h_o_s_t___c_o_n_f(5)),  i.e.  load  values  can  be
       reported  for  consumable  attributes or the consumable flag can be set
       for load attributes. The Grid  Engine  consumable  resource  management
       takes  both  the  load (measuring availability of the resource) and the
       internal bookkeeping into account, and makes sure that neither  exceeds
       a given limit.

       To  enable  consumable  resource management the basic availability of a
       resource has to be defined. This can be done on a cluster  global,  per
       host  and  per  queue  basis  while these categories may supersede each
       other in the given order (i.e. a host can restrict  availability  of  a
       cluster  resource and a queue can restrict host and cluster resources).
       The definition of resource availability  is  performed  with  the  ccoomm--
       pplleexx__vvaalluueess  entry in _h_o_s_t___c_o_n_f(5) and _q_u_e_u_e___c_o_n_f(5).  The ccoommpplleexx__vvaall--
       uueess definition of the "global" host specifies cluster global consumable
       settings. To each consumable complex attribute in a ccoommpplleexx__vvaalluueess list
       a value is assigned which denotes the maximum available amount for that
       resource.  The  internal  bookkeeping will subtract from this total the
       assumed resource consumption by all running jobs as  expressed  through
       the jobs' resource requests.

       NNoottee::  Jobs  can  be  forced  to request a resource and thus to specify
       their assumed consumption via the  'force'  value  of  the  rreeqquueessttaabbllee
       parameter (see above).

       NNoottee  aallssoo::  A default resource consumption value can be pre-defined by
       the administrator for consumable attributes not explicitly requested by
       the  job  (see the ddeeffaauulltt parameter below). This is meaningful only if
       requesting the attribute is not enforced as explained above.

       See the _G_r_i_d _E_n_g_i_n_e _I_n_s_t_a_l_l_a_t_i_o_n _a_n_d _A_d_m_i_n_i_s_t_r_a_t_i_o_n _G_u_i_d_e for  examples
       on the usage of the consumable resources facility.

   ddeeffaauulltt
       Meaningful  only  for  consumable  complex  attributes  (see ccoonnssuummaabbllee
       parameter above). Grid Engine assumes the resource  amount  denoted  in
       the ddeeffaauulltt parameter implicitly to be consumed by jobs dispatched to a
       host or  queue  managing  the  consumable  attribute.  Jobs  explicitly
       requesting  the  attribute  via  the _-_l option to _q_s_u_b(1) override this
       default value.

   uurrggeennccyy
       The urgency value allows influencing job priorities on a  per  resource
       basis.  The  urgency  value  affects the addition of each resource when
       determining the resource  request  related  urgency  contribution.  For
       numeric  type  resource  requests  the  addition  is the product of the
       urgency value, the jobs  assumed  slot  allocation  and  the  per  slot
       request  as  specified  via  _-_l  option  to  _q_s_u_b(1).   For string type
       requests the resources urgency value is directly used as added. Urgency
       values  are  of type real. See under _s_g_e___p_r_i_o_r_i_t_y(5) for an overview on
       job priorities.

SSEEEE AALLSSOO
       _s_g_e___i_n_t_r_o(1), _s_g_e___t_y_p_e_s(1), _q_c_o_n_f(1), _q_s_u_b(1), _u_p_t_i_m_e(1), _h_o_s_t___c_o_n_f(5),
       _q_u_e_u_e___c_o_n_f(5), _s_g_e___e_x_e_c_d(8), _s_g_e___q_m_a_s_t_e_r(8), _s_g_e___s_c_h_e_d_d(8),
       _G_r_i_d _E_n_g_i_n_e _I_n_s_t_a_l_l_a_t_i_o_n _a_n_d _A_d_m_i_n_i_s_t_r_a_t_i_o_n _G_u_i_d_e_.

CCOOPPYYRRIIGGHHTT
       See _s_g_e___i_n_t_r_o(1) for a full statement of rights and permissions.



GE 6.1                   $Date: 2007/07/19 08:17:17 $               COMPLEX(5)
