COMPLEX(5)

NAME
complex - Grid Engine complexes configuration file format

DESCRIPTION
Complex reflects the format of the Grid Engine complexes configuration. The definition of complexes provides all pertinent information concerning the resource attributes a user may request for a Grid Engine job via the -l option and for the interpretation of these parameters within the Grid Engine system.

The complexes configuration files should not be accessed directly. In order to add or modify complexes, the options -Ac, -ac, -Mc and -mc should be used instead. While the -Ac and -Mc options take a complex configuration file as an argument, the -ac and -mc options bring up an editor filled in with a template complex configuration or the configuration of an existing complex.

The Grid Engine complexes object integrates 4 different types of complexes:

  The Queue Complex
It is referenced by the special name "queue".

In its default form it contains a selection of parameters in the queue configuration as defined in The queue configuration parameters being requestable for a job by the user in principal are:

    qname
    hostname
    notify
    calendar
    min_cpu_interval
    tmpdir
    seq_no
    s_rt
    h_rt
    s_cpu
    h_cpu
    s_data
    h_data
    s_stack
    h_stack
    s_core
    h_core
    s_rss
    h_rss
The queue complex can be extended if further attributes are intended to be available for each queue. The queue complex defines the characteristics (such as the data type) of the attributes it contains. A value setting for the queue complex attributes is defined by the queue configuration for each queue in case of the standard parameters enlisted above, or by the complex_values entry in the queue configuration (see for details) if a parameter has been added to the default queue complex. If no definition for the value in the complex_values entry of the queue configuration is given in the latter case, the value is set as defined by the value field described below.

  The Host Complex
It is referenced by the special name "host" and contains the characteristics definition of all attributes which are intended to be managed on a host basis. The standard set of host related attributes consists of two categories, but it may be enhanced like the queue complex as described above. The first category is built by several queue configuration attributes which are particularly suitable to be managed on a host basis. These attributes are:

    slots
    s_vmem
    h_vmem
    s_fsize
    h_fsize
(please refer to for details).

Note: Defining these attributes in the host complex is no contradiction to having them also in the queue configuration. It allows maintaining the corresponding resources on a host level and at the same time on a queue level. Total virtual free memory (h_vmem) can be managed for a host, for example, and a subset of the total amount can be associated with a queue on that host.

The second attribute category in the standard host complex are the default load values Every periodically reports load to The reported load values are either the standard Grid Engine load values such as the CPU load average (see or load values defined by the Grid Engine administration (see the load_sensor parameter in the cluster configuration and the Grid Engine Installation and Administration Guide for details). The characteristics definition for the standard load values is part of the default host complex, while administrator defined load values require extension of the host complex. Please refer to the file <sge_root>/doc/load_parameters.asc for detailed information on the standard set of load values.

The host complex commonly is not only extended to include non-standard load parameters, but also to manage host related resources such as the number of software licenses being assigned to a host or the available disk space on a host local filesystem.

A concrete value for a particular host complex attribute is determined by either an associated queue configuration in the case of the queue configuration derived attributes, a reported load value or the explicit definition of a value in the complex_values entry of the corresponding host configuration (see If none of the above is available (e.g. the value is supposed to be a load parameter, but does not report a load value for it), the value field described below is used.

  The Global Complex
It is referenced by the special name "global".

The entries configured in the global complex refer to cluster wide resource attributes, such as the number of available "floating" licenses of a particular software or the free disk space on a network wide available filesystem. Global resource attributes can also be associated with load reports, if the corresponding load report contains the "GLOBAL" identifier (see the corresponding section in the Grid Engine Installation and Administration Guide for details). Global load values can be reported from any host in the cluster. There are no global load values reported by Grid Engine by default and hence there is no default global complex configuration.

Concrete values for global complex attributes are either determined by global load reports or by explicit definition in the complex_values parameter of the "global" host configuration (see If none of both is present (e.g. a load value has not yet been reported) the value field described below is used.

  User Defined Complexes
By setting up user defined complexes the Grid Engine administration has the ability to extend the set of attributes managed by Grid Engine while restricting the influence of those attributes to particular queues and/or hosts. A user complex is just a named collection of attributes and the corresponding definition as to how these attributes are to be handled by Grid Engine. One or more of these user defined complexes can be attached to a queue and/ or host via the complex_list queue and host configuration parameter (see and The attributes defined in all assigned complexes become available to the queue and the host respectively in addition to the default complex attributes.

Concrete values for user defined complexes have to be set by the complex_values parameter in the queue and host configuration or otherwise the value field described below is used.

FORMAT
The principal format of a complex configuration is that of a tabulated list. Each line starting with a '#' character is a comment line. Each line despite comment lines define one element of the complex. A element definition line consists of the following 6 column entries per line (in the order of appearance):

  name
The name of the complex element to be used to request this attribute for a job in the -l option. An attribute name may appear only once across all complexes, i.e. the complex attribute definition is unique.

  shortcut
A shortcut for name which may also be used to request this attribute for a job in the -l option. An attribute shortcut may appear only once across all complexes, so as to avoid the possibility of ambiguous complex attribute references.

  type
This setting determines how the corresponding values are to be treated Grid Engine internally in case of comparisons or in case of load scaling for the load complex entries:

•   With INT only raw integers are allowed.

•   With DOUBLE floating point numbers in double precision (decimal and scientific notation) can be specified.

•   With TIME time specifiers are allowed. Refer to for a format description.

•   With MEMORY memory size specifiers are allowed. Refer to for a format description.

•   With BOOL the strings TRUE and FALSE are allowed. When used in a load formula (refer to ) TRUE and FALSE get mapped into '1' and '0'.

•   With STRING all strings are allowed and is used for comparisons.

•   CSTRING is like STRING except comparisons are case insensitive.

•   HOST is like CSTRING but the string must be a valid hostname.

  value
The value field is a pre-defined value setting for an attribute, which only has an effect if it is not overwritten while attempting to determine a concrete value for the attribute with respect to a queue, a host or the Grid Engine cluster. The value field can be overwritten by

•   the queue configuration values of a referenced queue.

•   host specific and cluster related load values.

•   explicit specification of a value via the complex_values parameter in the queue or host configuration (see and for details.

If none of above is applicable, value is set for the attribute.

  relop
The relation operator. The relation operator is used when the value requested by the user for this parameter is compared against the corresponding value configured for the considered queues. If the result of the comparison is false, the job cannot run in this queue. Possible relation operators are "==", "<", ">", "<=" and ">=". The only valid operator for string type attributes is "==".

  requestable
The entry can be used in a resource request if this field is set to 'y' or 'yes'. If set to 'n' or 'no' this entry cannot be used by a user in order to request a queue or a class of queues. If the entry is set to 'forced' or 'f' the attribute has to be requested by a job or it is rejected.

  consumable
The consumable parameter can be set to either 'yes' ('y' abbreviated) or 'no' ('n'). It can be set to 'yes' only for numeric attributes (INT, MEMORY, TIME - see type above). If set to 'yes' the consumption of the corresponding resource can be managed by Grid Engine internal bookkeeping. In this case Grid Engine accounts for the consumption of this resource for all running jobs and ensures that jobs are only dispatched if the Grid Engine internal bookkeeping indicates enough available consumable resources. Consumables are an efficient means to manage limited resources such a available memory, free space on a file system, network bandwidth or floating software licenses.

Consumables can be combined with default or user defined load parameters (see and i.e. load values can be reported for consumable attributes or the consumable flag can be set for load attributes. The Grid Engine consumable resource management takes both the load (measuring availability of the resource) and the internal bookkeeping into account in this case, and makes sure that neither of both exceeds a given limit.

To enable consumable resource management the basic availability of a resource has to be defined. This can be done on a cluster global, per host and per queue basis while these categories may supersede each other in the given order (i.e. a host can restrict availability of a cluster resource and a queue can restrict host and cluster resources). The definition of resource availability is performed with the complex_values entry in and The complex_values definition of the "global" host specifies cluster global consumable settings. To each consumable complex attribute in a complex_values list a value is assigned which denotes the maximum available amount for that resource. The internal bookkeeping will subtract from this total the assumed resource consumption by all running jobs as expressed through the jobs' resource requests.

Note: Jobs can be forced to request a resource and thus to specify their assumed consumption via the 'force' value of the requestable parameter (see above).

Note also: A default resource consumption value can be pre-defined by the administrator for consumable attributes not explicitly requested by the job (see the default parameter below). This is meaningful only if requesting the attribute is not enforced as explained above.

See the Grid Engine Installation and Administration Guide for examples on the usage of the consumable resources facility.

  default
Meaningful only for consumable complex attributes (see consumable parameter above). Grid Engine assumes the resource amount denoted in the default parameter implicitly to be consumed by jobs being dispatched to a host or queue managing the consumable attribute. Jobs explicitly requesting the attribute via the -l option to override this default value.

SEE ALSO
Grid Engine Installation and Administration Guide.

COPYRIGHT
See for a full statement of rights and permissions.