JSV(1)                     Grid Engine File Formats                     JSV(1)



NNAAMMEE
       JSV - Grid Engine Job Submission Verifier

DDEESSCCRRIIPPTTIIOONN
       JSV  is  an abbreviation for Job Submission Verifier. A JSV is a script
       or binary that can be used to verify, modify or reject a job during the
       time of job submission.

       JSVs  will be triggered by submit clients like _q_s_u_b, _q_r_s_h, _q_s_h and _q_m_o_n
       on submit hosts (Client JSV) or they verify incoming jobs on the master
       host (Server JSV) or both.

CCOONNFFIIGGUURRAATTIIOONN
       JSVs  can  be  configured on various locations. Either a jjssvv__uurrll can be
       provided with the --jjssvv submit parameter during job submission, a corre-
       sponding  switch  can  be  added  to  one of the ssggee__rreeqquueesstt files or a
       jjssvv__uurrll can be configured in the global cluster  configuration  of  the
       Grid Engine installation.

       All defined JSV instances will be executed in following order:

          1) qsub -jsv ...
          2) $cwd/.sge_request
          3) $HOME/.sge_request
          4) $SGE_ROOT/$SGE_CELL/common/sge_request
          5) Global configuration

       The  Client  JSVs (1-3) can be defined by Grid Engine end users whereas
       the client JSV defined in the  global  sge_request  file  (4)  and  the
       server JSV (5) can only be defined by the Grid Engine administrators.

       Due  to  the  fact  that (4) and (5) are defined and configured by Grid
       Engine administrators  and  because  they  are  executed  as  last  JSV
       instances in the sequence of JSV scripts, an administrator has an addi-
       tional way to define certain policies for a cluster.

       As soon as one JSV instance rejects a job the whole process of  verifi-
       cation  is stopped and the end user will get a corresponding error mes-
       sage that the submission of the job has failed.

       If a JSV accepts a job or accepts a job after it applied serveral modi-
       fications  then  the following JSV instance will get the job parameters
       including all modifications as  input  for  the  verification  process.
       This is done as long as either the job is accepted or rejected.

       Find  more information how to use Client JSVs in _q_s_u_b(1) and for Server
       JSVs in _s_g_e___c_o_n_f(5)

LLIIFFEETTIIMMEE
       A Client or Server JSV is started as own  UNIX  process.  This  process
       communicates  either  with  a  Grid Engine client process or the master
       daemon by exchanging  commands,  job  parameters  and  other  data  via
       ssttddiinn/ssttddoouutt channels.

       Client JSV instances are started by client applications before a job is
       sent to qmaster. This instance does the job verification for the job to
       be submitted. After that verification the JSV process is stopped.

       Server  JSV  instances  are  started for each worker thread part of the
       qmaster process (for version 6.2 of Grid Engine  this  means  that  two
       processes  are  started).  Each  of  those processes have to verify job
       parameters for multiple jobs as long as  the  master  is  running,  the
       underlaying JSV configuration is not changed and no error occurs.

PPRROOTTOOCCOOLL
       After  a  JSV  script or binary is started it will get commands through
       its ssttddiinn stream and it has to respond with  certain  commands  on  the
       ssttddoouutt  stream.  Data  which  is  send  via  the ssttddeerrrr stream of a JSV
       instance is ignored. Each command which is send to/by a JSV script  has
       to  be terminated by a new line character ('\\nn') whereas new line char-
       acters are not allowed in the whole command string itself.

       In general commands which are exchanged between a JSV and  client/qmas-
       ter  have  following format. Commands and arguments are case sensitive.
       Find the EBNF command description below.

             ccoommmmaanndd := ccoommmmaanndd__nnaammee ' ' { aarrgguummeenntt ' ' } ;

       A ccoommmmaanndd starts with a ccoommmmaanndd__nnaammee followed by a space character  and
       a space separated list of aarrgguummeennttss.

PPRROOTTOOCCOOLL ((JJSSVV ssiiddee))
       Following  ccoommmmaannddss  have to be implemented by an JSV script so that it
       conforms to version 1.0 of the JSV protocol which was first implemented
       in Grid Engine 6.2u2:

       bbeeggiinn__ccoommmmaanndd := 'BBEEGGIINN' ;
              After   a   JSV  instance  has  received  all  eennvv__ccoommmmaannddss  and
              ppaarraamm__ccoommmmaannddss  of  a  job  which  should   be   verified,   the
              client/qmaster  will trigger the verification process by sending
              one bbeeggiinn__ccoommmmaanndd. After that it will  wait  for  ppaarraamm__ccoommmmaannddss
              and  eennvv__ccoommmmaannddss  which  are sent back from the JSV instance to
              modify the  job  specification.  As  part  of  the  verification
              process  a JSV script or binary has to use the rreessuulltt__ccoommmmaanndd to
              indicate that the verification process is finished for a job.

       eennvv__ccoommmmaanndd := EENNVV ' ' mmooddiiffiieerr ' ' nnaammee ' ' vvaalluuee ;

       mmooddiiffiieerr := 'AADDDD' | 'MMOODD' | 'DDEELL' ;
              The eennvv__ccoommmmaanndd is an optional command  which  has  only  to  be
              implemented  by  a JSV instance if the sseenndd__ddaattaa__ccoommmmaanndd is sent
              by this JSV before a the ssttaarrtteedd__ccoommmmaanndd was sent. Only in  that
              case  the client or master will use one or multiple eennvv__ccoommmmaannddss
              to pass the environment variables (nnaammee and vvaalluuee)  to  the  JSV
              instance which would be exported to the job environment when the
              job would be started. Client and qmaster will only sent eennvv__ccoomm--
              mmaannddss with the modifier 'AADDDD'.

              JSV instances modify the set of environment variables by sending
              back eennvv__ccoommmmaannddss and by using the mmooddiiffiieerrss AADDDD, MMOODD and DDEELL.

       ppaarraamm__ccoommmmaanndd := 'PPAARRAAMM' ' ' ppaarraamm__ppaarraammeetteerr ' ' vvaalluuee ;

       ppaarraamm__ppaarraammeetteerr := ssuubbmmiitt__ppaarraammeetteerr | ppsseeuuddoo__ppaarraammeetteerr ;
              The ppaarraamm__ccoommmmaanndd has two additional arguments which  are  sepa-
              rated  by  space characters. The first argument is either a ssuubb--
              mmiitt__ppaarraammeetteerr  as  it  is  specified  in  _q_s_u_b(1)  or  it  is  a
              ppsseeuuddoo__ppaarraammeetteerrss  as documented below.  The second parameter is
              the vvaalluuee of the corresponding ppaarraamm__ppaarraammeetteerr.

              Multiple ppaarraamm__ccoommmmaannddss will be sent to a JSV instance after the
              JSV  has  sent  a ssttaarrtteedd__ccoommmmaanndd. The sum of all ppaarraamm__ccoommmmaannddss
              which is sent represents a job specification of that  job  which
              should be verified.

              ssuubbmmiitt__ppaarraammeetteerrss  are  for  example  bb  (similar to the qqssuubb --bb
              switch) or mmaasstteerrqq (similar to qqssuubb  --mmaasstteerrqq  switch).  Find  a
              complete  list  of  ssuubbmmiitt__ppaarraammeetteerrss  in  the _q_s_u_b(1) man page.
              Please note that not in all cases the ppaarraamm__ppaarraammeetteerr  name  and
              the  corresponding  vvaalluuee  format  is  equivalent  with the qsub
              switch name and its argument format.  E.g. the qqssuubb --ppee  parame-
              ters  will  by  available  as  a set of parameters with the name
              ppee__nnaammee, ppee__mmiinn, ppee__mmaaxx or the switch combination --ssoofftt --ll  will
              be  passed  to JSV scripts as ll__ssoofftt parameter. For details con-
              cerning this differences consult also the _q_s_u_b(1) man page.

       ssttaarrtt__ccoommmmaanndd := 'SSTTAARRTT' ;
              The ssttaarrtt__ccoommmmaanndd has  no  additional  arguments.  This  command
              indicates  that  a new job verification should be started. It is
              the first command which will be sent to JSV script after it  has
              been  started  and it will initiate each new job verification. A
              JSV instance might trash cached values which  are  still  stored
              due  to  a previous job verification. The application which send
              the ssttaarrtt__ccoommmmaanndd will wait for a ssttaarrtteedd__ccoommmmaanndd before it con-
              tinues.

       qquuiitt__ccoommmmaanndd := 'QQUUIITT' ;
              The qquuiitt__ccoommmmaanndd has no additional arguments. If this command is
              sent to a JSV instance then it should terminate  itself  immedi-
              ately.

PPRROOTTOOCCOOLL ((cclliieenntt//qqmmaasstteerr ssiiddee))
       A  JSV  script or binary can send a set of commands to a client/qmaster
       process to indicate its state in the communication process,  to  change
       the  job  specification of a job which should be verified and to report
       messages or errors. Below you can find the commands  which  are  under-
       stood  by  the  client/qmaster  which will implement version 1.0 of the
       communication protocol which  was  first  implemented  in  Grid  Engine
       6.2u2:

       eerrrroorr__ccoommmmaanndd := 'EERRRROORR' mmeessssaaggee ;
              Any  time a JSV script encounters an error it might report it to
              the client/qmaster. If the error happens during a job  verifica-
              tion  the  job which is currently verified will be rejected. The
              JSV binary or script will also be restarted before it gets a new
              verification task.

       lloogg__ccoommmmaanndd := 'LLOOGG' lloogg__lleevveell ;

       lloogg__lleevveell := 'IINNFFOO' | 'WWAARRNNIINNGG' | 'EERRRROORR'
              lloogg__ccoommmmaannddss  can be used whenever the client or qmaster expects
              input from a JSV instance. This command can be  used  in  client
              JSVs  to  send  information  to  the user submitting the job. In
              client JSVs all messages, independed of the  lloogg__lleevveell  will  be
              printed  to  the  ssttddoouutt  stream of the used submit client. If a
              server JSV receives a lloogg__ccoommmmaanndd it will add the received  mes-
              sage  to  the  message  file respecting the specified lloogg__lleevveell.
              Please note that mmeessssaaggee might contain spaces but  no  new  line
              characters.

       ppaarraamm__ccoommmmaanndd (find definition above)
              By sending ppaarraamm__ccoommmmaannddss a JSV script can change the job speci-
              fication of the job which should be verified. If a JSV  instance
              later  on  sends  a  rreessuulltt__ccoommmmaanndd  which  indicates that a JSV
              instance should be accepted with correction then the values pro-
              vided  with  these ppaarraamm__ccoommmmaannddss will be used to modify the job
              before it is accepted by the Grid Engine system.

       rreessuulltt__ccoommmmaanndd := 'RREESSUULLTT' rreessuulltt__ttyyppee [ mmeessssaaggee ] ;

       rreessuulltt__ttyyppee := 'AACCCCEEPPTT' | 'CCOORRRREECCTT' | 'RREEJJEECCTT' | 'RREEJJEECCTT__WWAAIITT' ;
              After the verification of a job is done a JSV script  or  binary
              has  to send a rreessuulltt__ccoommmmaanndd which indicates what should happen
              with the job.  If the rreessuulltt__ttyyppee is AACCCCEEPPTTEEDD the  job  will  be
              accepted  as  it  was  initially  submitted by the end user. All
              ppaarraamm__ccoommmmaannddss and  eennvv__ccoommmmaannddss  which  might  have  been  sent
              before  the  rreessuulltt__ccoommmmaanndd  are  ignored  in  this  case.   The
              rreessuulltt__ttyyppee CCOORRRREECCTT indicates that the job  should  be  accepted
              after all modifications sent via ppaarraamm__ccoommmmaannddss and eennvv__ccoommmmaannddss
              are applied to the job.  RREEJJEECCTT and RREEJJEECCTT__WWAAIITT cause the client
              or qmaster instance to reject the job.

       sseenndd__ddaattaa__ccoommmmaanndd := 'SSEENNDD' ddaattaa__nnaammee ;

       ddaattaa__nnaammee := 'EENNVV';
              If  a  client/qmaster  receives  a  sseenndd__eennvv__ccoommmmaanndd  from a JSV
              instance before a ssttaarrtteedd__ccoommmmaanndd is sent, then it will not only
              pass  job  parameters  with ppaarraamm__ccoommmmaannddss but also eennvv__ccoommmmaannddss
              which provide the JSV with  the  information  which  environment
              variables would be exported to the job environment if the job is
              accepted and started later on.

              The job environment is not passed to JSV  instances  as  default
              because  the  job environment of the end user might contain data
              which might be interpreted wrong in the JSV  context  and  might
              therefore cause errors or security issues.

       ssttaarrtteedd__ccoommmmaanndd := 'SSTTAARRTTEEDD' ;
              By  sending the ssttaarrtteedd__ccoommmmaanndd a JSV instance indicates that it
              is ready to receive ppaarraamm__ccoommmmaannddss and eennvv__ccoommmmaannddss  for  a  new
              job  verification. It will only receive eennvv__ccoommmmaannddss if it sends
              a sseenndd__ddaattaa__ccoommmmaanndd before the ssttaarrtteedd__ccoommmmaanndd.

PPSSEEUUDDOO PPAARRAAMMEETTEERRSS
       CCLLIIEENNTT The corresponding value for  the  CCLLIIEENNTT  parameters  is  either
              'qqmmaasstteerr' or the name of a submit client like 'qqssuubb',
               'qqsshh', 'qqrrsshh', 'qqllooggiinn' and so on This parameter value can't be
              changed by JSV instances.  It will always be sent as part  of  a
              job verification.

       CCMMDDAARRGGSS
              Number  of  arguments  which will be passed to the job script or
              command  when the job execution is started.  It will  always  be
              sent  as  part of a job verification.  If no arguments should be
              passed to the job script or command it will have  the  value  0.
              This  parameter can be changed by JSV instances. If the value of
              CCMMDDAARRGGSS is bigger than the number of available CCMMDDAARRGG<<iidd>> param-
              eters  then  the missing parameters will be automatically passed
              as empty parameters to the job script.

       CCMMDDNNAAMMEE
              Either the path to the script or the command  name  in  case  of
              binary submission.  It will always be sent as part of a job ver-
              ification.

       CCOONNTTEEXXTT
              Either 'cclliieenntt' if the JSV which receives this ppaarraamm__ccoommmmaanndd was
              started  by a commandline client like _q_s_u_b, _q_s_h, ... or 'mmaasstteerr'
              if it was started by the _s_g_e___q_m_a_s_t_e_r process.  It will always be
              sent  as part of a job verification.  Changing the value of this
              parameters is not possible within JSV instances.

       GGRROOUUPP  Defines Primary group of the user which tries to submit the  job
              which  should  be verifyed. This parameter cannot be changed but
              is always sent as part of the  verification  process.  The  user
              name is passed as parameters with the name UUSSEERR.

       JJOOBB__IIDD Not  available in the client context (see CCOONNTTEEXXTT). Otherwise it
              contains the job number of the job which will  be  submitted  to
              Grid Engine when the verification process is successfull. JJOOBB__IIDD
              is  an  optional  parameter  which  can't  be  changed  by   JSV
              instances.

       UUSSEERR   Username  of the user which tries to submit the job which should
              be verified. Cannot be changed but is always sent as part of the
              verification  process.  The  group  name  is passed as parameter
              with the name GGRROOUUPP

       VVEERRSSIIOONN
              VVEERRSSIIOONN will always be  sent  as  part  of  a  job  verification
              process and it will always be the first parameter which is sent.
              It will contain a version number of the format  <major>.<minor>.
              In  version 6.2u2 and higher the value will be '11..00'.  The value
              of this parameter can't be changed.

EEXXAAMMPPLLEE
       Here is an example for  the  communication  of  a  client  with  a  JSV
       instance when following job is submitted:

       > qsub -pe p 3 -hard -l a=1,b=5 -soft -l q=all.q $SGE_ROOT/examples/jobs/sleeper.sh

       Data  in  the  first  column is sent from the client/qmaster to the JSV
       instance. That data contained in the second column is sent from the JSV
       script  to the client/qmaster. New line characters which terminate each
       line in the communication protocol are ommited.

          START
                                  SEND ENV
                                  STARTED
          PARAM VERSION 1.0
          PARAM CONTEXT client
          PARAM CLIENT qsub
          PARAM USER ernst
          PARAM GROUP staff
          PARAM CMDNAME /sge_root/examples/jobs/sleeper.sh
          PARAM CMDARGS 1
          PARAM CMDARG0 12
          PARAM l_hard a=1,b=5
          PARAM l_soft q=all.q
          PARAM M user@hostname
          PARAM N Sleeper
          PARAM o /dev/null
          PARAM pe_name pe1
          PARAM pe_min 3
          PARAM pe_max 3
          PARAM S /bin/sh
          BEGIN
                                  RESULT STATE ACCEPT


SSEEEE AALLSSOO
       _g_e___i_n_t_r_o(1), _q_a_l_t_e_r(1), _q_l_o_g_i_n(1), _q_m_a_k_e(1), _q_r_s_h(1), _q_s_h(1),  _q_s_u_b(1),
       _q_t_c_s_h(1),

CCOOPPYYRRIIGGHHTT
       See _g_e___i_n_t_r_o(1) for a full statement of rights and permissions.



GE 6.2u2                 $Date: 2009/02/11 12:33:12 $                   JSV(1)
