            Getting Accounting Information for Sun MPI jobs
            -----------------------------------------------

   This README explains how to work around to get accounting information for
   Sun MPI jobs. This requires to set up Solaris System accounting on each
   node of an HPC cluster running Sun HPC ClusterTools (CT) software. The
   accounting information can be retrived by executing Perl scripts
   interactively or manually. It should be noted that this package is
   provided on "as is" basis and may not work in certain conditions.


Content
-------

1. Setup for Solaris System Accounting
2. Accounting parser scripts
3. Usage
4. Limitations and Possible problems
5) Copyright


1. Setup for Solaris System Accounting
--------------------------------------

   Setup for Solaris System Accounting on every node running Sun HPC CT:

	- Install the SUNWaccr and SUNWaccu packages (if not installed yet).
		-- Use pkgadd

	- Install /etc/init.d/acct as the startup script for Run Level 2.
		-- # ln /etc/init.d/acct /etc/rc2.d/S22acct

	- Install /etc/init.d/acct as the startup script for Run Level 0.
		-- # ln /etc/init.d/acct /etc/rc0.d/K22acct

	- Add the following lines to the adm crontab file to start the ckpacct,
          runacct, and monacct programs automatically.
		
		# EDITOR=vi; export EDITOR
		# crontab -e adm
		0 * * * * /usr/lib/acct/ckpacct
		30 2 * * * /usr/lib/acct/runacct 2> /var/adm/acct/nite/fd2log
		30 7 1 * * /usr/lib/acct/monacct

	- Add the following line to the root crontab file to start the dodisk 
          program automatically.

		# crontab -e
		30 22 * * 4 /usr/lib/acct/dodisk

	- Edit /etc/acct/holidays to include national and local holidays.

	- Reboot the system, or type:
		# /etc/init.d/acct start


2. Accounting parser scripts
----------------------------

   The accounting parser scripts are found at Grid Engine installation
   $directory, SGE_ROOT/mpi/sunhpc/loose-integration/accounting.

	- There are two different scripts are provided.

		ctacct1.pl: Interactive parser to get accounting info for MPI 
                            jobs submitted thru SGE.
			    Need some detailed input from users

		ctacct2.pl: Automated parser called by "stopsunmpi.sh" script. 
			    Need to replace "stopsunmpi.sh" and MPRUN scripts
	                    with those found in the accounting directory.
			    Acct. info. mailed to the user.

3. Usage
--------

	- Both accounting scripts work for SGE jobs submitted by batch (qsub)
          and interactive (qrsh) commands.
	- ctacct1.pl -j <sge_jid> -h <comma-separated-hostnames> \
          -n <executable_name> -d
		-- Interactive script. Use it after completion of SGE jobs.
		-- "-d" for debugging

	- ctacct2.pl $JOB_ID $USER $TMPDIR $COD_JOB_SPOOL_DIR $DEBUG 
		-- Automated script which should be launched in 
                   "stopsunmpi.sh" script
		-- Argument order is important
		-- "DEBUG=1" for debugging.
		-- Other variables are SGE environment variables.


4. Limitations and Possible problems
------------------------------------

   - The scripts assume that all execution hosts are trused hosts. 

  - Spawning multiple processes: The parser script can not handle any 
    child processes to get their usage at the moment.

   - Muptiple SGE jobs by the same user
     -- If there're multiple  jobs with the same executable by 
        the same use, it's possible to get wrong accounting info 
        among those jobs because the parser may be confused.

   - Multiple node jobs
     -- If a job is running on multiple nodes and time is 
        different among nodes, the accounting scripts may fail 
        to get the all corresponding accounting information.

   - Does not work with Prism jobs due to spawning of child processes.


5) Copyright
------------

___INFO__MARK_BEGIN__
   This following copyright statement applies to this file only. It does not
   imply that this statement covers other files of a Grid Engine
   distribution.  See the directory '<sge_root>/3rdparty' for a summary of
   other copyrights of this Grid Engine distribution.

   The Contents of this file are made available subject to the terms of the
   Sun Industry Standards Source License Version 1.2

   Sun Microsystems Inc., March, 2001

   Sun Industry Standards Source License Version 1.2
   =================================================

   The contents of this file are subject to the Sun Industry Standards
   Source License Version 1.2 (the "License"); You may not use this file
   except in compliance with the License. You may obtain a copy of the
   License at http://gridengine.sunsource.net/Gridengine_SISSL_license.html

   Software provided under this License is provided on an "AS IS" basis,
   WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING,
   WITHOUT LIMITATION, WARRANTIES THAT THE SOFTWARE IS FREE OF DEFECTS,
   MERCHANTABLE, FIT FOR A PARTICULAR PURPOSE, OR NON-INFRINGING.

   See the License for the specific provisions governing your rights and
   obligations concerning the Software.

   The Initial Developer of the Original Code is: Sun Microsystems, Inc.

   Copyright: 2002 by Sun Microsystems, Inc.

   All Rights Reserved.
___INFO__MARK_END__
