Skip to end of metadata
Go to start of metadata

Install procedure has been moved to Installing GridEngine

Using

qrsh - interactive login to a free node.

qsub <job> - submit job to queue

qsub -e ./ -o ./ -S /bin/bash -N label <script> - a typical set of qsub parameters

qsub -b yes <program> - when you want to run a program.

qstat -f - show which nodes are running jobs

qstat -u "*" - show user information for all jobs in the queue

qstat -f -u "*" - show user information for all nodes in the cluster

qstat -j <job_number> - show detailed information about a particular job

qconf -sql - list all queues

qconf -sq all.q - show settings for all.q queue



Actiongridengine command
Start an interactive shell on a compute nodeqrsh
Submit a script to the work queueqsub <script>
Show the status of your jobsqstat
Show detailed information about a particular jobqstat -j <job number>




Wrapper Script

Typically for programs that are used frequently a wrapper script is created that include all the qsub parameters for that program.

Included examples for such scripts are found in $SGE_ROOT/examples/jobs/


Here is a simple qsub script. Place all of lines in the code block into a file called sample1.sh



#!/usr/bin/env bash

######################################################
# execute script in current directory
#$ -cwd
# want any .e/.o stuff to show up here too
#$ -e ./
#$ -o ./
# shell for qsub to use:
#$ -S /bin/bash
# label for the job; displayed by qstat
#$ -N sample1
######################################################

# if you normally run the program like this:
# qsub sample1.sh

echo "BEGIN"

sleep 10

echo "END"



When you submit this script to gridengine, it will print "BEGIN" pause for 10 seconds, then print "END" and finish.  Replace "sleep 10" with your program and that may all that may be needed to run the program.



Here is a more complex script.  Place all the lines in the code block into a file called sample2.sh

qsub jobs do not get environmental variable settings from the user account, so anything set in .bashrc or .bash_profile is ignored by qsub. For flexibility, such settings should be placed into a separate file and included in a qsub script using the source command:  source ~/customsettings.sh


#!/usr/bin/env bash
######################################################
#  execute script in current directory
#$ -cwd
#  want any .e/.o stuff to show up here too
#$ -e ./
#$ -o ./
#  shell for qsub to use:
#$ -S /bin/bash
#  label for the job; displayed by qstat
#$ -N sample2
######################################################

# if you normally run the program like this:
# qsub sample2.sh param1 param2 param3 ...
# then run this script like this:
# qsub qsub_runme.sh  "param1 param2 param3 ..."
# it's important to contain all parameters in quotes

source ~/custom_settings.sh

PROGRAM=~/runme

# $1 are all the parameters passed above

$PROGRAM $1


A more complex example:

#!/usr/bin/env bash
######################################################
#  execute script in current directory
#$ -cwd
#  want any .e/.o stuff to show up here too
#$ -e ./
#$ -o ./
#  shell for qsub to use:
#$ -S /bin/bash
#  label for the job; displayed by qstat
#$ -N runsmp_job
#$ -l mem_free=9G
#$ -pe smp 4
######################################################

# if you normally run the program like this:
# runme param1 param2 param3 ...
# then run this script like this:
# qsub qsub_runme.sh "param1 param2 param3 ..."
# it's important to contain all parameters in quotes

source ~/custom_smp_settings.sh

PROGRAM=~/runsmp

START=$(date +%s)
NSTART=$(date +%s.%N)

# uncomment the following if you want a delay before
# the script is run -- helpful when debugging.
#time=20
#sleep $time

# $1 are all the parameters passed above

$PROGRAM $1

END=$(date +%s)
NEND=$(date +%s.%N)

let TOTAL="${END} - ${START}"
DIFF=$(echo "${NEND} - ${NSTART}" | bc)

echo "Script took ${TOTAL} seconds"
echo "Script tool ${DIFF} Nanoseconds"

tuning parameters

-l mem_free=17G Don't run on a node unless there is at least 17GB free.

Array Jobs

Array jobs are the preferred method of submitting many jobs which only differ due to an iterative process. For example: a large data set that has been split into smaller sets but are all being run through the same analysis.

Here is an example:

#!/usr/bin/env bash
######################################################
#  execute script in current directory
#$ -cwd
#  want any .e/.o stuff to show up here too
#$ -e ./
#$ -o ./
#  shell for qsub to use:
#$ -S /bin/bash
#  label for the job; displayed by qstat
#$ -N array_job
#$ -t 1-200
######################################################

# if you normally run the program like this:
# runme param1 param2 param3 ...
# then run this script like this:
# qsub qsub_runme.sh  "param1 param2 param3 ..."
# it's important to contain all parameters in quotes

source ~/custom_array_settings.sh

PROGRAM=~/runmeagain

# $1 are all the parameters passed above

$PROGRAM --input ~/data/input.$SGE_TASK_ID  --output ~/results/output.$SGE_TASK_ID

Where there exist 200 input files and $SGE_TASK_ID iterates between 1 and 200

More examples here:
http://wiki.gridengine.info/wiki/index.php/Simple-Job-Array-Howto
http://www.machlea.com/bohemian/?p=4


Job Dependency

Don't run job2 until job1 has completed.


Example:


#!/usr/bin/env bash
######################################################
# qsub_dependency.sh
# Function: Report various environmental variables
# Usage: qsub -N <job_name> qsub_dependency.sh 
######################################################
# execute script in current directory
#$ -cwd
# want any .e/.o stuff to show up here too
#$ -e ./
#$ -o ./
# shell for qsub to use:
#$ -S /bin/bash
# label for the job; displayed by qstat
## #$ -N dependency######################################################time=30
sleep ${time}

echo "Hostname: ${HOSTNAME}"
echo "N Hosts: ${NHOSTS}"
echo "N Slots: ${NSLOTS}"
echo "N Queues: ${NQUEUES}"
echo "Job ID: ${JOB_ID}"
echo "Job Name: ${JOB_NAME}"


qsub -N job1 qsub_dependency.sh
qsub -N job2 -hold_jid job1  qsub_dependency.sh

http://arc.leeds.ac.uk/using-the-systems/why-have-a-scheduler/advanced-sge-job-dependencies/

Interactive login

qrsh

Queue states

https://gist.github.com/cmaureir/4fa2d34bc9a1bd194af1

References

Note: after Oracle purchased Sun much of the SGE documentation has been pulled from public use.

  • No labels
{"serverDuration": 73, "requestCorrelationId": "cf963377df5c84f7"}View graphic versionView graphic versionView graphic version