PRunner - A simplified job runner

Synopsis

prunner [OPTIONS] -c <node count> <executable> [args]

Description

The PRunner is the wrapper runner of KOALA runners that simplify the job submission process. With the PRunner, the user does not have to prepare the Job Description File (JDF); the job request requirements are all specified on the command line. The PRunner is capable of submitting flexible single component jobs on remote machines via SSH only. At the moment, the MPI implementation supported by the PRunner is the OpenMPI. It should be noted that he KRunner can be used to submit the MPICH applications.

Options

-l <LEVEL> : set log4j <FATAL|ERROR|WARN|DEBUG> output level
-flex : the job request is flexible
-optComm : if possible, try to optimize communication
-cm : if possible, try to minimize the number of clusters used
-x <clusters> : comma separated list of clusters not to be used
-np <processes> : number of processes to run per node
-host <cluster> : the preferred cluster to run the job
-wall <time> : estimated runtime of the job in minutes
-lib <libraries> : shared libraries to export when launching the job
-stdout <file> : write standard output to a file
-stderr <file> : write standard error to a file
-h|-help|--help : print this message

Examples

The following are examples of running jobs with the PRunner.

A sequential job submission

In this example we run a simple program that determines the interface containing the external IP address of the 5 arbitrary nodes of the cluster fs2.das3.science.uva.nl.

[hashim@fs3 bin]$ prunner -host fs2 -c 5 /home/hashim/bin/ifs
Ksched - Assigned job ID 78792
Ksched - Job 78792 Assigned LOW_PRIORITY
Ksched - Reservation for component 1 succeed
Ksched - Placed component 1 on fs2.das3.science.uva.nl
Runner - Submitting for execution component 1 to fs2.das3.science.uva.nl
Ksched - Claiming for processors for job 78792 begins
DRMAA - Component1@ fs2.das3.science.uva.nl: QUEUED
addr node230.das3.science.uva.nl: interface eth0:0
addr node210.das3.science.uva.nl: interface eth0:0
addr node212.das3.science.uva.nl: interface eth0:0
addr node207.das3.science.uva.nl: interface eth0:0
addr node217.das3.science.uva.nl: interface eth0:0
DRMAA - Component1@ fs2.das3.science.uva.nl: ACTIVE
Runner - Job 78792 has completed successfully

An OpenMPI job execution

In this example we run a flexible mpi application that calculates pi. The job is run on 200 cores where we choose to use the -optComm switch. This switch asks the KOALA, if possible, to use the clusters that have minimum average latency.

[hashim@fs3 ~]$ prunner -c 100 -np 2 -flex -optComm ~/bin/cpi-quite.ompi
Ksched - Job 78800 Assigned LOW_PRIORITY
Ksched - Assigned job ID 78800
Ksched - Splitted the flexible job 78800 into 2 components
Ksched - Placed component 1 on fs2.das3.science.uva.nl
Ksched - Reservation for component 1 succeed
Ksched - Placed component 2 on fs0.das3.cs.vu.nl
Ksched - Reservation for component 2 succeed
Runner - Submitting for execution component 1 to fs2.das3.science.uva.nl
Runner - Submitting for execution component 2 to fs0.das3.cs.vu.nl
Ksched - Claiming for processors for job 78800 begins
Runner - Stagein executable
Runner - Stagein executable
DM - Stage in file /home/hashim/bin/cpi-quite.ompi to fs2.das3.science.uva.nl
#
DM - Transferred /home/hashim/bin/cpi-quite.ompi to fs2.das3.science.uva.nl
DM - Stage in file /home/hashim/bin/cpi-quite.ompi to fs0.das3.cs.vu.nl
#
DM - Transferred /home/hashim/bin/cpi-quite.ompi to fs0.das3.cs.vu.nl
DRMAA - Component1@ fs2.das3.science.uva.nl: QUEUED
DRMAA - Component2@ fs0.das3.cs.vu.nl: QUEUED
DRMAA - Component1@ fs2.das3.science.uva.nl: ACTIVE
DRMAA - Component2@ fs0.das3.cs.vu.nl: ACTIVE
pi is approximately 3.1415926544231270, Error is 0.0000000008333338
wall clock time = 0.067551
Runner - Job 78800 has completed successfully

KOALA News

  • January 2013: MR-Runner upgraded! Now the MR-Runner deploys Hadoop-1.0.0 clusters, compatible with Pig-0.10.0. 

  • December 2012KOALA 2.1 released! Deploy MapReduce clusters on DAS-4 with the Koala MR-Runner

  • November 2012:  Best Paper Award at MTAGS12 workshop (co-located with SC12) with work on MapReduce!

  • November 2009KOALA 2.0 released! You can now run Parameter sweep applications (PSAs) with KOALA CSRunner

  • April 2008: New KOALA runner! The OMRunner enables DRMAA and OpenMPI job submissions. 

  • July 2007: Paper accepted at Grid07 conference with work on scheduling malleable jobs in KOALA.

  • May 2007: KOALA has now been ported successfully to DAS-3. All the KOALA runners are operational apart from the DRunner.

  • April 2007: The KOALA IRunner has been updated to include recommendations made by the Ibis group