The Runners
If you want to know more about the runners, please visit our Publications page on this site.
The Runners Framework
The runners architecture shown above provides a set of APIs to the runners for commonly used grid job submission operations such as interfacing with the KOALA scheduler, for job scheduling and interfacing with a grid middleware for deploying jobs on grids, monitoring job submission files and transferring input and output files.
The AbstractRunner provides to the runners a set of APIs for job scheduling with the KOALA scheduler, and application level operations such as deploying jobs on grids, input/output file transfers, and monitoring and responding to failures. The runners have complete freedom to implement their own mechanisms for the application level operations, or alternatively, to use the implementantions bound to a certain grid middleware provided by the runners framework.
The DRMAARunners provides to the runners a set of APIs for deploying jobs using OGF's Distributed Resource Management Application API (DRMAA). DRMAARunners uses SSH public-key approach to authenticate users when submitting jobs to remote clusters. Secure FTP (SFTP) provided by SSH is a default mechanism to transfer files, however, other mechanisms for file transfers can be used.
The GlobusRunners provides to the runners a set of APIs for deploying jobs using Globus middleware. To use the runners based on the GlobusRunners, Globus user certificates are required. The Globus GridFTP, and GASS provide the default mechanisms for file transfers, however, other mechanisms can also be used.
The GATRunners provides to the runners a set of APIs for deploying jobs using the Java Grid Application Toolkit (JavaGAT). This implementation is still in progress.
Users can also write their own implementations by simply extending the AbstractRunner class and implementing its abstract methods.
Writing a Simple Runner
To write a KOALA runner, first make sure you have the latest build of the KOALA runners from our svn (see the download page). Assume that we want to write a runner that uses DRMAA and SSH for job submission.
First, create a runner Java class that extends the DRMAARunners class. The following method stubs are inherited from the DRMAARunners class and need to be implemented by the runner. The method stubs are explained below.
package org.koala.runnersFramework.runners;
import org.koala.internals.JCompRunnerInfo;
import org.koala.runnersFramework.DRMAARunners;
public class TestRunner extends DRMAARunners {
public TestRunner() {}
@Override
public void parseParameters(String[] args) {
}
@Override
public void postPhase(boolean jobRunWasSuccessful) {
}
@Override
public boolean prePhase() {
return false;
}
@Override
public void precondSubmit(JCompRunnerInfo component) {
}
@Override
public int submitComponent(JCompRunnerInfo component) {
return 0;
}
/**
* @param args
*/
public static void main(String[] args) {
}
}The prePhase() method is used to prepare the job before scheduling such as initializing parameters and setting the submission environment. This method is first to be called by the Runners Framework after the job has been submitted.
After a component has been placed, i.e., its execution site has been determined, the precondSubmit(JCompRunnerInfo component) method is invoked by the Runners Framework. This method can be used to prepare the component before submission. For example, transferring of input files to the execution site, setting the shared library paths, and for a parallel job, synchronizing the start of the job components.
The method submitComponent(JCompRunnerInfo component) should implement how a job component is submitted to the execution site. This method which is called after the precondSubmit method, implements middleware-specific submission mechanisms.
After the job has finished its execution the prePhase() method is called. This method can be used to transfer output files back to the submission site. Afther this method has been executed the runner exits.
KOALA News
- January 2013: MR-Runner upgraded! Now the MR-Runner deploys Hadoop-1.0.0 clusters, compatible with Pig-0.10.0.
- December 2012: KOALA 2.1 released! Deploy MapReduce clusters on DAS-4 with the Koala MR-Runner.
- November 2012: Best Paper Award at MTAGS12 workshop (co-located with SC12) with work on MapReduce!
- November 2009: KOALA 2.0 released! You can now run Parameter sweep applications (PSAs) with KOALA CSRunner.
- April 2008: New KOALA runner! The OMRunner enables DRMAA and OpenMPI job submissions.
- July 2007: Paper accepted at Grid07 conference with work on scheduling malleable jobs in KOALA.
- May 2007: KOALA has now been ported successfully to DAS-3. All the KOALA runners are operational apart from the DRunner.
- April 2007: The KOALA IRunner has been updated to include recommendations made by the Ibis group .
