org.springframework.data.hadoop.mapreduce
Class JobExecutor

java.lang.Object
  extended by org.springframework.data.hadoop.mapreduce.JobExecutor
All Implemented Interfaces:
Aware, BeanFactoryAware, DisposableBean, InitializingBean
Direct Known Subclasses:
JobRunner

public abstract class JobExecutor
extends Object
implements InitializingBean, DisposableBean, BeanFactoryAware

Common class shared for executing Hadoop Jobs.

Author:
Costin Leau, Thomas Risberg

Nested Class Summary
protected static interface JobExecutor.JobListener
           
 
Field Summary
protected  Log log
           
 
Constructor Summary
JobExecutor()
           
 
Method Summary
 void afterPropertiesSet()
           
 void destroy()
           
protected  Collection<Job> findJobs()
           
 boolean isKillJobsAtShutdown()
          Indicates whether the configured jobs should be 'killed' when the application shuts down or not.
 boolean isVerbose()
          Indicates whether the job execution is verbose (the default) or not.
 boolean isWaitForCompletion()
          Indicates whether the 'runner' should wait for the job to complete (default).
 void setBeanFactory(BeanFactory beanFactory)
           
 void setExecutor(Executor executor)
          Sets the TaskExecutor used for executing the Hadoop job.
 void setJob(Job job)
          Sets the job to execute.
 void setJobNames(String... jobName)
          Sets the jobs to execute by (bean) name.
 void setJobs(Collection<Job> jobs)
          Sets the jobs to execute.
 void setKillJobAtShutdown(boolean killJobsAtShutdown)
          Indicates whether the configured jobs should be 'killed' when the application shuts down (default) or not.
 void setVerbose(boolean verbose)
          Indicates whether the job execution is verbose (the default) or not.
 void setWaitForCompletion(boolean waitForJob)
          Indicates whether the 'runner' should wait for the job to complete (default) after submission or not.
protected  Collection<Job> startJobs()
           
protected  Collection<Job> startJobs(JobExecutor.JobListener listener)
           
protected  Collection<Job> stopJobs()
          Stops running job.
protected  Collection<Job> stopJobs(JobExecutor.JobListener listener)
          Stops running job.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

log

protected Log log
Constructor Detail

JobExecutor

public JobExecutor()
Method Detail

afterPropertiesSet

public void afterPropertiesSet()
                        throws Exception
Specified by:
afterPropertiesSet in interface InitializingBean
Throws:
Exception

destroy

public void destroy()
             throws Exception
Specified by:
destroy in interface DisposableBean
Throws:
Exception

stopJobs

protected Collection<Job> stopJobs()
Stops running job.

Returns:
list of stopped jobs.
Throws:
Exception

stopJobs

protected Collection<Job> stopJobs(JobExecutor.JobListener listener)
Stops running job.

Parameters:
listener - job listener
Returns:
list of stopped jobs.
Throws:
Exception

startJobs

protected Collection<Job> startJobs()

startJobs

protected Collection<Job> startJobs(JobExecutor.JobListener listener)

findJobs

protected Collection<Job> findJobs()

setJob

public void setJob(Job job)
Sets the job to execute.

Parameters:
job - The job to execute.

setJobs

public void setJobs(Collection<Job> jobs)
Sets the jobs to execute.

Parameters:
jobs - The job to execute.

setJobNames

public void setJobNames(String... jobName)
Sets the jobs to execute by (bean) name. This is the default method used by the hdp name space to allow lazy initialization and potential scoping to kick in.

Parameters:
jobName - The job to execute.

isWaitForCompletion

public boolean isWaitForCompletion()
Indicates whether the 'runner' should wait for the job to complete (default).

Returns:
whether to wait for the job to complete or not.

setWaitForCompletion

public void setWaitForCompletion(boolean waitForJob)
Indicates whether the 'runner' should wait for the job to complete (default) after submission or not.

Parameters:
waitForJob - whether to wait for the job to complete or not.

isVerbose

public boolean isVerbose()
Indicates whether the job execution is verbose (the default) or not.

Returns:
whether the job execution is verbose or not.

setVerbose

public void setVerbose(boolean verbose)
Indicates whether the job execution is verbose (the default) or not.

Parameters:
verbose - whether the job execution is verbose or not.

setBeanFactory

public void setBeanFactory(BeanFactory beanFactory)
                    throws BeansException
Specified by:
setBeanFactory in interface BeanFactoryAware
Throws:
BeansException

setExecutor

public void setExecutor(Executor executor)
Sets the TaskExecutor used for executing the Hadoop job. By default, SyncTaskExecutor is used, meaning the calling thread is used. While this replicates the Hadoop behavior, it prevents running jobs from being killed if the application shuts down. For a fine-tuned control, a dedicated Executor is recommended.

Parameters:
executor - the task executor to use execute the Hadoop job.

isKillJobsAtShutdown

public boolean isKillJobsAtShutdown()
Indicates whether the configured jobs should be 'killed' when the application shuts down or not.

Returns:
whether or not to kill the configured jobs at shutdown

setKillJobAtShutdown

public void setKillJobAtShutdown(boolean killJobsAtShutdown)
Indicates whether the configured jobs should be 'killed' when the application shuts down (default) or not. For long-running or fire-and-forget jobs that live beyond the starting application, set this to false. Note that if setWaitForCompletion(boolean) is true, this flag is considered to be true as otherwise the application cannot shut down (since it has to keep waiting for the job).

Parameters:
killJobsAtShutdown - whether or not to kill configured jobs when the application shuts down