Spring for Apache Hadoop

org.springframework.yarn.batch.am
Class AbstractBatchAppmaster

java.lang.Object
  extended by org.springframework.yarn.support.LifecycleObjectSupport
      extended by org.springframework.yarn.am.AbstractAppmaster
          extended by org.springframework.yarn.am.AbstractServicesAppmaster
              extended by org.springframework.yarn.am.AbstractEventingAppmaster
                  extended by org.springframework.yarn.batch.am.AbstractBatchAppmaster
All Implemented Interfaces:
java.util.EventListener, org.springframework.beans.factory.Aware, org.springframework.beans.factory.BeanFactoryAware, org.springframework.beans.factory.InitializingBean, org.springframework.context.ApplicationListener<AbstractYarnEvent>, org.springframework.context.Lifecycle, org.springframework.context.Phased, org.springframework.context.SmartLifecycle, ContainerLauncherInterceptor, YarnAppmaster, BatchYarnAppmaster
Direct Known Subclasses:
BatchAppmaster

public abstract class AbstractBatchAppmaster
extends AbstractEventingAppmaster
implements BatchYarnAppmaster, ContainerLauncherInterceptor

Base application master for running batch jobs.

Author:
Janne Valkealahti

Constructor Summary
AbstractBatchAppmaster()
           
 
Method Summary
 void addPartitionedStepExecutionStateListener(PartitionedStepExecutionStateListener listener)
          Adds the partitioned step execution state listener.
 void addStepSplits(org.springframework.batch.core.StepExecution masterStepExecution, java.lang.String remoteStepName, java.util.Set<org.springframework.batch.core.StepExecution> stepExecutions, java.util.Map<org.springframework.batch.core.StepExecution,ContainerRequestHint> resourceRequests)
          Adds the step splits.
 java.util.List<org.springframework.batch.core.StepExecution> getStepExecutions()
          Gets the step executions.
 YarnJobLauncher getYarnJobLauncher()
           
protected  void onContainerAllocated(org.apache.hadoop.yarn.api.records.Container container)
          Invoked when ContainerAllocationEvent is received as an application event.
protected  void onContainerCompleted(org.apache.hadoop.yarn.api.records.ContainerStatus status)
          Invoked when ContainerCompletedEvent is received as an application event.
protected  void onContainerLaunched(org.apache.hadoop.yarn.api.records.Container container)
          Invoked when ContainerLaunchedEvent is received as an application event.
protected  void onInit()
          Global application master instance specific ApplicationAttemptId is build during this init method.
 org.apache.hadoop.yarn.api.records.ContainerLaunchContext preLaunch(org.apache.hadoop.yarn.api.records.Container container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext context)
          Invoked before the ContainerLaunchContext is used to launch the container.
 void setYarnJobLauncher(YarnJobLauncher yarnJobLauncher)
           
 
Methods inherited from class org.springframework.yarn.am.AbstractEventingAppmaster
onApplicationEvent, onContainerLaunchRequestFailed
 
Methods inherited from class org.springframework.yarn.am.AbstractServicesAppmaster
getAllocator, getLauncher, getMonitor, setAllocator, setLauncher, setMonitor
 
Methods inherited from class org.springframework.yarn.am.AbstractAppmaster
addAppmasterStateListener, doStop, finishAppmaster, getApplicationAttemptId, getAppmasterClientService, getAppmasterService, getAppmasterTrackService, getCommands, getConfiguration, getContainerAssign, getEnvironment, getParameters, getResourceLocalizer, getTemplate, notifyCompleted, registerAppmaster, setCommands, setCommands, setConfiguration, setContainerAssign, setEnvironment, setFinalApplicationStatus, setParameters, setResourceLocalizer, setTemplate
 
Methods inherited from class org.springframework.yarn.support.LifecycleObjectSupport
afterPropertiesSet, doStart, getBeanFactory, getPhase, getTaskExecutor, getTaskScheduler, getYarnEventPublisher, isAutoStartup, isRunning, setAutoStartup, setBeanFactory, setPhase, setTaskExecutor, setTaskScheduler, setYarnEventPublisher, start, stop, stop
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface org.springframework.yarn.am.YarnAppmaster
addAppmasterStateListener, setEnvironment, setParameters, submitApplication
 

Constructor Detail

AbstractBatchAppmaster

public AbstractBatchAppmaster()
Method Detail

onInit

protected void onInit()
               throws java.lang.Exception
Description copied from class: AbstractAppmaster
Global application master instance specific ApplicationAttemptId is build during this init method.

Overrides:
onInit in class AbstractAppmaster
Throws:
java.lang.Exception - exception
See Also:
LifecycleObjectSupport.onInit()

onContainerAllocated

protected void onContainerAllocated(org.apache.hadoop.yarn.api.records.Container container)
Description copied from class: AbstractEventingAppmaster
Invoked when ContainerAllocationEvent is received as an application event. Wrapped Container is passed to a method.

Overrides:
onContainerAllocated in class AbstractEventingAppmaster
Parameters:
container - the container

onContainerLaunched

protected void onContainerLaunched(org.apache.hadoop.yarn.api.records.Container container)
Description copied from class: AbstractEventingAppmaster
Invoked when ContainerLaunchedEvent is received as an application event. Wrapped Container is passed to a method.

Overrides:
onContainerLaunched in class AbstractEventingAppmaster
Parameters:
container - the container

onContainerCompleted

protected void onContainerCompleted(org.apache.hadoop.yarn.api.records.ContainerStatus status)
Description copied from class: AbstractEventingAppmaster
Invoked when ContainerCompletedEvent is received as an application event. Wrapped ContainerStatus is passed to a method.

Overrides:
onContainerCompleted in class AbstractEventingAppmaster
Parameters:
status - the container status

preLaunch

public org.apache.hadoop.yarn.api.records.ContainerLaunchContext preLaunch(org.apache.hadoop.yarn.api.records.Container container,
                                                                           org.apache.hadoop.yarn.api.records.ContainerLaunchContext context)
Description copied from interface: ContainerLauncherInterceptor
Invoked before the ContainerLaunchContext is used to launch the container.

Specified by:
preLaunch in interface ContainerLauncherInterceptor
Parameters:
container - the Container
context - the ContainerLaunchContext
Returns:
Unchanged or modified ContainerLaunchContext

setYarnJobLauncher

@Autowired(required=false)
public void setYarnJobLauncher(YarnJobLauncher yarnJobLauncher)

getYarnJobLauncher

public YarnJobLauncher getYarnJobLauncher()

addPartitionedStepExecutionStateListener

public void addPartitionedStepExecutionStateListener(PartitionedStepExecutionStateListener listener)
Adds the partitioned step execution state listener.

Specified by:
addPartitionedStepExecutionStateListener in interface BatchYarnAppmaster
Parameters:
listener - the listener

getStepExecutions

public java.util.List<org.springframework.batch.core.StepExecution> getStepExecutions()
Gets the step executions.

Specified by:
getStepExecutions in interface BatchYarnAppmaster
Returns:
the step executions

addStepSplits

public void addStepSplits(org.springframework.batch.core.StepExecution masterStepExecution,
                          java.lang.String remoteStepName,
                          java.util.Set<org.springframework.batch.core.StepExecution> stepExecutions,
                          java.util.Map<org.springframework.batch.core.StepExecution,ContainerRequestHint> resourceRequests)
Adds the step splits.

Specified by:
addStepSplits in interface BatchYarnAppmaster
Parameters:
masterStepExecution - the partitioned steps parent step execution
remoteStepName - the remote step name
stepExecutions - the step executions splits
resourceRequests - the request data for step executions

Spring for Apache Hadoop