|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||
java.lang.Objectorg.springframework.yarn.batch.partition.AbstractBatchPartitionHandler
public abstract class AbstractBatchPartitionHandler
Base implementation of Spring Batch PartitionHandler handling
partitioning for Yarn containers.
| Constructor Summary | |
|---|---|
AbstractBatchPartitionHandler()
Instantiates a new batch partition handler. |
|
AbstractBatchPartitionHandler(BatchYarnAppmaster batchAppmaster)
Instantiates a new batch partition handler. |
|
| Method Summary | |
|---|---|
protected Map<StepExecution,org.springframework.yarn.am.container.ContainerRequestHint> |
createResourceRequestData(Set<StepExecution> stepExecutions)
Subclass may override this method to assign a specific ContainerRequestHint to
a StepExecution. |
protected abstract Set<StepExecution> |
createStepExecutionSplits(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
|
org.springframework.yarn.am.container.ContainerResolver |
getContainerResolver()
Gets the container resolver. |
String |
getStepName()
Gets the step name. |
Collection<StepExecution> |
handle(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
|
void |
setBatchAppmaster(BatchYarnAppmaster batchAppmaster)
Sets the batch appmaster. |
void |
setContainerResolver(org.springframework.yarn.am.container.ContainerResolver containerResolver)
Sets the container resolver. |
void |
setStepName(String stepName)
Sets the step name. |
void |
setYarnAppmaster(org.springframework.yarn.am.YarnAppmaster yarnAppmaster)
|
protected void |
waitCompleteState(StepExecution masterStepExecution)
Uses CountDownLatch to wait completion status from
application master. |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
|---|
public AbstractBatchPartitionHandler()
public AbstractBatchPartitionHandler(BatchYarnAppmaster batchAppmaster)
batchAppmaster - the batch appmaster| Method Detail |
|---|
public void setBatchAppmaster(BatchYarnAppmaster batchAppmaster)
batchAppmaster - the new batch appmaster@Autowired(required=false) public void setYarnAppmaster(org.springframework.yarn.am.YarnAppmaster yarnAppmaster)
protected abstract Set<StepExecution> createStepExecutionSplits(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
throws Exception
Exception
protected Map<StepExecution,org.springframework.yarn.am.container.ContainerRequestHint> createResourceRequestData(Set<StepExecution> stepExecutions)
throws Exception
ContainerRequestHint to
a StepExecution. This would be needed in cases where step should be executed
in a specific host or rack considering data locality. Default implementation
returns an empty map.
stepExecutions - Set of step executions
Exception - If error occurred
public final Collection<StepExecution> handle(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
throws Exception
handle in interface PartitionHandlerExceptionpublic String getStepName()
public void setStepName(String stepName)
stepName - the new step namepublic org.springframework.yarn.am.container.ContainerResolver getContainerResolver()
public void setContainerResolver(org.springframework.yarn.am.container.ContainerResolver containerResolver)
containerResolver - the new container resolverprotected void waitCompleteState(StepExecution masterStepExecution)
CountDownLatch to wait completion status from
application master. Status is considered to be complete if either
master itself or parent step execution sends complete status.
masterStepExecution - the parent step execution
|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||