|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||
java.lang.Objectorg.springframework.yarn.batch.partition.AbstractBatchPartitionHandler
public abstract class AbstractBatchPartitionHandler
Base implementation of Spring Batch PartitionHandler handling
partitioning for Yarn containers.
| Constructor Summary | |
|---|---|
AbstractBatchPartitionHandler(AbstractBatchAppmaster batchAppmaster)
Instantiates a new batch partition handler. |
|
| Method Summary | |
|---|---|
protected Map<StepExecution,org.springframework.yarn.am.container.ContainerRequestHint> |
createResourceRequestData(Set<StepExecution> stepExecutions)
Subclass may override this method to assign a specific ContainerRequestHint to
a StepExecution. |
protected abstract Set<StepExecution> |
createStepExecutionSplits(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
|
org.springframework.yarn.am.container.ContainerResolver |
getContainerResolver()
Gets the container resolver. |
String |
getStepName()
Gets the step name. |
Collection<StepExecution> |
handle(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
|
void |
setContainerResolver(org.springframework.yarn.am.container.ContainerResolver containerResolver)
Sets the container resolver. |
void |
setStepName(String stepName)
Sets the step name. |
protected void |
waitCompleteState(StepExecution masterStepExecution)
Uses CountDownLatch to wait completion status from
application master. |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
|---|
public AbstractBatchPartitionHandler(AbstractBatchAppmaster batchAppmaster)
batchAppmaster - the batch appmaster| Method Detail |
|---|
protected abstract Set<StepExecution> createStepExecutionSplits(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
throws Exception
Exception
protected Map<StepExecution,org.springframework.yarn.am.container.ContainerRequestHint> createResourceRequestData(Set<StepExecution> stepExecutions)
throws Exception
ContainerRequestHint to
a StepExecution. This would be needed in cases where step should be executed
in a specific host or rack considering data locality. Default implementation
returns an empty map.
stepExecutions - Set of step executions
Exception - If error occurred
public final Collection<StepExecution> handle(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
throws Exception
handle in interface PartitionHandlerExceptionpublic String getStepName()
public void setStepName(String stepName)
stepName - the new step namepublic org.springframework.yarn.am.container.ContainerResolver getContainerResolver()
public void setContainerResolver(org.springframework.yarn.am.container.ContainerResolver containerResolver)
containerResolver - the new container resolverprotected void waitCompleteState(StepExecution masterStepExecution)
CountDownLatch to wait completion status from
application master. Status is considered to be complete if either
master itself or parent step execution sends complete status.
masterStepExecution - the parent step execution
|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||