|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||
java.lang.Objectorg.springframework.yarn.batch.partition.AbstractPartitionHandler
public abstract class AbstractPartitionHandler
Base implementation of Spring Batch PartitionHandler handling
partitioning for Yarn containers.
| Constructor Summary | |
|---|---|
AbstractPartitionHandler()
Instantiates a new abstract partition handler. |
|
AbstractPartitionHandler(BatchYarnAppmaster batchAppmaster)
Instantiates a new abstract partition handler. |
|
| Method Summary | |
|---|---|
protected Map<StepExecution,org.springframework.yarn.am.container.ContainerRequestHint> |
createRequestData(Set<StepExecution> stepExecutions)
Subclass may override this method to assign a specific ContainerRequestHint to
a StepExecution. |
protected abstract Set<StepExecution> |
createSplits(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
Creates the splits. |
String |
getKeySplitLocations()
Gets the key split locations. |
String |
getStepName()
Gets the remote step name. |
Collection<StepExecution> |
handle(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
|
void |
setBatchAppmaster(BatchYarnAppmaster batchAppmaster)
Sets the batch appmaster. |
void |
setKeySplitLocations(String keySplitLocations)
The name of the key for the split locations in each ExecutionContext. |
void |
setStepName(String stepName)
Sets the remote step name. |
void |
setYarnAppmaster(org.springframework.yarn.am.YarnAppmaster yarnAppmaster)
Sets the batch appmaster. |
protected void |
waitCompleteState(StepExecution masterStepExecution)
Uses CountDownLatch to wait completion status from
application master. |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
|---|
public AbstractPartitionHandler()
public AbstractPartitionHandler(BatchYarnAppmaster batchAppmaster)
batchAppmaster - the batch appmaster| Method Detail |
|---|
public final Collection<StepExecution> handle(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
throws Exception
handle in interface PartitionHandlerExceptionpublic void setBatchAppmaster(BatchYarnAppmaster batchAppmaster)
batchAppmaster - the new batch appmastersetYarnAppmaster(YarnAppmaster)public void setYarnAppmaster(org.springframework.yarn.am.YarnAppmaster yarnAppmaster)
YarnAppmaster to be able to allow
application context to auto-wire BatchYarnAppmaster
if it's type in context is defined as YarnAppmaster.
yarnAppmaster - the new yarn appmastersetBatchAppmaster(BatchYarnAppmaster)public void setKeySplitLocations(String keySplitLocations)
ExecutionContext.
Defaults to "splitLocations".
keySplitLocations - the value of the keypublic String getKeySplitLocations()
public String getStepName()
public void setStepName(String stepName)
stepName - the new remote step name
protected abstract Set<StepExecution> createSplits(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
throws Exception
stepSplitter - the step splitterstepExecution - the step execution
Exception - the exception
protected Map<StepExecution,org.springframework.yarn.am.container.ContainerRequestHint> createRequestData(Set<StepExecution> stepExecutions)
throws Exception
ContainerRequestHint to
a StepExecution. This would be needed in cases where step should be executed
in a specific host or rack considering data locality.
Default implementation returns an empty map.
stepExecutions - Set of step executions
Exception - If error occurredprotected void waitCompleteState(StepExecution masterStepExecution)
CountDownLatch to wait completion status from
application master. Status is considered to be complete if either
master itself or parent step execution sends complete status.
masterStepExecution - the parent step execution
|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||