|
Spring for Apache Hadoop | ||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||
java.lang.Objectorg.springframework.yarn.batch.partition.AbstractPartitionHandler
public abstract class AbstractPartitionHandler
Base implementation of Spring Batch PartitionHandler handling
partitioning for Yarn containers.
| Constructor Summary | |
|---|---|
AbstractPartitionHandler()
Instantiates a new abstract partition handler. |
|
AbstractPartitionHandler(BatchYarnAppmaster batchAppmaster)
Instantiates a new abstract partition handler. |
|
| Method Summary | |
|---|---|
protected java.util.Map<org.springframework.batch.core.StepExecution,ContainerRequestHint> |
createRequestData(java.util.Set<org.springframework.batch.core.StepExecution> stepExecutions)
Subclass may override this method to assign a specific ContainerRequestHint to
a StepExecution. |
protected abstract java.util.Set<org.springframework.batch.core.StepExecution> |
createSplits(org.springframework.batch.core.partition.StepExecutionSplitter stepSplitter,
org.springframework.batch.core.StepExecution stepExecution)
Creates the splits. |
java.lang.String |
getKeySplitLocations()
Gets the key split locations. |
java.lang.String |
getStepName()
Gets the remote step name. |
java.util.Collection<org.springframework.batch.core.StepExecution> |
handle(org.springframework.batch.core.partition.StepExecutionSplitter stepSplitter,
org.springframework.batch.core.StepExecution stepExecution)
|
void |
setBatchAppmaster(BatchYarnAppmaster batchAppmaster)
Sets the batch appmaster. |
void |
setKeySplitLocations(java.lang.String keySplitLocations)
The name of the key for the split locations in each ExecutionContext. |
void |
setStepName(java.lang.String stepName)
Sets the remote step name. |
void |
setYarnAppmaster(YarnAppmaster yarnAppmaster)
Sets the batch appmaster. |
protected void |
waitCompleteState(org.springframework.batch.core.StepExecution masterStepExecution)
Uses CountDownLatch to wait completion status from
application master. |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
|---|
public AbstractPartitionHandler()
public AbstractPartitionHandler(BatchYarnAppmaster batchAppmaster)
batchAppmaster - the batch appmaster| Method Detail |
|---|
public final java.util.Collection<org.springframework.batch.core.StepExecution> handle(org.springframework.batch.core.partition.StepExecutionSplitter stepSplitter,
org.springframework.batch.core.StepExecution stepExecution)
throws java.lang.Exception
handle in interface org.springframework.batch.core.partition.PartitionHandlerjava.lang.Exceptionpublic void setBatchAppmaster(BatchYarnAppmaster batchAppmaster)
batchAppmaster - the new batch appmastersetYarnAppmaster(YarnAppmaster)@Autowired(required=false) public void setYarnAppmaster(YarnAppmaster yarnAppmaster)
YarnAppmaster to be able to allow
application context to auto-wire BatchYarnAppmaster
if it's type in context is defined as YarnAppmaster.
yarnAppmaster - the new yarn appmastersetBatchAppmaster(BatchYarnAppmaster)public void setKeySplitLocations(java.lang.String keySplitLocations)
ExecutionContext.
Defaults to "splitLocations".
keySplitLocations - the value of the keypublic java.lang.String getKeySplitLocations()
public java.lang.String getStepName()
public void setStepName(java.lang.String stepName)
stepName - the new remote step name
protected abstract java.util.Set<org.springframework.batch.core.StepExecution> createSplits(org.springframework.batch.core.partition.StepExecutionSplitter stepSplitter,
org.springframework.batch.core.StepExecution stepExecution)
throws java.lang.Exception
stepSplitter - the step splitterstepExecution - the step execution
java.lang.Exception - the exception
protected java.util.Map<org.springframework.batch.core.StepExecution,ContainerRequestHint> createRequestData(java.util.Set<org.springframework.batch.core.StepExecution> stepExecutions)
throws java.lang.Exception
ContainerRequestHint to
a StepExecution. This would be needed in cases where step should be executed
in a specific host or rack considering data locality.
Default implementation returns an empty map.
stepExecutions - Set of step executions
java.lang.Exception - If error occurredprotected void waitCompleteState(org.springframework.batch.core.StepExecution masterStepExecution)
CountDownLatch to wait completion status from
application master. Status is considered to be complete if either
master itself or parent step execution sends complete status.
masterStepExecution - the parent step execution
|
Spring for Apache Hadoop | ||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||