|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||
java.lang.Objectorg.springframework.yarn.batch.partition.AbstractPartitionHandler
org.springframework.yarn.batch.partition.SplitterPartitionHandler
public class SplitterPartitionHandler
Implementation of Spring Batch PartitionHandler which does
partitioning based on number of input files from HDFS.
| Constructor Summary | |
|---|---|
SplitterPartitionHandler()
Instantiates a new splitter partition handler. |
|
SplitterPartitionHandler(AbstractBatchAppmaster batchAppmaster)
Instantiates a new splitter partition handler. |
|
| Method Summary | |
|---|---|
protected Map<StepExecution,org.springframework.yarn.am.container.ContainerRequestHint> |
createRequestData(Set<StepExecution> stepExecutions)
Subclass may override this method to assign a specific ContainerRequestHint to
a StepExecution. |
protected Set<StepExecution> |
createSplits(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
Creates the splits. |
| Methods inherited from class org.springframework.yarn.batch.partition.AbstractPartitionHandler |
|---|
getKeySplitLocations, getStepName, handle, setBatchAppmaster, setKeySplitLocations, setStepName, setYarnAppmaster, waitCompleteState |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
|---|
public SplitterPartitionHandler()
public SplitterPartitionHandler(AbstractBatchAppmaster batchAppmaster)
batchAppmaster - the batch appmaster| Method Detail |
|---|
protected Set<StepExecution> createSplits(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
throws Exception
AbstractPartitionHandler
createSplits in class AbstractPartitionHandlerstepSplitter - the step splitterstepExecution - the step execution
Exception - the exception
protected Map<StepExecution,org.springframework.yarn.am.container.ContainerRequestHint> createRequestData(Set<StepExecution> stepExecutions)
throws Exception
AbstractPartitionHandlerContainerRequestHint to
a StepExecution. This would be needed in cases where step should be executed
in a specific host or rack considering data locality.
Default implementation returns an empty map.
createRequestData in class AbstractPartitionHandlerstepExecutions - Set of step executions
Exception - If error occurred
|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||