public class SplitterPartitionHandler extends AbstractPartitionHandler
PartitionHandler which does
partitioning based on number of input files from HDFS.| Constructor and Description |
|---|
SplitterPartitionHandler()
Instantiates a new splitter partition handler.
|
SplitterPartitionHandler(AbstractBatchAppmaster batchAppmaster)
Instantiates a new splitter partition handler.
|
| Modifier and Type | Method and Description |
|---|---|
protected Map<StepExecution,org.springframework.yarn.am.container.ContainerRequestHint> |
createRequestData(Set<StepExecution> stepExecutions)
Subclass may override this method to assign a specific
ContainerRequestHint to
a StepExecution. |
protected Set<StepExecution> |
createSplits(StepExecutionSplitter stepSplitter,
StepExecution stepExecution)
Creates the splits.
|
getKeySplitLocations, getStepName, handle, setBatchAppmaster, setKeySplitLocations, setStepName, setYarnAppmaster, waitCompleteStatepublic SplitterPartitionHandler()
public SplitterPartitionHandler(AbstractBatchAppmaster batchAppmaster)
batchAppmaster - the batch appmasterprotected Set<StepExecution> createSplits(StepExecutionSplitter stepSplitter, StepExecution stepExecution) throws Exception
AbstractPartitionHandlercreateSplits in class AbstractPartitionHandlerstepSplitter - the step splitterstepExecution - the step executionException - the exceptionprotected Map<StepExecution,org.springframework.yarn.am.container.ContainerRequestHint> createRequestData(Set<StepExecution> stepExecutions) throws Exception
AbstractPartitionHandlerContainerRequestHint to
a StepExecution. This would be needed in cases where step should be executed
in a specific host or rack considering data locality.
Default implementation returns an empty map.
createRequestData in class AbstractPartitionHandlerstepExecutions - Set of step executionsException - If error occurred