T - the type of an entity to writeK - the type of a partition keypublic abstract class AbstractPartitionDataStoreWriter<T,K> extends LifecycleObjectSupport implements PartitionDataStoreWriter<T,K>
PartitionDataStoreWriter.| Constructor and Description |
|---|
AbstractPartitionDataStoreWriter(Configuration configuration,
Path basePath,
CodecInfo codec,
PartitionStrategy<T,K> partitionStrategy)
Instantiates a new abstract data store partition writer.
|
| Modifier and Type | Method and Description |
|---|---|
void |
close() |
protected abstract DataStoreWriter<T> |
createWriter(Configuration configuration,
Path basePath,
CodecInfo codec)
Need to be implemented by a subclass for an actual writer.
|
protected void |
destroyWriter(Path path)
Destroys a writer with a given
Path if exist. |
protected void |
doStart()
Subclasses may implement this method with the start behaviour.
|
protected void |
doStop()
Subclasses may implement this method with the stop behaviour.
|
void |
flush() |
Path |
getBasePath()
Gets the base path.
|
CodecInfo |
getCodec()
Gets the codec.
|
Configuration |
getConfiguration()
Gets the hadoop configuration.
|
FileNamingStrategyFactory<FileNamingStrategy> |
getFileNamingStrategyFactory()
Gets the file naming strategy factory.
|
long |
getIdleTimeout()
Gets the idle timeout.
|
java.lang.String |
getInWritingPrefix()
Gets the in writing prefix.
|
java.lang.String |
getInWritingSuffix()
Gets the in writing suffix.
|
int |
getMaxOpenAttempts()
Gets the max open attempts.
|
RolloverStrategyFactory<RolloverStrategy> |
getRolloverStrategyFactory()
Gets the rollover strategy factory.
|
boolean |
isAppendable()
Checks if append is enabled.
|
boolean |
isOverwrite()
Checks if overwrite is enabled.
|
protected void |
onInit()
Subclasses may implement this for initialization logic.
|
void |
setAppendable(boolean append)
Set stream as append mode.
|
void |
setFileNamingStrategyFactory(FileNamingStrategyFactory<FileNamingStrategy> fileNamingStrategyFactory)
Sets the file naming strategy factory.
|
void |
setIdleTimeout(long idleTimeout)
Sets the idle timeout.
|
void |
setInWritingPrefix(java.lang.String prefix)
Sets the in writing prefix.
|
void |
setInWritingSuffix(java.lang.String suffix)
Sets the in writing suffix.
|
void |
setMaxOpenAttempts(int maxOpenAttempts)
Sets the max open attempts.
|
void |
setOverwrite(boolean overwrite)
Sets the flag indicating if written files may be overwritten.
|
void |
setRolloverStrategyFactory(RolloverStrategyFactory<RolloverStrategy> rolloverStrategyFactory)
Sets the rollover strategy factory.
|
void |
write(T entity)
Write an entity.
|
void |
write(T entity,
K partitionKey)
Write an entity with an explicit partitioning key.
|
afterPropertiesSet, getBeanFactory, getPhase, getStoreEventPublisher, getTaskExecutor, getTaskScheduler, isAutoStartup, isRunning, setAutoStartup, setBeanFactory, setPhase, setStoreEventPublisher, setTaskExecutor, setTaskScheduler, start, stop, stoppublic AbstractPartitionDataStoreWriter(Configuration configuration, Path basePath, CodecInfo codec, PartitionStrategy<T,K> partitionStrategy)
configuration - the hadoop configurationbasePath - the hdfs pathcodec - the compression codec infopartitionStrategy - the partition strategypublic void write(T entity) throws java.io.IOException
DataWriterwrite in interface DataWriter<T>java.io.IOException - if an I/O error occurspublic void flush()
throws java.io.IOException
flush in interface java.io.Flushablejava.io.IOExceptionpublic void close()
throws java.io.IOException
close in interface java.io.Closeableclose in interface java.lang.AutoCloseablejava.io.IOExceptionpublic void write(T entity, K partitionKey) throws java.io.IOException
PartitionDataStoreWriterwrite in interface PartitionDataStoreWriter<T,K>entity - the entity to writepartitionKey - the partition keyjava.io.IOException - if an I/O error occursprotected void onInit()
throws java.lang.Exception
LifecycleObjectSupportInitializingBean phase.
Implementor should always call super method not to break initialization chain.onInit in class LifecycleObjectSupportjava.lang.Exceptionprotected void doStart()
LifecycleObjectSupportLifecycleObjectSupport.lifecycleLock.doStart in class LifecycleObjectSupportprotected void doStop()
LifecycleObjectSupportLifecycleObjectSupport.lifecycleLock.doStop in class LifecycleObjectSupportpublic void setFileNamingStrategyFactory(FileNamingStrategyFactory<FileNamingStrategy> fileNamingStrategyFactory)
fileNamingStrategyFactory - the new file naming strategy factorypublic FileNamingStrategyFactory<FileNamingStrategy> getFileNamingStrategyFactory()
public void setRolloverStrategyFactory(RolloverStrategyFactory<RolloverStrategy> rolloverStrategyFactory)
rolloverStrategyFactory - the new rollover strategy factorypublic RolloverStrategyFactory<RolloverStrategy> getRolloverStrategyFactory()
public void setIdleTimeout(long idleTimeout)
idleTimeout - the new idle timeoutpublic void setInWritingSuffix(java.lang.String suffix)
suffix - the new in writing suffixpublic java.lang.String getInWritingSuffix()
public void setInWritingPrefix(java.lang.String prefix)
prefix - the new in writing prefixpublic java.lang.String getInWritingPrefix()
public void setOverwrite(boolean overwrite)
FALSE meaning StoreException
is thrown if file is about to get overwritten.overwrite - the new overwritepublic boolean isOverwrite()
setOverwrite(boolean)public boolean isAppendable()
public void setAppendable(boolean append)
append - the append flagpublic long getIdleTimeout()
public Configuration getConfiguration()
public Path getBasePath()
public CodecInfo getCodec()
public void setMaxOpenAttempts(int maxOpenAttempts)
maxOpenAttempts - the new max open attemptspublic int getMaxOpenAttempts()
protected abstract DataStoreWriter<T> createWriter(Configuration configuration, Path basePath, CodecInfo codec)
configuration - the configurationbasePath - the base pathcodec - the codec