|
Spring for Apache Hadoop | ||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||
java.lang.Objectorg.springframework.yarn.fs.AbstractResourceLocalizer
org.springframework.yarn.fs.DefaultResourceLocalizer
public class DefaultResourceLocalizer
Default implementation of ResourceLocalizer which
is only capable of re-using files already in HDFS and preparing
correct parameters for created LocalResource entries.
| Constructor Summary | |
|---|---|
DefaultResourceLocalizer(org.apache.hadoop.conf.Configuration configuration,
java.util.Collection<LocalResourcesFactoryBean.TransferEntry> transferEntries,
java.util.Collection<LocalResourcesFactoryBean.CopyEntry> copyEntries)
Instantiates a new default resource localizer. |
|
DefaultResourceLocalizer(org.apache.hadoop.conf.Configuration configuration,
java.util.Collection<LocalResourcesFactoryBean.TransferEntry> transferEntries,
java.util.Collection<LocalResourcesFactoryBean.CopyEntry> copyEntries,
org.apache.hadoop.fs.Path stagingDirectory)
Instantiates a new default resource localizer. |
|
| Method Summary | |
|---|---|
boolean |
AddRawContent(java.lang.String key,
byte[] value)
Adds a content into a to be written entries. |
protected void |
doFileCopy(org.apache.hadoop.fs.FileSystem fs)
Do file copy. |
protected java.util.Map<java.lang.String,org.apache.hadoop.yarn.api.records.LocalResource> |
doFileTransfer(org.apache.hadoop.fs.FileSystem fs)
Do file transfer. |
void |
setRawFileContents(java.util.Map<java.lang.String,byte[]> rawFileContents)
Sets the raw file contents. |
| Methods inherited from class org.springframework.yarn.fs.AbstractResourceLocalizer |
|---|
clean, copy, deleteStagingEntries, distribute, getConfiguration, getLock, getResources, isCopied, isDistributed, resolve, resolveStagingDirectory, setCopied, setDistributed, setStagingDirectory, setStagingId |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Methods inherited from interface org.springframework.yarn.fs.SmartResourceLocalizer |
|---|
clean, copy, distribute, resolve, setStagingDirectory, setStagingId |
| Methods inherited from interface org.springframework.yarn.fs.ResourceLocalizer |
|---|
getResources |
| Constructor Detail |
|---|
public DefaultResourceLocalizer(org.apache.hadoop.conf.Configuration configuration,
java.util.Collection<LocalResourcesFactoryBean.TransferEntry> transferEntries,
java.util.Collection<LocalResourcesFactoryBean.CopyEntry> copyEntries)
configuration - the configurationtransferEntries - the transfer entriescopyEntries - the copy entries
public DefaultResourceLocalizer(org.apache.hadoop.conf.Configuration configuration,
java.util.Collection<LocalResourcesFactoryBean.TransferEntry> transferEntries,
java.util.Collection<LocalResourcesFactoryBean.CopyEntry> copyEntries,
org.apache.hadoop.fs.Path stagingDirectory)
configuration - the configurationtransferEntries - the transfer entriescopyEntries - the copy entriesstagingDirectory - the staging directory| Method Detail |
|---|
public boolean AddRawContent(java.lang.String key,
byte[] value)
key - the key considered as a file namevalue - the content of a file to be written
public void setRawFileContents(java.util.Map<java.lang.String,byte[]> rawFileContents)
rawFileContents - the raw file contents
protected void doFileCopy(org.apache.hadoop.fs.FileSystem fs)
throws java.lang.Exception
AbstractResourceLocalizer
doFileCopy in class AbstractResourceLocalizerfs - the fs
java.lang.Exception - the exception
protected java.util.Map<java.lang.String,org.apache.hadoop.yarn.api.records.LocalResource> doFileTransfer(org.apache.hadoop.fs.FileSystem fs)
throws java.lang.Exception
AbstractResourceLocalizer
doFileTransfer in class AbstractResourceLocalizerfs - the fs
java.lang.Exception - the exception
|
Spring for Apache Hadoop | ||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||