|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||
java.lang.Objectorg.springframework.yarn.fs.DefaultResourceLocalizer
public class DefaultResourceLocalizer
Default implementation of ResourceLocalizer which
is only capable of re-using files already in HDFS and preparing
correct parameters for created LocalResource entries.
| Constructor Summary | |
|---|---|
DefaultResourceLocalizer(Configuration configuration,
Collection<LocalResourcesFactoryBean.TransferEntry> transferEntries,
Collection<LocalResourcesFactoryBean.CopyEntry> copyEntries)
Instantiates a new default resource localizer. |
|
DefaultResourceLocalizer(Configuration configuration,
Collection<LocalResourcesFactoryBean.TransferEntry> transferEntries,
Collection<LocalResourcesFactoryBean.CopyEntry> copyEntries,
Path stagingDirectory)
Instantiates a new default resource localizer. |
|
| Method Summary | |
|---|---|
boolean |
clean()
Cleans all leftovers what has been created during the distribute process. |
void |
distribute()
If underlying implementation needs to do operations on hdfs filesystem or any other preparation work, calling of this method should make implementation ready to return resources from ResourceLocalizer.getResources()
command. |
protected void |
doFileCopy(FileSystem fs)
Do file copy. |
protected Map<String,LocalResource> |
doFileTransfer(FileSystem fs)
Gets a map of localized resources. |
Map<String,LocalResource> |
getResources()
Gets a map of LocalResource instances. |
void |
setStagingDirectory(Path stagingDirectory)
Sets the staging directory. |
void |
setStagingId(String stagingId)
Sets the staging id. |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
|---|
public DefaultResourceLocalizer(Configuration configuration,
Collection<LocalResourcesFactoryBean.TransferEntry> transferEntries,
Collection<LocalResourcesFactoryBean.CopyEntry> copyEntries)
configuration - the configurationtransferEntries - the transfer entriescopyEntries - the copy entries
public DefaultResourceLocalizer(Configuration configuration,
Collection<LocalResourcesFactoryBean.TransferEntry> transferEntries,
Collection<LocalResourcesFactoryBean.CopyEntry> copyEntries,
Path stagingDirectory)
configuration - the configurationtransferEntries - the transfer entriescopyEntries - the copy entriesstagingDirectory - the staging directory| Method Detail |
|---|
public Map<String,LocalResource> getResources()
ResourceLocalizerLocalResource instances. Underlying
instances of LocalResources needs to be fully
initialised including resource size and timestamp.
getResources in interface ResourceLocalizerLocalResource instancespublic void setStagingDirectory(Path stagingDirectory)
ResourceLocalizer
setStagingDirectory in interface ResourceLocalizerstagingDirectory - the new staging directorypublic void setStagingId(String stagingId)
ResourceLocalizerPath
set in ResourceLocalizer.setStagingDirectory(Path) to post fix unique
runtime staging path. If not set simultaneous instances of
same application may override files.
setStagingId in interface ResourceLocalizerstagingId - the new staging idpublic void distribute()
ResourceLocalizerResourceLocalizer.getResources()
command.
distribute in interface ResourceLocalizerpublic boolean clean()
ResourceLocalizer
clean in interface ResourceLocalizer
protected void doFileCopy(FileSystem fs)
throws IOException
fs - the fs
IOException - Signals that an I/O exception has occurred.
protected Map<String,LocalResource> doFileTransfer(FileSystem fs)
throws IOException,
URISyntaxException
fs - the file system
IOException - if problem occurred getting file status
URISyntaxException - if file path is wrong
|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||