|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||
java.lang.Objectorg.springframework.core.io.DefaultResourceLoader
org.springframework.data.hadoop.fs.HdfsResourceLoader
public class HdfsResourceLoader
Spring ResourceLoader over Hadoop FileSystem.
| Nested Class Summary |
|---|
| Nested classes/interfaces inherited from class org.springframework.core.io.DefaultResourceLoader |
|---|
DefaultResourceLoader.ClassPathContextResource |
| Field Summary |
|---|
| Fields inherited from interface org.springframework.core.io.support.ResourcePatternResolver |
|---|
CLASSPATH_ALL_URL_PREFIX |
| Fields inherited from interface org.springframework.core.io.ResourceLoader |
|---|
CLASSPATH_URL_PREFIX |
| Fields inherited from interface org.springframework.core.Ordered |
|---|
HIGHEST_PRECEDENCE, LOWEST_PRECEDENCE |
| Constructor Summary | |
|---|---|
HdfsResourceLoader(Configuration config)
Constructs a new HdfsResourceLoader instance. |
|
HdfsResourceLoader(Configuration config,
URI uri)
Constructs a new HdfsResourceLoader instance. |
|
HdfsResourceLoader(Configuration config,
URI uri,
String user)
Constructs a new HdfsResourceLoader instance. |
|
HdfsResourceLoader(FileSystem fs)
Constructs a new HdfsResourceLoader instance. |
|
| Method Summary | |
|---|---|
void |
afterPropertiesSet()
|
void |
close()
|
void |
destroy()
|
protected String |
determineRootDir(String location)
Determine the root directory for the given location. |
protected Set<Resource> |
doFindMatchingFileSystemResources(Path rootDir,
String subPattern)
Find all resources in the file system that match the given location pattern via the Ant-style PathMatcher. |
protected Set<Resource> |
doFindPathMatchingFileResources(Resource rootDirResource,
String subPattern)
Find all resources in the hdfs file system that match the given location pattern via the Ant-style PathMatcher. |
protected void |
doRetrieveMatchingFiles(String fullPattern,
Path dir,
Set<Path> result)
Recursively retrieve files that match the given pattern, adding them to the given result list. |
protected Resource[] |
findPathMatchingResources(String locationPattern)
Find all resources that match the given location pattern via the Ant-style PathMatcher. |
ClassLoader |
getClassLoader()
|
FileSystem |
getFileSystem()
Returns the Hadoop file system used by this resource loader. |
int |
getOrder()
|
Resource |
getResource(String location)
|
protected Resource |
getResourceByPath(String path)
|
Resource[] |
getResources(String locationPattern)
|
protected Set<Path> |
retrieveMatchingFiles(Path rootDir,
String pattern)
Retrieve files that match the given path pattern, checking the given directory and its subdirectories. |
void |
setHandleNoprefix(boolean handleNoprefix)
Sets the handle noprefix. |
void |
setResourcePatternResolver(ResourcePatternResolver resourcePatternResolver)
Sets the resource pattern resolver. |
void |
setUseCodecs(boolean useCodecs)
Indicates whether to use (or not) the codecs found inside the Hadoop configuration. |
| Methods inherited from class org.springframework.core.io.DefaultResourceLoader |
|---|
setClassLoader |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
|---|
public HdfsResourceLoader(Configuration config)
HdfsResourceLoader instance.
config - Hadoop configuration to use.
public HdfsResourceLoader(Configuration config,
URI uri,
String user)
HdfsResourceLoader instance.
config - Hadoop configuration to use.uri - Hadoop file system URI.user - Hadoop user for accessing the file system.
public HdfsResourceLoader(Configuration config,
URI uri)
HdfsResourceLoader instance.
config - Hadoop configuration to use.uri - Hadoop file system URI.public HdfsResourceLoader(FileSystem fs)
HdfsResourceLoader instance.
fs - Hadoop file system to use.| Method Detail |
|---|
protected Resource getResourceByPath(String path)
getResourceByPath in class DefaultResourceLoaderpublic Resource getResource(String location)
getResource in interface ResourceLoadergetResource in class DefaultResourceLoader
public Resource[] getResources(String locationPattern)
throws IOException
getResources in interface ResourcePatternResolverIOExceptionpublic int getOrder()
getOrder in interface Ordered
public void destroy()
throws IOException
destroy in interface DisposableBeanIOException
public void close()
throws IOException
close in interface CloseableIOException
public void afterPropertiesSet()
throws Exception
afterPropertiesSet in interface InitializingBeanExceptionpublic ClassLoader getClassLoader()
getClassLoader in interface ResourceLoadergetClassLoader in class DefaultResourceLoaderpublic void setHandleNoprefix(boolean handleNoprefix)
handleNoprefix - the new handle noprefixpublic FileSystem getFileSystem()
public void setUseCodecs(boolean useCodecs)
useCodecs - whether to use any codecs defined in the Hadoop configurationpublic void setResourcePatternResolver(ResourcePatternResolver resourcePatternResolver)
resourcePatternResolver - the new resource pattern resolver
protected Resource[] findPathMatchingResources(String locationPattern)
throws IOException
locationPattern - the location pattern to match
IOException - in case of I/O errors
protected Set<Resource> doFindPathMatchingFileResources(Resource rootDirResource,
String subPattern)
throws IOException
rootDirResource - the root directory as ResourcesubPattern - the sub pattern to match (below the root directory)
IOException - in case of I/O errors
protected Set<Resource> doFindMatchingFileSystemResources(Path rootDir,
String subPattern)
throws IOException
rootDir - the root directory in the file systemsubPattern - the sub pattern to match (below the root directory)
IOException - in case of I/O errorsPathMatcher
protected Set<Path> retrieveMatchingFiles(Path rootDir,
String pattern)
throws IOException
rootDir - the directory to start frompattern - the pattern to match against, * relative to the root directory
IOException - if directory contents could not be retrieved
protected void doRetrieveMatchingFiles(String fullPattern,
Path dir,
Set<Path> result)
throws IOException
fullPattern - the pattern to match against, with prepended root directory pathdir - the current directoryresult - the Set of matching File instances to add to
IOException - if directory contents could not be retrievedprotected String determineRootDir(String location)
Used for determining the starting point for file matching,
resolving the root directory location and passing it
into doFindPathMatchingPathResources, with the
remainder of the location as pattern.
Will return "/dir/" for the pattern "/dir/*.xml", for example.
location - the location to check
|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||