org.springframework.data.hadoop.mapreduce
Class JarExecutor

java.lang.Object
  extended by org.springframework.data.hadoop.mapreduce.JarExecutor
All Implemented Interfaces:
Aware, BeanClassLoaderAware, InitializingBean
Direct Known Subclasses:
JarRunner

public abstract class JarExecutor
extends Object

Customized executor for Hadoop jars.

Author:
Costin Leau

Constructor Summary
JarExecutor()
           
 
Method Summary
 void afterPropertiesSet()
           
protected  ClassLoader createClassLoaderForJar(Resource jar, ClassLoader parentCL, Configuration cfg)
           
protected  Object invokeTargetObject(Configuration cfg, Object target, Class<Object> targetClass, String[] args)
           
protected  Class<T> loadClass(String className, ClassLoader cl)
           
protected  void postExecution(Configuration cfg)
           
protected  void preExecution(Configuration cfg)
           
protected  Configuration resolveConfiguration()
           
protected  Class<T> resolveTargetClass(Configuration cfg)
           
protected  Object resolveTargetObject(Class<Object> type)
           
protected  int runCode()
           
 void setArchives(Resource... archives)
          Sets the archives to be unarchive to the map reduce cluster.
 void setArguments(String... arguments)
          Sets the arguments.
 void setBeanClassLoader(ClassLoader classLoader)
           
 void setCloseFs(boolean closeFs)
          Indicates whether or not to close the Hadoop file-systems resulting from the custom code execution.
 void setConfiguration(Configuration configuration)
          Sets the configuration.
 void setFiles(Resource... files)
          Sets the files to be copied to the map reduce cluster.
 void setJar(Resource jar)
          Sets the target code jar.
 void setLibs(Resource... libJars)
          Sets the jar files to include in the classpath.
 void setMainClass(String className)
          Sets the target class by name.
 void setProperties(Properties properties)
          Sets the properties.
 void setUser(String user)
          Sets the user impersonation (optional) for running this job.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

JarExecutor

public JarExecutor()
Method Detail

createClassLoaderForJar

protected ClassLoader createClassLoaderForJar(Resource jar,
                                              ClassLoader parentCL,
                                              Configuration cfg)

resolveTargetObject

protected Object resolveTargetObject(Class<Object> type)

invokeTargetObject

protected Object invokeTargetObject(Configuration cfg,
                                    Object target,
                                    Class<Object> targetClass,
                                    String[] args)
                             throws Exception
Throws:
Exception

preExecution

protected void preExecution(Configuration cfg)

postExecution

protected void postExecution(Configuration cfg)

afterPropertiesSet

public void afterPropertiesSet()
                        throws Exception
Specified by:
afterPropertiesSet in interface InitializingBean
Throws:
Exception

setMainClass

public void setMainClass(String className)
Sets the target class by name.

Parameters:
className - the target class name

runCode

protected int runCode()
               throws Exception
Throws:
Exception

resolveConfiguration

protected Configuration resolveConfiguration()
                                      throws Exception
Throws:
Exception

resolveTargetClass

protected Class<T> resolveTargetClass(Configuration cfg)
                               throws Exception
Throws:
Exception

loadClass

protected Class<T> loadClass(String className,
                             ClassLoader cl)

setJar

public void setJar(Resource jar)
Sets the target code jar.

Parameters:
jar - target jar

setArguments

public void setArguments(String... arguments)
Sets the arguments.

Parameters:
arguments - The arguments to set.

setConfiguration

public void setConfiguration(Configuration configuration)
Sets the configuration.

Parameters:
configuration - The configuration to set.

setProperties

public void setProperties(Properties properties)
Sets the properties.

Parameters:
properties - The properties to set.

setBeanClassLoader

public void setBeanClassLoader(ClassLoader classLoader)
Specified by:
setBeanClassLoader in interface BeanClassLoaderAware

setCloseFs

public void setCloseFs(boolean closeFs)
Indicates whether or not to close the Hadoop file-systems resulting from the custom code execution. Default is true. Turn this to false if the code reuses the same file-system used by the rest of the application.

Parameters:
closeFs - the new close fs

setLibs

public void setLibs(Resource... libJars)
Sets the jar files to include in the classpath. Note that a pattern can be used (e.g. mydir/*.jar), which the Spring container will automatically resolve.

Parameters:
libJars - The jar files to include in the classpath.

setFiles

public void setFiles(Resource... files)
Sets the files to be copied to the map reduce cluster. Note that a pattern can be used (e.g. mydir/*.txt), which the Spring container will automatically resolve.

Parameters:
files - The files to copy.

setArchives

public void setArchives(Resource... archives)
Sets the archives to be unarchive to the map reduce cluster. Note that a pattern can be used (e.g. mydir/*.zip), which the Spring container will automatically resolve.

Parameters:
archives - The archives to unarchive on the compute machines.

setUser

public void setUser(String user)
Sets the user impersonation (optional) for running this job. Should be used when running against a Hadoop Kerberos cluster.

Parameters:
user - user/group information