org.apache.oozie.command.coord
Class CoordSubmitXCommand

java.lang.Object
  extended by org.apache.oozie.command.XCommand<T>
      extended by org.apache.oozie.command.TransitionXCommand<String>
          extended by org.apache.oozie.command.SubmitTransitionXCommand
              extended by org.apache.oozie.command.coord.CoordSubmitXCommand
All Implemented Interfaces:
Callable<String>, XCallable<String>
Direct Known Subclasses:
CoordUpdateXCommand

public class CoordSubmitXCommand
extends SubmitTransitionXCommand

This class provides the functionalities to resolve a coordinator job XML and write the job information into a DB table.

Specifically it performs the following functions: 1. Resolve all the variables or properties using job configurations. 2. Insert all datasets definition as part of the and tags. 3. Validate the XML at runtime.


Field Summary
protected  org.apache.hadoop.conf.Configuration conf
           
static String CONF_CHECK_MAX_FREQUENCY
           
static String CONF_DEFAULT_CONCURRENCY
           
static String CONF_DEFAULT_MAX_TIMEOUT
          Default MAX timeout in minutes, after which coordinator input check will timeout
static String CONF_DEFAULT_THROTTLE
           
static String CONF_DEFAULT_TIMEOUT_NORMAL
          Default timeout for normal jobs, in minutes, after which coordinator input check will timeout
static String CONF_MAT_THROTTLING_FACTOR
           
static String CONF_QUEUE_SIZE
           
static String CONFIG_DEFAULT
           
 String COORD_INPUT_EVENTS
           
 String COORD_INPUT_EVENTS_DATA_IN
           
 String COORD_OUTPUT_EVENTS
           
 String COORD_OUTPUT_EVENTS_DATA_OUT
           
static String COORDINATOR_XML_FILE
           
protected  CoordinatorJobBean coordJob
           
protected  boolean dryrun
           
protected  JPAService jpaService
           
 
Fields inherited from class org.apache.oozie.command.TransitionXCommand
insertList, job, updateList
 
Fields inherited from class org.apache.oozie.command.XCommand
DEFAULT_LOCK_TIMEOUT, DEFAULT_REQUEUE_DELAY, eventService, instrumentation, INSTRUMENTATION_GROUP, LOG, logInfo
 
Constructor Summary
CoordSubmitXCommand(boolean dryrun, org.apache.hadoop.conf.Configuration conf)
          Constructor to create the Coordinator Submit Command.
CoordSubmitXCommand(org.apache.hadoop.conf.Configuration conf)
          Constructor to create the Coordinator Submit Command.
CoordSubmitXCommand(org.apache.hadoop.conf.Configuration conf, String bundleId, String coordName)
          Constructor to create the Coordinator Submit Command by bundle job.
 
Method Summary
 org.jdom.Element basicResolveAndIncludeDS(String appXml, org.apache.hadoop.conf.Configuration conf, CoordinatorJobBean coordJob)
          The method resolve all the variables that are defined in configuration.
protected  String getDryRun(CoordinatorJobBean coordJob)
          Gets the dryrun output.
 String getEntityKey()
          Return the entity key for the command.
 org.apache.oozie.client.Job getJob()
          Get the Job for the command.
protected  void includeDataSets(org.jdom.Element resolvedXml, org.apache.hadoop.conf.Configuration conf)
          Include referred datasets into XML.
protected  void initEvaluators()
          Initialize all the required EL Evaluators.
protected  boolean isLockRequired()
          Indicate if the the command requires locking.
protected  void loadState()
          Load the necessary state to perform the precondition check and to execute the command.
protected  void mergeDefaultConfig()
          Merge default configuration with user-defined configuration.
 void notifyParent()
          This will be used to notify the parent about the status of that perticular job.
 void performWrites()
          This will be used to perform atomically all the writes within this command.
protected  void queueMaterializeTransitionXCommand(String jobId)
          Queue MaterializeTransitionXCommand
protected  String readAndValidateXml()
          Read the application XML and validate against coordinator Schema
protected  String readDefinition(String appPath)
          Read coordinator definition.
protected  org.jdom.Element resolveInitial(org.apache.hadoop.conf.Configuration conf, String appXml, CoordinatorJobBean coordJob)
          Resolve basic entities using job Configuration.
protected  String storeToDB(String appXML, org.jdom.Element eJob, CoordinatorJobBean coordJob)
          Write a coordinator job into database
protected  String submit()
          Submit the job
protected  String submitJob()
           
 void updateJob()
          Update the parent job.
protected  void verifyPrecondition()
          Verify the precondition for the command after a lock has been obtain, just before executing the command.
 
Methods inherited from class org.apache.oozie.command.SubmitTransitionXCommand
execute, transitToNext
 
Methods inherited from class org.apache.oozie.command.TransitionXCommand
generateEvents, setJob
 
Methods inherited from class org.apache.oozie.command.XCommand
call, call, eagerLoadState, eagerVerifyPrecondition, executeInterrupts, getCreatedTime, getInstrumentation, getKey, getLockTimeOut, getLog, getName, getPriority, getRequeueDelay, getType, inInterruptMode, isReQueueRequired, queue, queue, resetUsed, setInterruptMode, toString
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

conf

protected org.apache.hadoop.conf.Configuration conf

dryrun

protected boolean dryrun

jpaService

protected JPAService jpaService

CONFIG_DEFAULT

public static final String CONFIG_DEFAULT
See Also:
Constant Field Values

COORDINATOR_XML_FILE

public static final String COORDINATOR_XML_FILE
See Also:
Constant Field Values

COORD_INPUT_EVENTS

public final String COORD_INPUT_EVENTS
See Also:
Constant Field Values

COORD_OUTPUT_EVENTS

public final String COORD_OUTPUT_EVENTS
See Also:
Constant Field Values

COORD_INPUT_EVENTS_DATA_IN

public final String COORD_INPUT_EVENTS_DATA_IN
See Also:
Constant Field Values

COORD_OUTPUT_EVENTS_DATA_OUT

public final String COORD_OUTPUT_EVENTS_DATA_OUT
See Also:
Constant Field Values

coordJob

protected CoordinatorJobBean coordJob

CONF_DEFAULT_TIMEOUT_NORMAL

public static final String CONF_DEFAULT_TIMEOUT_NORMAL
Default timeout for normal jobs, in minutes, after which coordinator input check will timeout

See Also:
Constant Field Values

CONF_DEFAULT_CONCURRENCY

public static final String CONF_DEFAULT_CONCURRENCY
See Also:
Constant Field Values

CONF_DEFAULT_THROTTLE

public static final String CONF_DEFAULT_THROTTLE
See Also:
Constant Field Values

CONF_MAT_THROTTLING_FACTOR

public static final String CONF_MAT_THROTTLING_FACTOR
See Also:
Constant Field Values

CONF_DEFAULT_MAX_TIMEOUT

public static final String CONF_DEFAULT_MAX_TIMEOUT
Default MAX timeout in minutes, after which coordinator input check will timeout

See Also:
Constant Field Values

CONF_QUEUE_SIZE

public static final String CONF_QUEUE_SIZE
See Also:
Constant Field Values

CONF_CHECK_MAX_FREQUENCY

public static final String CONF_CHECK_MAX_FREQUENCY
See Also:
Constant Field Values
Constructor Detail

CoordSubmitXCommand

public CoordSubmitXCommand(org.apache.hadoop.conf.Configuration conf)
Constructor to create the Coordinator Submit Command.

Parameters:
conf - : Configuration for Coordinator job

CoordSubmitXCommand

public CoordSubmitXCommand(org.apache.hadoop.conf.Configuration conf,
                           String bundleId,
                           String coordName)
Constructor to create the Coordinator Submit Command by bundle job.

Parameters:
conf - : Configuration for Coordinator job
bundleId - : bundle id
coordName - : coord name

CoordSubmitXCommand

public CoordSubmitXCommand(boolean dryrun,
                           org.apache.hadoop.conf.Configuration conf)
Constructor to create the Coordinator Submit Command.

Parameters:
dryrun - : if dryrun
conf - : Configuration for Coordinator job
Method Detail

submit

protected String submit()
                 throws CommandException
Description copied from class: SubmitTransitionXCommand
Submit the job

Specified by:
submit in class SubmitTransitionXCommand
Returns:
the id
Throws:
CommandException - thrown if unable to submit

submitJob

protected String submitJob()
                    throws CommandException
Throws:
CommandException

getDryRun

protected String getDryRun(CoordinatorJobBean coordJob)
                    throws Exception
Gets the dryrun output.

Parameters:
jobId - the job id
Returns:
the dry run
Throws:
Exception - the exception

queueMaterializeTransitionXCommand

protected void queueMaterializeTransitionXCommand(String jobId)
Queue MaterializeTransitionXCommand


readAndValidateXml

protected String readAndValidateXml()
                             throws CoordinatorJobException
Read the application XML and validate against coordinator Schema

Returns:
validated coordinator XML
Throws:
CoordinatorJobException - thrown if unable to read or validate coordinator xml

mergeDefaultConfig

protected void mergeDefaultConfig()
                           throws CommandException
Merge default configuration with user-defined configuration.

Throws:
CommandException - thrown if failed to read or merge configurations

basicResolveAndIncludeDS

public org.jdom.Element basicResolveAndIncludeDS(String appXml,
                                                 org.apache.hadoop.conf.Configuration conf,
                                                 CoordinatorJobBean coordJob)
                                          throws CoordinatorJobException,
                                                 Exception
The method resolve all the variables that are defined in configuration. It also include the data set definition from dataset file into XML.

Parameters:
appXml - : Original job XML
conf - : Configuration of the job
coordJob - : Coordinator job bean to be populated.
Returns:
Resolved and modified job XML element.
Throws:
CoordinatorJobException - thrown if failed to resolve basic entities or include referred datasets
Exception - thrown if failed to resolve basic entities or include referred datasets

initEvaluators

protected void initEvaluators()
Initialize all the required EL Evaluators.


resolveInitial

protected org.jdom.Element resolveInitial(org.apache.hadoop.conf.Configuration conf,
                                          String appXml,
                                          CoordinatorJobBean coordJob)
                                   throws CoordinatorJobException,
                                          Exception
Resolve basic entities using job Configuration.

Parameters:
conf - :Job configuration
appXml - : Original job XML
coordJob - : Coordinator job bean to be populated.
Returns:
Resolved job XML element.
Throws:
CoordinatorJobException - thrown if failed to resolve basic entities
Exception - thrown if failed to resolve basic entities

includeDataSets

protected void includeDataSets(org.jdom.Element resolvedXml,
                               org.apache.hadoop.conf.Configuration conf)
                        throws CoordinatorJobException
Include referred datasets into XML.

Parameters:
resolvedXml - : Job XML element.
conf - : Job configuration
Throws:
CoordinatorJobException - thrown if failed to include referred datasets into XML

readDefinition

protected String readDefinition(String appPath)
                         throws CoordinatorJobException
Read coordinator definition.

Parameters:
appPath - application path.
Returns:
coordinator definition.
Throws:
CoordinatorJobException - thrown if the definition could not be read.

storeToDB

protected String storeToDB(String appXML,
                           org.jdom.Element eJob,
                           CoordinatorJobBean coordJob)
                    throws CommandException
Write a coordinator job into database

Parameters:
appXML - : Coordinator definition xml
eJob - : XML element of job
coordJob - : Coordinator job bean
Returns:
Job id
Throws:
CommandException - thrown if unable to save coordinator job to db

getEntityKey

public String getEntityKey()
Description copied from class: XCommand
Return the entity key for the command.

Specified by:
getEntityKey in interface XCallable<String>
Specified by:
getEntityKey in class XCommand<String>
Returns:
the entity key for the command.

isLockRequired

protected boolean isLockRequired()
Description copied from class: XCommand
Indicate if the the command requires locking.

Subclasses should override this method if they require locking.

Specified by:
isLockRequired in class XCommand<String>
Returns:
true/false

loadState

protected void loadState()
                  throws CommandException
Description copied from class: XCommand
Load the necessary state to perform the precondition check and to execute the command.

Subclasses must implement this method and load the state needed to do the precondition check and execute the command.

Specified by:
loadState in class XCommand<String>
Throws:
CommandException

verifyPrecondition

protected void verifyPrecondition()
                           throws CommandException
Description copied from class: XCommand
Verify the precondition for the command after a lock has been obtain, just before executing the command.

Specified by:
verifyPrecondition in class XCommand<String>
Throws:
CommandException - thrown if the precondition is not met.

notifyParent

public void notifyParent()
                  throws CommandException
Description copied from class: TransitionXCommand
This will be used to notify the parent about the status of that perticular job.

Specified by:
notifyParent in class TransitionXCommand<String>
Throws:
CommandException

updateJob

public void updateJob()
               throws CommandException
Description copied from class: TransitionXCommand
Update the parent job.

Specified by:
updateJob in class TransitionXCommand<String>
Throws:
CommandException

getJob

public org.apache.oozie.client.Job getJob()
Description copied from class: TransitionXCommand
Get the Job for the command.

Overrides:
getJob in class TransitionXCommand<String>
Returns:
the job

performWrites

public void performWrites()
                   throws CommandException
Description copied from class: TransitionXCommand
This will be used to perform atomically all the writes within this command.

Specified by:
performWrites in class TransitionXCommand<String>
Throws:
CommandException


Copyright © 2015 Apache Software Foundation. All Rights Reserved.