software.amazon.awscdk.services.stepfunctions.tasks

Interface SparkSubmitJobDriver

All Superinterfaces:

software.amazon.jsii.JsiiSerializable

All Known Implementing Classes:

SparkSubmitJobDriver.Jsii$Proxy

@Generated(value="jsii-pacmak/1.72.0 (build 4b8828b)",
           date="2023-01-03T16:22:40.738Z")
 @Stability(value=Stable)
public interface SparkSubmitJobDriver
extends software.amazon.jsii.JsiiSerializable

The information about job driver for Spark submit.

Example:

 EmrContainersStartJobRun.Builder.create(this, "EMR Containers Start Job Run")
         .virtualCluster(VirtualClusterInput.fromVirtualClusterId("de92jdei2910fwedz"))
         .releaseLabel(ReleaseLabel.EMR_6_2_0)
         .jobName("EMR-Containers-Job")
         .jobDriver(JobDriver.builder()
                 .sparkSubmitJobDriver(SparkSubmitJobDriver.builder()
                         .entryPoint(TaskInput.fromText("local:///usr/lib/spark/examples/src/main/python/pi.py"))
                         .build())
                 .build())
         .applicationConfig(List.of(ApplicationConfiguration.builder()
                 .classification(Classification.SPARK_DEFAULTS)
                 .properties(Map.of(
                         "spark.executor.instances", "1",
                         "spark.executor.memory", "512M"))
                 .build()))
         .build();
 

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	SparkSubmitJobDriver.Builder A builder for SparkSubmitJobDriver
static class	SparkSubmitJobDriver.Jsii$Proxy An implementation for SparkSubmitJobDriver

Method Summary

Modifier and Type	Method and Description
static SparkSubmitJobDriver.Builder	builder()
TaskInput	getEntryPoint() The entry point of job application.
default TaskInput	getEntryPointArguments() The arguments for a job application in a task input object containing an array of strings.
default String	getSparkSubmitParameters() The Spark submit parameters that are used for job runs.

Methods inherited from interface software.amazon.jsii.JsiiSerializable

$jsii$toJson

Method Detail

getEntryPoint

@Stability(value=Stable)
 @NotNull
TaskInput getEntryPoint()

The entry point of job application.

Length Constraints: Minimum length of 1. Maximum length of 256.

getEntryPointArguments

@Stability(value=Stable)
 @Nullable
default TaskInput getEntryPointArguments()

The arguments for a job application in a task input object containing an array of strings.

Length Constraints: Minimum length of 1. Maximum length of 10280.

Default: - No arguments defined

getSparkSubmitParameters

@Stability(value=Stable)
 @Nullable
default String getSparkSubmitParameters()

The Spark submit parameters that are used for job runs.

Length Constraints: Minimum length of 1. Maximum length of 102400.

Default: - No spark submit parameters

builder

@Stability(value=Stable)
static SparkSubmitJobDriver.Builder builder()

Returns:

a SparkSubmitJobDriver.Builder of SparkSubmitJobDriver