AdpEmrActivity

Instance Constructors

new AdpEmrActivity(id: String, name: Option[String], step: Seq[String], preStepCommand: Option[Seq[String]], postStepCommand: Option[Seq[String]], input: Option[Seq[AdpRef[AdpDataNode]]], output: Option[Seq[AdpRef[AdpDataNode]]], workerGroup: Option[String], runsOn: Option[AdpRef[AdpEmrCluster]], dependsOn: Option[Seq[AdpRef[AdpActivity]]], precondition: Option[Seq[AdpRef[AdpPrecondition]]], onFail: Option[Seq[AdpRef[AdpSnsAlarm]]], onSuccess: Option[Seq[AdpRef[AdpSnsAlarm]]], onLateAction: Option[Seq[AdpRef[AdpSnsAlarm]]], attemptTimeout: Option[String], lateAfterTimeout: Option[String], maximumRetries: Option[String], retryDelay: Option[String], failureAndRerunMode: Option[String], maxActiveInstances: Option[String])

step
One or more steps for the cluster to run. To specify multiple steps, up to 255, add multiple step fields. Use comma-separated arguments after the JAR name; for example, "s3://example-bucket/MyWork.jar,arg1,arg2,arg3".
preStepCommand
Shell scripts to be run before any steps are run. To specify multiple scripts, up to 255, add multiple preStepCommand fields.
postStepCommand
Shell scripts to be run after all steps are finished. To specify multiple scripts, up to 255, add multiple postStepCommand fields.
input
The input data source.
output
The location for the output
runsOn
The Amazon EMR cluster to run this cluster.

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
val attemptTimeout: Option[String]

The timeout time interval for an object attempt.
The timeout time interval for an object attempt. If an attempt does not complete within the start time plus this time interval, AWS Data Pipeline marks the attempt as failed and your retry settings determine the next steps taken.

Definition Classes
AdpEmrActivity → AdpActivity
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
val dependsOn: Option[Seq[AdpRef[AdpActivity]]]

One or more references to other Activities that must reach the FINISHED state before this activity will start.
One or more references to other Activities that must reach the FINISHED state before this activity will start.

Definition Classes
AdpEmrActivity → AdpActivity
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
val failureAndRerunMode: Option[String]

Determines whether pipeline object failures and rerun commands cascade through pipeline object dependencies
Determines whether pipeline object failures and rerun commands cascade through pipeline object dependencies
Possible values include cascade and none.

Definition Classes
AdpEmrActivity → AdpActivity
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
val id: String

The ID of the object, IDs must be unique within a pipeline definition
The ID of the object, IDs must be unique within a pipeline definition

Definition Classes
AdpEmrActivity → AdpDataPipelineObject → AdpObject
val input: Option[Seq[AdpRef[AdpDataNode]]]

The input data source.
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
val lateAfterTimeout: Option[String]

The time period in which the object run must start.
The time period in which the object run must start. If the object does not start within the scheduled start time plus this time interval, it is considered late

Definition Classes
AdpEmrActivity → AdpActivity
val maxActiveInstances: Option[String]

The maximum number of concurrent active instances of a component.
The maximum number of concurrent active instances of a component. Re-runs do not count toward the number of active instances.

Definition Classes
AdpEmrActivity → AdpActivity
val maximumRetries: Option[String]

The maximum number of times to retry the action.
The maximum number of times to retry the action. The default value is 2, which results in 3 tries total (1 original attempt plus 2 retries). The maximum value is 5 (6 total attempts).

Definition Classes
AdpEmrActivity → AdpActivity
val name: Option[String]

The optional, user-defined label of the object.
The optional, user-defined label of the object. If you do not provide a name for an object in a pipeline definition, AWS Data Pipeline automatically duplicates the value of id.

Definition Classes
AdpEmrActivity → AdpDataPipelineObject → AdpDataPipelineAbstractObject
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
val onFail: Option[Seq[AdpRef[AdpSnsAlarm]]]

The SNS alarm to raise when the activity fails.
The SNS alarm to raise when the activity fails.

Definition Classes
AdpEmrActivity → AdpActivity
val onLateAction: Option[Seq[AdpRef[AdpSnsAlarm]]]

The SNS alarm to raise when the activity fails to start on time.
The SNS alarm to raise when the activity fails to start on time.

Definition Classes
AdpEmrActivity → AdpActivity
val onSuccess: Option[Seq[AdpRef[AdpSnsAlarm]]]

The SNS alarm to raise when the activity succeeds.
The SNS alarm to raise when the activity succeeds.

Definition Classes
AdpEmrActivity → AdpActivity
val output: Option[Seq[AdpRef[AdpDataNode]]]

The location for the output
val postStepCommand: Option[Seq[String]]

Shell scripts to be run after all steps are finished.
Shell scripts to be run after all steps are finished. To specify multiple scripts, up to 255, add multiple postStepCommand fields.
val preStepCommand: Option[Seq[String]]

Shell scripts to be run before any steps are run.
Shell scripts to be run before any steps are run. To specify multiple scripts, up to 255, add multiple preStepCommand fields.
val precondition: Option[Seq[AdpRef[AdpPrecondition]]]

A condition that must be met before the object can run.
A condition that must be met before the object can run. To specify multiple conditions, add multiple precondition fields. The activity cannot run until all its conditions are met.

Definition Classes
AdpEmrActivity → AdpActivity
val retryDelay: Option[String]

The timeout duration between two retry attempts.
The timeout duration between two retry attempts. The default is 10 minutes.

Definition Classes
AdpEmrActivity → AdpActivity
val runsOn: Option[AdpRef[AdpEmrCluster]]

The Amazon EMR cluster to run this cluster.
val step: Seq[String]

One or more steps for the cluster to run.
One or more steps for the cluster to run. To specify multiple steps, up to 255, add multiple step fields. Use comma-separated arguments after the JAR name; for example, "s3://example-bucket/MyWork.jar,arg1,arg2,arg3".
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
val type: String

The type of object.
The type of object. Use one of the predefined AWS Data Pipeline object types.

Definition Classes
AdpEmrActivity → AdpDataPipelineObject
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
val workerGroup: Option[String]

The worker group.
The worker group. This is used for routing tasks. If you provide a runsOn value and workerGroup exists, workerGroup is ignored.

Definition Classes
AdpEmrActivity → AdpActivity

Related Doc: package aws

Instance Constructors

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

val attemptTimeout: Option[String]

def clone(): AnyRef

val dependsOn: Option[Seq[AdpRef[AdpActivity]]]

final def eq(arg0: AnyRef): Boolean

val failureAndRerunMode: Option[String]

def finalize(): Unit

final def getClass(): Class[_]

val id: String

val input: Option[Seq[AdpRef[AdpDataNode]]]

final def isInstanceOf[T0]: Boolean

val lateAfterTimeout: Option[String]

val maxActiveInstances: Option[String]

val maximumRetries: Option[String]

val name: Option[String]

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

val onFail: Option[Seq[AdpRef[AdpSnsAlarm]]]

val onLateAction: Option[Seq[AdpRef[AdpSnsAlarm]]]

val onSuccess: Option[Seq[AdpRef[AdpSnsAlarm]]]

val output: Option[Seq[AdpRef[AdpDataNode]]]

val postStepCommand: Option[Seq[String]]

val preStepCommand: Option[Seq[String]]

val precondition: Option[Seq[AdpRef[AdpPrecondition]]]

val retryDelay: Option[String]

val runsOn: Option[AdpRef[AdpEmrCluster]]

val step: Seq[String]

final def synchronized[T0](arg0: ⇒ T0): T0

val type: String

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

val workerGroup: Option[String]

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AdpActivity

Inherited from AdpDataPipelineObject

Inherited from AdpDataPipelineAbstractObject

Inherited from AdpObject

Inherited from AnyRef

Inherited from Any

Ungrouped