class TransformExecutionStateApi extends RealTimeAware
Exposes methods and properties that are specific to the currently executing transform.
- Alphabetic
- By Inheritance
- TransformExecutionStateApi
- RealTimeAware
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new TransformExecutionStateApi(scope: PipelineExecutionScope, onMessageLogged: (String) => Unit, setCurrentTransformProgress: (Double) => Unit)
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @HotSpotIntrinsicCandidate() @native()
- val content: PipelineTransform
The properties of the transform configuration.
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @HotSpotIntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @HotSpotIntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def log(message: => Any): Unit
Appends a message to the execution log.
Appends a message to the execution log. The timestamp is automatically included.
- def manualSaveDataFrame(frame: DataFrame, part: Option[Int] = scope.parallelPart): Unit
Writes a collection of data to the underlying storage and associates it with the transform.
Writes a collection of data to the underlying storage and associates it with the transform. This is used to take control of the writing process during transform execution rather than simply writing the return value of the transform. If using this it is recommended to return an empty data frame from the transform.
- frame
The Apache Spark data frame to write.
- part
The part number to write, if relevant.
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @HotSpotIntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @HotSpotIntrinsicCandidate() @native()
- val options: Options
The collection of options configured for the transform.
- def setArtifact(name: String, content: => Array[Byte]): Unit
Stores an artifact and associates it with the transform.
Stores an artifact and associates it with the transform. This can de downloaded from the execution result.
- name
The name of the artifact.
- content
The byte content of the artifact to store.
- def setProgress(progress: Double): Unit
Update the progress value for a specific transform.
Update the progress value for a specific transform. The system switches to custom progress logging on the first call to this method, therefore one should start by calling setProgress(0.0).
- progress
A value between 0 and 1
- def state: Map[String, Any]
The state map of the transform.
The state map of the transform. Use this to persist information across transforms.
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def utcNowJava: LocalDateTime
The current UTC date and time.
The current UTC date and time.
- Definition Classes
- RealTimeAware
- def utcTodayJava: LocalDateTime
The current UTC date.
The current UTC date.
- Definition Classes
- RealTimeAware
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)