Packages

c

com.datasolvr.services.spark

TransformExecutionStateApi

class TransformExecutionStateApi extends RealTimeAware

Exposes methods and properties that are specific to the currently executing transform.

Linear Supertypes
RealTimeAware, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. TransformExecutionStateApi
  2. RealTimeAware
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new TransformExecutionStateApi(scope: PipelineExecutionScope, onMessageLogged: (String) ⇒ Unit, setCurrentTransformProgress: (Double) ⇒ Unit)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @HotSpotIntrinsicCandidate()
  6. val content: PipelineTransform

    The properties of the transform configuration.

  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. def log(message: ⇒ Any): Unit

    Appends a message to the execution log.

    Appends a message to the execution log. The timestamp is automatically included.

  13. def manualSaveDataFrame(frame: DataFrame, part: Option[Int] = scope.parallelPart): Unit

    Writes a collection of data to the underlying storage and associates it with the transform.

    Writes a collection of data to the underlying storage and associates it with the transform. This is used to take control of the writing process during transform execution rather than simply writing the return value of the transform. If using this it is recommended to return an empty data frame from the transform.

    frame

    The Apache Spark data frame to write.

    part

    The part number to write, if relevant.

  14. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  15. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  16. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  17. val options: Options

    The collection of options configured for the transform.

  18. def setArtifact(name: String, content: ⇒ Array[Byte]): Unit

    Stores an artifact and associates it with the transform.

    Stores an artifact and associates it with the transform. This can de downloaded from the execution result.

    name

    The name of the artifact.

    content

    The byte content of the artifact to store.

  19. def setProgress(progress: Double): Unit

    Update the progress value for a specific transform.

    Update the progress value for a specific transform. The system switches to custom progress logging on the first call to this method, therefore one should start by calling setProgress(0.0).

    progress

    A value between 0 and 1

  20. def state: Map[String, Any]

    The state map of the transform.

    The state map of the transform. Use this to persist information across transforms.

  21. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  22. def toString(): String
    Definition Classes
    AnyRef → Any
  23. def utcNowJava: LocalDateTime

    The current UTC date and time.

    The current UTC date and time.

    Definition Classes
    RealTimeAware
  24. def utcTodayJava: LocalDateTime

    The current UTC date.

    The current UTC date.

    Definition Classes
    RealTimeAware
  25. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  27. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated
    Deprecated

Inherited from RealTimeAware

Inherited from AnyRef

Inherited from Any

Ungrouped