Packages

c

com.datasolvr.services.spark

PipelineExecutionStateApi

class PipelineExecutionStateApi extends AnyRef

Exposes methods and properties that may be used independent of the specific transform.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. PipelineExecutionStateApi
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new PipelineExecutionStateApi(scope: PipelineExecutionScope)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @HotSpotIntrinsicCandidate()
  6. val content: PipelineContent

    The properties of the pipeline configuration.

  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. def manualSaveDataFrameForTransform(frame: DataFrame, transform: String, part: Option[Int] = parallelPartOption): Unit

    Writes a collection of data to the underlying storage and associates it with a given transform.

    Writes a collection of data to the underlying storage and associates it with a given transform. This is used to take control of the writing process during transform execution rather than simply writing the return value of the transform. If using this it is recommended to return an empty data frame from the transform.

    frame

    The Apache Spark data frame to write.

    transform

    The id of the transform which the data belongs to.

    part

    The part number to write, if relevant.

  13. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  14. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  15. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  16. val options: Options

    The collection of options configured for the pipeline.

  17. lazy val parallelPart: Int

    The parallel part number of the current execution.

  18. lazy val parallelPartOption: Option[Int]

    The parallel part number of the current execution, or

    The parallel part number of the current execution, or 

    None

    if it does not exist. None }}}

  19. def shouldCancel: Boolean

    Informs the runtime to cancel the current execution.

    Informs the runtime to cancel the current execution. Only used for test runs.

  20. val snapshot: PipelineSnapshotInfo

    The properties of the snapshot of the pipeline configuration.

  21. def state: Map[String, Map[String, Any]]

    Provides access to the state maps of the pipeline, keyed by transform id.

  22. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  23. val testMode: Boolean

    Whether executing in test mode.

    Whether executing in test mode. Assert on this to avoid undesired side effects during test runs.

  24. val timelineDate: LocalDateTime

    The timeline date of the current execution.

  25. def toString(): String
    Definition Classes
    AnyRef → Any
  26. val user: String

    The currently executing user.

    The currently executing user. This will be a service user for non-test execution.

  27. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  28. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  29. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated
    Deprecated

Inherited from AnyRef

Inherited from Any

Ungrouped