OutputFacade

Companion object OutputFacade

trait OutputFacade extends ServiceFacadeWithDatabase with ServiceFacade[OutputFacade]

The interface for the external API of the Output Service.

The Output Service is responsible for the management of output data and metadata from runs of sources, pipelines and batches. It creates point-in-time snapshots of the configuration which are used to maintain a complete revision history knowing exactly which other data sets a given data set depends on, and which version of the configuration was used to build them.

Linear Supertypes

ServiceFacade[OutputFacade], ServiceFacadeWithDatabase, ServiceFacadeBase, AnyRef, Any

Known Subclasses

OutputWebFacade

Ordering

Alphabetic
By Inheritance

Inherited

OutputFacade
ServiceFacade
ServiceFacadeWithDatabase
ServiceFacadeBase
AnyRef
Any

Hide All
Show All

Visibility

Public
All

Abstract Value Members

abstract val environment: String

Definition Classes
ServiceFacadeBase
abstract def getArtifactContentForSnapshotForPipeline(pipelineId: String, pipelineSnapshotVersion: UUID, transformId: String, name: String, pipelineTag: String, outputStream: OutputStream): Unit
Gets the content for a specific artifact in a given pipeline snapshot.
Gets the content for a specific artifact in a given pipeline snapshot.
pipelineId
The unique id of the pipeline.
pipelineSnapshotVersion
The UUID of the snapshot.
transformId
The unique id of the transform in the pipeline.
name
The name of the artifact.
pipelineTag
The tag of the snapshot.
outputStream
The stream where data is written to.
abstract def getArtifactsForSnapshotForPipeline(id: String, snapshotId: UUID): Array[ArtifactsInfo]
Gets the artifact metadata associated a specific pipeline and snapshot.
Gets the artifact metadata associated a specific pipeline and snapshot.
id
The unique id of the pipeline.
snapshotId
The UUID of the snapshot.
abstract def getCacheStatistics: Array[CacheStatistics]

Definition Classes
ServiceFacadeBase
abstract def getDatabase(outputStream: OutputStream): Unit

Definition Classes
ServiceFacadeWithDatabase
abstract def getDatabasePath: String

Definition Classes
ServiceFacadeWithDatabase
abstract def getFile(path: String, outputStream: OutputStream): Unit

Definition Classes
ServiceFacadeBase
abstract def getLatestForSnapshotsForBatch(id: String, tag: String, date: LocalDateTime): BatchSnapshotInfo
Gets the most recent snapshot relative to a given date for a specific batch and tag.
Gets the most recent snapshot relative to a given date for a specific batch and tag.
id
The unique id of the batch.
tag
The tag of the snapshot.
date
The date to use as the upper bound. LocalDateTime.MAX not supported. Use DateTimeSupport.maxDate instead.
abstract def getLatestForSnapshotsForPipeline(id: String, tag: String, date: LocalDateTime): PipelineSnapshotInfo
Gets the most recent snapshot relative to a given date for a specific pipeline and tag.
Gets the most recent snapshot relative to a given date for a specific pipeline and tag.
id
The unique id of the pipeline.
tag
The tag of the snapshot.
date
The date to use as the upper bound. LocalDateTime.MAX not supported. Use DateTimeSupport.maxDate instead.
abstract def getLatestForSnapshotsForSource(id: String, tag: String, date: LocalDateTime): SourceSnapshotInfo
Gets the most recent snapshot relative to a given date for a specific source and tag.
Gets the most recent snapshot relative to a given date for a specific source and tag.
id
The unique id of the source.
tag
The tag of the snapshot.
date
The date to use as the upper bound. LocalDateTime.MAX not supported. Use DateTimeSupport.maxDate instead.
abstract def getLogLevel: String

Definition Classes
ServiceFacadeBase
abstract def getOutputForSnapshotsForPipeline(id: String, snapshotIds: Array[UUID], transformId: String, outputStream: OutputStream, locale: String = getOutputDefaultLocale, limit: Option[Int] = None, maxDecimals: Int = getOutputDefaultMaxDecimals, serializeAsJson: Boolean = false): Unit
Gets the output data set for a specific pipeline transform and specific snapshots.
Gets the output data set for a specific pipeline transform and specific snapshots. Data is written incrementally, but this may still take a long time to complete for large data sets.
id
The unique id of the pipeline.
snapshotIds
The UUID values of the snapshots.
transformId
The unique id of the transform in the pipeline.
outputStream
The stream where data is written to.
locale
The locale to use when formatting numbers, i.e. 'da'.
limit
A limit on the number of data rows to write.
maxDecimals
The maximum number of decimals to include when formatting numbers.
serializeAsJson
Serialize the output as JSON array of arrays or as a csv with a ';' as the separator in either case the first row/line is the header.
abstract def getOutputForSnapshotsForSource(id: String, snapshotIds: Array[UUID], outputStream: OutputStream, locale: String = getOutputDefaultLocale, limit: Option[Int] = None, maxDecimals: Int = getOutputDefaultMaxDecimals, serializeAsJson: Boolean = false): Unit
Gets the output data set for a specific source and specific snapshots.
Gets the output data set for a specific source and specific snapshots. Data is written incrementally, but this may still take a long time to complete for large data sets.
id
The unique id of the source.
snapshotIds
The UUID values of the snapshots.
outputStream
The stream data is written to.
locale
The locale to use when formatting numbers, i.e. 'da'.
limit
A limit on the number of data rows to write.
maxDecimals
The maximum number of decimals to include when formatting numbers.
serializeAsJson
Serialize the output as JSON array of arrays or as a csv with a ';' as the separator in either case the first row/line is the header.
abstract def getPerformanceMetrics: PerformanceMetrics

Definition Classes
ServiceFacadeBase
abstract def getSnapshotForBatch(id: String, tag: String, date: LocalDateTime): BatchSnapshotInfo
Gets a specific snapshot for a specific batch.
Gets a specific snapshot for a specific batch. Batch snapshots are uniquely determined by their batch id, tag and date.
id
The unique id of the batch.
tag
The tag of the snapshot.
date
The date of the snapshot.
abstract def getSnapshotForPipeline(id: String, snapshotId: UUID): PipelineSnapshotInfo
Gets a specific snapshot for a specific pipeline.
Gets a specific snapshot for a specific pipeline.
id
The unique id of the pipeline.
snapshotId
The UUID of the snapshot.
abstract def getSnapshotForSource(id: String, snapshotId: UUID): SourceSnapshotInfo
Gets a specific snapshot for a specific source.
Gets a specific snapshot for a specific source.
id
The unique id of the source.
snapshotId
The UUID of the snapshot.
abstract def getSnapshotsForBatch(id: String, includeVolatile: Boolean = false, snapshotIds: Option[Array[LocalDateTime]] = None): Array[BatchSnapshotInfo]
Gets all snapshots for a specific batch filtered on given parameters.
Gets all snapshots for a specific batch filtered on given parameters.
id
The unique id of the batch.
includeVolatile
Whether to include snapshots with an expiration date.
snapshotIds
An optional set of specific snapshot ids to use as filter.
abstract def getSnapshotsForBatchForTag(id: String, tag: String, includeVolatile: Boolean = false, snapshotIds: Option[Array[LocalDateTime]] = None): Array[BatchSnapshotInfo]
Gets all snapshots for a specific batch and tag filtered on given parameters.
Gets all snapshots for a specific batch and tag filtered on given parameters.
id
The unique id of the batch.
tag
A tag to require on the snapshot.
includeVolatile
Whether to include snapshots with an expiration date.
snapshotIds
An optional set of specific snapshot ids to use as filter.
abstract def getSnapshotsForPipeline(id: String, includeDeleted: Boolean = false, includeVolatile: Boolean = false, snapshotIds: Option[Array[UUID]] = None): Array[PipelineSnapshotInfo]
Gets all snapshots for a specific pipeline filtered on given parameters.
Gets all snapshots for a specific pipeline filtered on given parameters.
id
The unique id of the pipeline.
includeVolatile
Whether to include snapshots with an expiration date.
snapshotIds
An optional set of specific snapshot ids to use as filter.
abstract def getSnapshotsForPipelineForTag(id: String, tag: String, includeDeleted: Boolean = false, includeVolatile: Boolean = false, snapshotIds: Option[Array[UUID]] = None): Array[PipelineSnapshotInfo]
Gets all snapshots for a specific pipeline and tag filtered on given parameters.
Gets all snapshots for a specific pipeline and tag filtered on given parameters.
id
The unique id of the pipeline.
tag
A tag to require on the snapshot.
includeDeleted
Whether to include snapshots that have either been deleted or are not yet computed.
includeVolatile
Whether to include snapshots with an expiration date.
snapshotIds
An optional set of specific snapshot ids to use as filter.
abstract def getSnapshotsForSource(id: String, includeDeleted: Boolean = false, includeVolatile: Boolean = false, snapshotIds: Option[Array[UUID]] = None): Array[SourceSnapshotInfo]
Gets all snapshots for a specific source filtered on given parameters.
Gets all snapshots for a specific source filtered on given parameters.
id
The unique id of the source.
includeVolatile
Whether to include snapshots with an expiration date.
snapshotIds
An optional set of specific snapshot ids to use as filter.
abstract def getSnapshotsForSourceForTag(id: String, tag: String, includeDeleted: Boolean = false, includeVolatile: Boolean = false, snapshotIds: Option[Array[UUID]] = None): Array[SourceSnapshotInfo]
Gets all snapshots for a specific source and tag filtered on given parameters.
Gets all snapshots for a specific source and tag filtered on given parameters.
id
The unique id of the source.
tag
A tag to require on the snapshot.
includeDeleted
Whether to include snapshots that have either been deleted or are not yet computed.
includeVolatile
Whether to include snapshots with an expiration date.
snapshotIds
An optional set of specific snapshot ids to use as filter.
abstract def getTableMetadata(tableName: String): TableMetadataInfo
Gets the metadata associated with a given data set.
Gets the metadata associated with a given data set. For internal use only.
tableName
The name of the data set generated from OutputAccess.id.
abstract def getVersion: String

Definition Classes
ServiceFacadeBase
abstract def isReady: Boolean

Definition Classes
ServiceFacadeBase
abstract def postCachePurge(id: Option[String]): Unit

Definition Classes
ServiceFacadeBase
abstract def postCheckpointForSnapshotForPipeline(id: String, snapshotId: UUID, checkpoint: PipelineCheckpoint): Unit
Appends a checkpoint to a specific pipeline snapshot.
Appends a checkpoint to a specific pipeline snapshot. Checkpoints are used to allow partial reuse of data in a previously created snapshot. This way a transient failure would not necessarily require running all transforms in the pipeline again. For internal use only.
id
The unique id of the pipeline.
snapshotId
The UUID of the snapshot.
checkpoint
The checkpoint information to append.
abstract def postDatabaseBackup(encryptionEnabled: Option[Boolean], encryptionMasterKeyIdentifier: Option[String], outputRepository: String = ...): Unit

Definition Classes
ServiceFacadeWithDatabase
abstract def postDeleteColumnsForSnapshotForPipeline(id: String, snapshotId: UUID, transform: String, columns: List[String]): Unit
Attempts to restore a deleted pipeline snapshot to a persisted state.
Attempts to restore a deleted pipeline snapshot to a persisted state. The restore is dependent on versioning in the underlying data store.
id
The unique id of the pipeline.
snapshotId
The UUID value of the snapshot.
transform
The name of the transform to delete from.
columns
The names of the columns to delete snapshot.
abstract def postDeleteColumnsForSnapshotForSource(id: String, snapshotId: UUID, columns: List[String]): Unit
Deletes one or more columns from a persisted source snapshot.
Deletes one or more columns from a persisted source snapshot.
id
The unique id of the source.
snapshotId
The UUID value of the snapshot.
columns
The names of the columns to delete snapshot.
abstract def postDeleteForSnapshotForBatch(id: String, tag: String, dates: LocalDateTime*): Unit
Deletes a list of specific batch snapshots and marks all included source and pipeline snapshots, that are not used by other batch snapshots, as deleted.
Deletes a list of specific batch snapshots and marks all included source and pipeline snapshots, that are not used by other batch snapshots, as deleted.
id
The unique id of the batch.
tag
The tag of the snapshot.
dates
The dates for the snapshots.
abstract def postDeleteForSnapshotForPipeline(id: String, snapshotIds: UUID*): Unit
Marks a list of snapshots for a specific pipeline as deleted.
Marks a list of snapshots for a specific pipeline as deleted. The underlying data will eventually be purged dependent on EnvironmentVariables.DATASOLVR_OUTPUT_PURGE_AFTER_DAYS.
id
The unique id of the pipeline.
snapshotIds
The UUID values for the snapshots.
abstract def postDeleteForSnapshotForSource(id: String, snapshotIds: UUID*): Unit
Marks a list of snapshots for a specific source as deleted.
Marks a list of snapshots for a specific source as deleted. The underlying data will eventually be purged dependent on EnvironmentVariables.DATASOLVR_OUTPUT_PURGE_AFTER_DAYS.
id
The unique id of the source.
snapshotIds
The UUID values for the snapshots.
abstract def postGenerateForSnapshotForBatch(batch: BatchContent, tag: String, date: LocalDateTime = postSnapshotDefaultDate()): BatchSnapshotInfo
Generates a snapshot for a given batch definition, tag and date.
Generates a snapshot for a given batch definition, tag and date. This snapshot is not stored in the database and cannot be referenced.
batch
The batch definition.
tag
The tag to use when creating the snapshot.
date
The date to associate with the snapshot.
abstract def postGenerateForSnapshotForPipeline(pipeline: PipelineContent, date: LocalDateTime = postSnapshotDefaultDate(), sourceSnapshots: List[SourceSnapshotInfo] = List(), pipelineSnapshots: List[PipelineSnapshotInfo] = List()): PipelineSnapshotInfo
Generates a snapshot for a given pipeline definition and date.
Generates a snapshot for a given pipeline definition and date. This snapshot is not stored in the database and cannot be referenced.
pipeline
The pipeline definition.
date
The date to associate with the snapshot.
sourceSnapshots
A collection of generated source snapshots that can be used when building the snapshot.
pipelineSnapshots
A collection of generated pipeline snapshots that can be used when building the snapshot.
abstract def postGenerateForSnapshotForSource(source: SourceContent, date: LocalDateTime = postSnapshotDefaultDate()): SourceSnapshotInfo
Generates a snapshot for a given source definition and date.
Generates a snapshot for a given source definition and date. This snapshot is not stored in the database and cannot be referenced.
source
The source definition.
date
The date to associate with the snapshot.
abstract def postIsPersistedForSnapshotForPipeline(id: String, snapshotIds: Array[UUID], isPersisted: Boolean): Unit
Update the persistence status of a collection of snapshots for a specific pipeline.
Update the persistence status of a collection of snapshots for a specific pipeline. For internal use only.
id
The unique id of the pipeline.
snapshotIds
The UUID values of the snapshots.
isPersisted
The persistence value to set.
abstract def postIsPersistedForSnapshotForSource(id: String, snapshotIds: Array[UUID], isPersisted: Boolean): Unit
Update the persistence status of a collection of snapshots for a specific source.
Update the persistence status of a collection of snapshots for a specific source. For internal use only.
id
The unique id of the source.
snapshotIds
The UUID values of the snapshots.
isPersisted
The persistence value to set.
abstract def postRestart: Unit

Definition Classes
ServiceFacadeBase
abstract def postRestoreForSnapshotForPipeline(id: String, snapshotId: UUID): Unit
Attempts to restore a deleted pipeline snapshot to a persisted state.
Attempts to restore a deleted pipeline snapshot to a persisted state. The restore is dependent on versioning in the underlying data store.
id
The unique id of the pipeline.
snapshotId
The UUID value of the snapshot.
abstract def postRestoreForSnapshotForSource(id: String, snapshotId: UUID): Unit
Attempts to restore a deleted source snapshot to a persisted state.
Attempts to restore a deleted source snapshot to a persisted state. The restore is dependent on versioning in the underlying data store.
id
The unique id of the source.
snapshotId
The UUID value of the snapshot.
abstract def postSnapshotForBatch(id: String, tag: String, date: LocalDateTime = postSnapshotDefaultDate(), expires: Option[LocalDateTime] = None, updates: BatchUpdates = BatchUpdates(List(), List())): BatchSnapshotInfo
Creates a snapshot for a specific batch for a given tag and date.
Creates a snapshot for a specific batch for a given tag and date. This creates snapshots for all sources and pipelines with results missing from the batch, with a requested update or with a dependency on a requested update. Untagged pipelines and sources may be used if no matching tag exists.
id
The unique id of the batch.
tag
The tag to use when creating the snapshot.
date
The date to associate with the snapshot.
expires
An optional expiration time for the snapshot.
updates
A specification of source and pipelines that should be updated. This will also update dependencies in the batch.
abstract def postSnapshotForPipeline(id: String, tag: String, date: LocalDateTime = postSnapshotDefaultDate(), allowOverwrite: Boolean = false): PipelineSnapshotInfo
Creates a snapshot for a specific pipeline for a given tag and date.
Creates a snapshot for a specific pipeline for a given tag and date. Untagged pipelines and sources may be used if no matching tag exists.
id
The unique id of the pipeline.
tag
The tag to use when creating the snapshot
date
The date to associate with the snapshot.
allowOverwrite
Whether to allow deletion of an existing snapshot with the same date and tag, and no dependent snapshots.
abstract def postSnapshotForSource(id: String, tag: String, date: LocalDateTime = postSnapshotDefaultDate(), allowOverwrite: Boolean = false): SourceSnapshotInfo
Creates a snapshot for a specific source for a given tag and date.
Creates a snapshot for a specific source for a given tag and date. Untagged sources and files may be used if no matching tag exists.
id
The unique id of the source.
tag
The tag to use when creating the snapshot
date
The date to associate with the snapshot.
allowOverwrite
Whether to allow deletion of an existing snapshot with the same date and tag, and no dependent snapshots.
abstract def postTableMetadata(metadata: TableMetadataInfo): Unit
Updates the collection of data set metadata.
Updates the collection of data set metadata. For internal use only.
metadata
The metadata to insert or update.
abstract def postVacuumDatabase: Unit

Definition Classes
ServiceFacadeWithDatabase
abstract def setLogLevel(level: String): Unit

Definition Classes
ServiceFacadeBase
abstract def withTimeout(seconds: Int): OutputFacade

Definition Classes
ServiceFacade
abstract def withToken(token: UUID): OutputFacade

Definition Classes
ServiceFacade

Concrete Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( ... ) @native() @HotSpotIntrinsicCandidate()
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
Annotations
@native() @HotSpotIntrinsicCandidate()
val getOutputDefaultLocale: String
The default value for locale used in getOutputForSnapshotsForSource, getOutputForSnapshotsForPipeline.
The default value for locale used in getOutputForSnapshotsForSource, getOutputForSnapshotsForPipeline. The value is equal to EnvironmentVariables.DATASOLVR_DEFAULT_OUTPUT_LOCALE.
val getOutputDefaultMaxDecimals: Int
The default value for maxDecimals used in getOutputForSnapshotsForSource, getOutputForSnapshotsForPipeline.
The default value for maxDecimals used in getOutputForSnapshotsForSource, getOutputForSnapshotsForPipeline. The value is equal to EnvironmentVariables.DATASOLVR_DEFAULT_MAX_DECIMALS.
def hashCode(): Int

Definition Classes
AnyRef → Any
Annotations
@native() @HotSpotIntrinsicCandidate()
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
Annotations
@native() @HotSpotIntrinsicCandidate()
final def notifyAll(): Unit

Definition Classes
AnyRef
Annotations
@native() @HotSpotIntrinsicCandidate()
val postSnapshotDefaultDate: () ⇒ LocalDateTime
The default value for date used in postSnapshotForSource, postSnapshotForPipeline and postSnapshotForBatch.
The default value for date used in postSnapshotForSource, postSnapshotForPipeline and postSnapshotForBatch. The value is a function that returns the current UTC time.
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... ) @native()
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
def waitIsReady(timeoutSeconds: Int, pollIntervalSeconds: Int): Unit

Definition Classes
ServiceFacadeBase

Deprecated Value Members

def finalize(): Unit

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] ) @Deprecated
Deprecated

Packages

OutputFacade

Companion object OutputFacade

trait OutputFacade extends ServiceFacadeWithDatabase with ServiceFacade[OutputFacade]

Abstract Value Members

Concrete Value Members

Deprecated Value Members

Inherited from ServiceFacade[OutputFacade]

Inherited from ServiceFacadeWithDatabase

Inherited from ServiceFacadeBase

Inherited from AnyRef

Inherited from Any

Ungrouped

Packages

OutputFacade 

Companion object OutputFacade

trait OutputFacade extends ServiceFacadeWithDatabase with ServiceFacade[OutputFacade]

Abstract Value Members

Concrete Value Members

Deprecated Value Members

Inherited from ServiceFacade[OutputFacade]

Inherited from ServiceFacadeWithDatabase

Inherited from ServiceFacadeBase

Inherited from AnyRef

Inherited from Any

Ungrouped

OutputFacade