trait RunnerFacade extends ServiceFacade[RunnerFacade]
The interface for the external API of the Runner Service.
The Runner Service is responsible for executing jobs to compute source and pipeline output or updating dashboards. It also provides functionality to assist the development process such as test execution of source and pipeline definitions, compilation checks and intellisense support.
- Alphabetic
- By Inheritance
- RunnerFacade
- ServiceFacade
- ServiceFacadeBase
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Abstract Value Members
-
abstract
val
environment: String
- Definition Classes
- ServiceFacadeBase
-
abstract
def
getCacheStatistics: Array[CacheStatistics]
- Definition Classes
- ServiceFacadeBase
-
abstract
def
getExecutableForTransformForPipeline(pipeline: PipelineContent, transform: String): String
Builds a scala code snippet that includes the transform code of a pipeline definition and compiles as a standalone file, assuming all library dependencies are present.
Builds a scala code snippet that includes the transform code of a pipeline definition and compiles as a standalone file, assuming all library dependencies are present.
- pipeline
The pipeline definition.
- transform
The id of the transform to use.
-
abstract
def
getFile(path: String, outputStream: OutputStream): Unit
- Definition Classes
- ServiceFacadeBase
-
abstract
def
getLogLevel: String
- Definition Classes
- ServiceFacadeBase
-
abstract
def
getPerformanceMetrics: PerformanceMetrics
- Definition Classes
- ServiceFacadeBase
-
abstract
def
getSparkUIEndpoint: String
The endpoint for contacting the API of the Apache Spark UI.
The endpoint for contacting the API of the Apache Spark UI. This is used to report progress on job execution similar to what is seen the Spark UI.
-
abstract
def
getTestResultsOutputForTransformForPipeline(pipeline: PipelineContent, transform: String, date: LocalDateTime = postDefaultDate(), sampleSize: Option[Int] = Some(postDefaultSampleSize), outputSize: Option[Int] = Some(postDefaultOutputSize), locale: String = getOutputDefaultLocale, maxDecimals: Int = getOutputDefaultMaxDecimals, serializeAsJson: Boolean = false): TestResultOutput
Gets the results of a test execution for given pipeline definition and transform.
Gets the results of a test execution for given pipeline definition and transform. This does not schedule a new test execution, but assumes that a completed execution matching the parameters exists.
- pipeline
The pipeline definition.
- transform
The id of the transform.
- date
The date to use for running the pipeline.
- sampleSize
An optional sample size which is applied to all imports in the pipeline.
- outputSize
An optional output size which is applied as a limit on the output data set.
- locale
The locale to use when formatting numbers, i.e. 'da'.
- maxDecimals
The maximum number of decimals to include when formatting numbers.
- serializeAsJson
When true the result's TestResultOutput.dataTableContent contains a json string else it contains a csv string.
-
abstract
def
getTestResultsStatusForPipeline(pipeline: PipelineContent, date: LocalDateTime = postDefaultDate(), sampleSize: Option[Int] = Some(postDefaultSampleSize), outputSize: Option[Int] = Some(postDefaultOutputSize)): Array[TransformTestResultStatus]
Gets the status of a test execution for each transform in a given pipeline definition.
Gets the status of a test execution for each transform in a given pipeline definition.
- pipeline
The pipeline definition.
- date
The date to use for running the pipeline.
- sampleSize
An optional sample size which is applied to all imports in the pipeline.
- outputSize
An optional output size which is applied as a limit on the output data set.
-
abstract
def
getVersion: String
- Definition Classes
- ServiceFacadeBase
-
abstract
def
isReady: Boolean
- Definition Classes
- ServiceFacadeBase
-
abstract
def
postAnalyzeCodeForColumns(code: String, tag: String, date: LocalDateTime): Array[Column]
Get auto-complete column candidates at a given point in a SQL code snippet.
Get auto-complete column candidates at a given point in a SQL code snippet.
- code
The information on the code snippet and completion point.
-
abstract
def
postAnalyzeCodeForColumnsForPipeline(pipeline: PipelineContent, transform: String): Array[Column]
Get auto-complete column candidates at a given point in a SQL transform in a pipeline.
Get auto-complete column candidates at a given point in a SQL transform in a pipeline.
- pipeline
The pipeline definition.
- transform
The transform in the pipeline.
-
abstract
def
postAutoCompleteCode(codeRequest: AutoCompleteRequest): Array[AutoComplete]
Get auto-complete candidates at a given point in a code snippet.
Get auto-complete candidates at a given point in a code snippet.
- codeRequest
The information on the code snippet and completion point.
-
abstract
def
postAutoCompleteCodeForPipeline(pipeline: PipelineContent, transform: String, offset: Int): Array[AutoComplete]
Get auto-complete candidates at a given point in a transform in a pipeline.
Get auto-complete candidates at a given point in a transform in a pipeline.
- pipeline
The pipeline definition.
- transform
The id of the transform in the pipeline.
- offset
The character offset for which to obtain auto-complete candidates.
-
abstract
def
postCachePurge(id: Option[String]): Unit
- Definition Classes
- ServiceFacadeBase
-
abstract
def
postCancelJobGroupForUser(): Unit
Cancel all running test executions started by the calling user.
-
abstract
def
postCompileCode(codeRequest: CompileRequest): Array[Annotation]
Get compilation errors for a code snippet in the form of code annotations.
Get compilation errors for a code snippet in the form of code annotations.
- codeRequest
The information on the code snippet to compile.
-
abstract
def
postCompileCodeForPipeline(pipeline: PipelineContent, transform: String): Array[Annotation]
Get compilation errors for a transform in a pipeline definition in the form of code annotations.
Get compilation errors for a transform in a pipeline definition in the form of code annotations.
- pipeline
The pipeline definition.
- transform
The transform in the pipeline.
-
abstract
def
postFormatCode(request: FormatRequest): String
Formats a code snippet.
-
abstract
def
postGenerateDashboardForSourceSnapshotInfo(snapshotInfo: SourceSnapshotInfo, targetColumn: String): DashboardContent
Generates a dashboard definition based on a source snapshot and a target column.
Generates a dashboard definition based on a source snapshot and a target column. This method attempts to generate a meaningful dashboard definition for a given source snapshot and primary target column of interest.
- snapshotInfo
The source snapshot to generate the dashboard from.
- targetColumn
The name of the primary column to use for deciding which charts to present.
-
abstract
def
postGenerateForSource(description: SourceDescription, options: Map[String, String] = Map(), tag: String = Tags.empty): SourceContent
Generates a complete source definition based on the a source description.
Generates a complete source definition based on the a source description. This method attempts to generate a meaningful source data description by running the source from the the source description and looking at the schema and contents of the resulting data set.
- description
A description of how to run the source.
- options
A collections of options the source should be run with.
- tag
An optional tag the source should be run with.
-
abstract
def
postGenerateSourcesForFile(file: FileContent): Array[SourceContent]
Generates source definitions from given file content.
Generates source definitions from given file content. For simple file content this is essentially the same as calling postGenerateForSource, but this method is also capable of transforming Excel file content into a collection of source definitions - one for each sheet.
- file
The file content to use.
-
abstract
def
postRestart: Unit
- Definition Classes
- ServiceFacadeBase
-
abstract
def
postTestForPipeline(pipeline: PipelineContent, date: LocalDateTime = postDefaultDate(), sampleSize: Option[Int] = Some(postDefaultSampleSize), outputSize: Option[Int] = Some(postDefaultOutputSize), transform: Option[String] = None, locale: String = getOutputDefaultLocale, maxDecimals: Int = getOutputDefaultMaxDecimals): Unit
Starts a test execution for a given pipeline definition and (optionally) a specific transform.
Starts a test execution for a given pipeline definition and (optionally) a specific transform.
- pipeline
The pipeline definition.
- date
The date to use for running the pipeline.
- sampleSize
An optional sample size which is applied to all imports in the pipeline.
- outputSize
An optional output size which is applied as a limit on the output data set.
- transform
The id of the transform, or
Noneto run the full pipeline.- locale
The locale to use when formatting numbers, i.e. 'da'.
- maxDecimals
The maximum number of decimals to include when formatting numbers.
-
abstract
def
postTestForSource(source: SourceContent, date: LocalDateTime = postDefaultDate(), sampleSize: Option[Int] = Some(postDefaultSampleSize), outputSize: Option[Int] = Some(postDefaultOutputSize), locale: String = getOutputDefaultLocale, maxDecimals: Int = getOutputDefaultMaxDecimals, serializeAsJson: Boolean = false): TestResult
Starts a test execution for a given source definition.
Starts a test execution for a given source definition.
- source
The source definition.
- date
The date to use for running the source.
- sampleSize
An optional sample size which is applied to all imports in the pipeline.
- outputSize
An optional output size which is applied as a limit on the output data set.
- locale
The locale to use when formatting numbers, i.e. 'da'.
- maxDecimals
The maximum number of decimals to include when formatting numbers.
- serializeAsJson
When true the result's TestResultOutput.dataTableContent contains a json string else it contains a csv string.
-
abstract
def
postValidateForPlugins(plugins: PluginsContent): Unit
Check a given collection of custom functions, transforms and modules for validity.
Check a given collection of custom functions, transforms and modules for validity.
- plugins
The content to check.
-
abstract
def
setLogLevel(level: String): Unit
- Definition Classes
- ServiceFacadeBase
-
abstract
def
withTimeout(seconds: Int): RunnerFacade
- Definition Classes
- ServiceFacade
-
abstract
def
withToken(token: UUID): RunnerFacade
- Definition Classes
- ServiceFacade
Concrete Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native() @HotSpotIntrinsicCandidate()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
val
getOutputDefaultLocale: String
The default value for
localeused in the API.The default value for
localeused in the API. The value is equal to EnvironmentVariables.DATASOLVR_DEFAULT_OUTPUT_LOCALE. -
val
getOutputDefaultMaxDecimals: Int
The default value for
maxDecimalsused in the API.The default value for
maxDecimalsused in the API. The value is equal to EnvironmentVariables.DATASOLVR_DEFAULT_MAX_DECIMALS. -
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
val
postDefaultDate: () ⇒ LocalDateTime
The default value for
dateused in the API.The default value for
dateused in the API. The value is a function that returns the current UTC time. -
val
postDefaultOutputSize: Int
The default value for
outputSizeused in the API.The default value for
outputSizeused in the API. The value is 1000. -
val
postDefaultSampleSize: Int
The default value for
sampleSizeused in the API.The default value for
sampleSizeused in the API. The value is 200. -
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
def
waitIsReady(timeoutSeconds: Int, pollIntervalSeconds: Int): Unit
- Definition Classes
- ServiceFacadeBase
Deprecated Value Members
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] ) @Deprecated
- Deprecated