object GeneralSourceOptions extends OptionDescriptionBuilder
Defines the options that can be used on any source.
- Alphabetic
- By Inheritance
- GeneralSourceOptions
- OptionDescriptionBuilder
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- val CSV_DELIMITER: String
The seperator to expect when reading CSV input.
The seperator to expect when reading CSV input. Default is ';'.
- val INCLUDE_SUBFOLDERS: String
Whether to include mails in subfolders when reading from inboxes.
Whether to include mails in subfolders when reading from inboxes. Only used with ConnectionProtocols.exchange. Default is
false. - val INDEX_BY: String
A column to use for indexing of the data.
A column to use for indexing of the data. This will make queries and joins on this column faster. The number of unique values in the column should not exceed 10000.
- val IN_MEMORY_CACHE_TIME: String
The number of minutes to cache sources, that are only run in-memory, for queries.
The number of minutes to cache sources, that are only run in-memory, for queries. Default is 0.
- val IN_MEMORY_ONLY: String
Whether to only run the source in-memory (as opposed to persisted).
Whether to only run the source in-memory (as opposed to persisted). Default is
false. - def Mandatory(name: String, validation: (String) => Option[String] = s => None): OptionDescription
- Attributes
- protected
- Definition Classes
- OptionDescriptionBuilder
- def Optional(name: String, default: Option[String] = None, validation: (String) => Option[String] = s => None): OptionDescription
- Attributes
- protected
- Definition Classes
- OptionDescriptionBuilder
- val PARTITION_BY: String
Partitions the source input using the given comparable column.
Partitions the source input using the given comparable column. This is used to avoid timeouts on large amounts of input tables in databases. The column should be indexed in the database. Default is not to partition input. This is used together with PARTITION_SIZE or PARTITION_FILE.
- val PARTITION_FILE: String
The id of a file containing a list of partition values.
The id of a file containing a list of partition values. Instead of computing the partitions from the source data, in chunks of size PARTITION_SIZE, the partitions are read from the file. The file should contain lines, with each line being a value of the same type as the column specified in PARTITION_BY. Each line in the file corresponds to the border between two partitions, i.e. the number of partitions is one more than the number of rows. Using this kind of fixed partitioning works best if the column specified in PARTITION_BY has an index and the values are evenly distributed.
- val PARTITION_SIZE: String
The maximum number rows in each partition.
The maximum number rows in each partition. This is used to avoid timeouts on large amounts of input tables in databases. The column should be indexed in the database. Default is not to partition input.
- val REQUIRE_NON_EMPTY: String
Whether to require non-empty for the source to complete a run successfully.
Whether to require non-empty for the source to complete a run successfully. This is used when waiting for an external source to make output available, since failed runs can be retried automatically. Default is
false. - val STRING_ENCODING: String
The text encoding to expect when reading text input Default is UTF-8.
- val TRIM_STRINGS: String
Whether to trim string columns for leading and trailing whitespace
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def baseDescriptions: List[OptionDescription]
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @HotSpotIntrinsicCandidate() @native()
- def descriptions(descriptions: OptionDescription*): OptionDescriptions
- Attributes
- protected
- Definition Classes
- OptionDescriptionBuilder
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def get: OptionDescriptions
- Definition Classes
- GeneralSourceOptions → OptionDescriptionBuilder
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @HotSpotIntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @HotSpotIntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def legacyOptions: Set[String]
- Definition Classes
- GeneralSourceOptions → OptionDescriptionBuilder
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @HotSpotIntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @HotSpotIntrinsicCandidate() @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)