Redis Data Integration configuration file
Redis Data Integration configuration file reference
Properties
| Name | Type | Description | Required |
|---|---|---|---|
| sources (Source collectors) |
object |
||
| processors (Configuration details of Redis Data Integration Processors) |
object, null |
||
| targets (Target connections) |
object |
sources: Source collectors
Additional Properties
| Name | Type | Description | Required |
|---|
processors: Configuration details of Redis Data Integration Processors
Properties
| Name | Type | Description | Required |
|---|---|---|---|
| on_failed_retry_interval (Interval (in seconds) on which to perform retry on failure) |
integer, string |
Default: 5Pattern: ^\${.*}$Minimum: 1 |
|
| read_batch_size (The batch size for reading data from source database) |
integer, string |
Default: 2000Pattern: ^\${.*}$Minimum: 1 |
|
| debezium_lob_encoded_placeholder (Enable Debezium LOB placeholders) |
string |
Default: "X19kZWJleml1bV91bmF2YWlsYWJsZV92YWx1ZQ==" |
|
| dedup (Enable deduplication mechanism) |
boolean |
Default: false |
|
| dedup_max_size (Max size of the deduplication set) |
integer |
Default: 1024Minimum: 1 |
|
| dedup_strategy (Deduplication strategy: reject - reject messages(dlq), ignore - ignore messages) |
string |
(DEPRECATED) Property 'dedup_strategy' is now deprecated. The only supported strategy is 'ignore'. Please remove from the configuration. Default: "ignore"Enum: "reject", "ignore" |
|
| duration (Time (in ms) after which data will be read from stream even if read_batch_size was not reached) |
integer, string |
Default: 100Pattern: ^\${.*}$Minimum: 1 |
|
| write_batch_size (The batch size for writing data to target Redis database. Should be less or equal to the read_batch_size) |
integer, string |
Default: 200Pattern: ^\${.*}$Minimum: 1 |
|
| error_handling (Error handling strategy: ignore - skip, dlq - store rejected messages in a dead letter queue) |
string |
Default: "dlq"Pattern: ``^${.*}$ |
ignore |
| dlq_max_messages (Dead letter queue max messages per stream) |
integer, string |
Default: 1000Pattern: ^\${.*}$Minimum: 1 |
|
| target_data_type (Target data type: hash/json - RedisJSON module must be in use in the target DB) |
string |
Default: "hash"Pattern: ``^${.*}$ |
hash |
| json_update_strategy (Target update strategy: replace/merge - RedisJSON module must be in use in the target DB) |
string |
(DEPRECATED) Property 'json_update_strategy' will be deprecated in future releases. Use 'on_update' job-level property to define the json update strategy. Default: "replace"Pattern: ``^${.*}$ |
replace |
| initial_sync_processes (Number of processes RDI Engine creates to process the initial sync with the source) |
integer, string |
Default: 4Pattern: ^\${.*}$Minimum: 1Maximum: 32 |
|
| idle_sleep_time_ms (Idle sleep time (in milliseconds) between batches) |
integer, string |
Default: 200Pattern: ^\${.*}$Minimum: 1Maximum: 999999 |
|
| idle_streams_check_interval_ms (Interval (in milliseconds) for checking new streams when the stream processor is idling) |
integer, string |
Default: 1000Pattern: ^\${.*}$Minimum: 1Maximum: 999999 |
|
| busy_streams_check_interval_ms (Interval (in milliseconds) for checking new streams when the stream processor is busy) |
integer, string |
Default: 5000Pattern: ^\${.*}$Minimum: 1Maximum: 999999 |
|
| wait_enabled (Checks if the data has been written to the replica shard) |
boolean |
Default: false |
|
| wait_timeout (Timeout in milliseconds when checking write to the replica shard) |
integer, string |
Default: 1000Pattern: ^\${.*}$Minimum: 1 |
|
| retry_on_replica_failure (Ensures that the data has been written to the replica shard and keeps retrying if not) |
boolean |
Default: true |
Additional Properties: not allowed
targets: Target connections
Properties
| Name | Type | Description | Required |
|---|---|---|---|
| connection (Connection details) |
object |
targets.connection: Connection details
Properties (Pattern)
| Name | Type | Description | Required |
|---|---|---|---|
| .* | |||
| additionalProperties |