redis.write

Summary: Write to a Redis Enterprise database

Overview

Write to a Redis Enterprise database

**Properties**

| Name                    | Type      | Description                                                                                | Required |
| ----------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------- |
| **connection**          | `string`  | Name of Redis connection specified in `config.yaml`.
Defaults to connection named `target`.                                                                                | no       |
| **data_type**
      | `string`  | Type of Redis target data structure.
Enum: `hash`(default), `json`, `set`, `sorted_set`, `stream`, `string`.
Takes precedence over system property `target_data_type`. | no       |
| [**nest**](#nest)       | `object`  | Nest (embed) object within a different key.
If nesting is specified, the following parameters are ignored: `key`, `args` and `on_update`.                                  | no       |
| **key**                 | `object`  | Definition of the target Redis key.
                                                                                | yes      |
| ∟ **expression** | `string`  | Expression used to calculate the target key.                                                                                | yes      |
| ∟ **language**   | `string`  | Language used to define the expression.
Enum: `jmespath`, `sql`.                                                                                | yes      |
| [**args**](#args)       | `object`  | Arguments for modifying the target key.
Specific to the data type.                                                                                | no       |
| **mapping**             | `array`   | Array of fields (or `field: alias` pairs) to be written to a Redis key.
Supported for hashes, json documents and streams only.                                             | no       |
| **on_update**           | `string`  | Target key update strategy
Enum: `merge`, `replace` (default).                                                                                | no       |
| **expire**              | `integer` | TTL in seconds for the modified key to expire.
If not specified (or `expire: 0`), the target key will never expire.                                                        | no       |

> Notes:

- Job parameters always override system properties. In particular, `data_type` will override `target_data_type` and `on_update` will override `json_update_strategy` properties respectively.
- Mapping for **JSON documents** supports nested paths (e.g. `path.to.field`) which results in creating a nested element in Redis key. When a dot is used in a field name, it must be escaped with a backslash (e.g. `path\.to\.field`). Nested paths are not supported for hashes and streams.
- For **strings** Write-behind will automatically assume `on_update: replace` regardless of what was declared in the job file. Appends and increments are not currently supported.
- For **streams** Write-behind will ignore `on_update` property since they are append only.

> Notes:

- Job parameters always override system properties. In particular, `data_type` will override `target_data_type` and `on_update` will override `json_update_strategy` properties respectively.
- Mapping for JSON documents supports nested paths (e.g. `path.to.field`) which results in creating a nested element in Redis key. When a dot is used in a field name, it must be escaped with a backslash (e.g. `path\.to\.field`). Nested paths are not supported for hashes and streams.

**Example**

[code example]

args: object

Arguments for modifying the target key

**Properties**

| Name       | Type     | Description                                                                                | Required |
| ---------- | -------- | ---------------------------------------------------------------------------------------------- | -------- |
| **score**  | `string` | Field name used as a score for sorted sets.
Valid for sorted sets only.                    | yes      |
| **member** | `string` | Field name used as a member for sets and sorted sets.
Valid for sets and sorted sets only. | yes      |
| **value**  | `string` | Field name used as a value for strings.
Valid for strings only.                            | yes      |

nest: object

Nest (embed) object within a different key

**Properties**

| Name                     | Type     | Description                                                                                | Required |
| ------------------------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- |
| **parent**               | `object` | Parent object definition.                                                                                | yes      |
| ∟ **server_name** | `string` | Server name.                                                                                | no       |
| ∟ **schema**      | `string` | Schema name.                                                                                | no       |
| ∟ **table**       | `string` | Parent table.                                                                                | yes      |
| **parent_key**           | `string` | Field name used to identify the parent key (usually FK).                                                                                | yes      |
| **child_key**            | `string` | Optional field name used to identify the parent key value (if different from **parent_key**) in the child record. If not specified, then the field name defined in **parent_key** is used to lookup the value. | no       |
| **nesting_key**          | `string` | Field name used to create the nesting key (usually PK).                                                                                | yes      |
| **path**                 | `string` | Path, where the nested object should reside in a parent document.<br/>Must start with the root (e.g. `$.<children-elements-here>`)                                                                             | yes      |
| **structure**            | `string` | Data structure used to represent the object in a parent document (`map` is the only supported value).                                                                                | no       |

> Notes:

- When `nest` object is defined, Write-behind will automatically assume `data_type: json` and `on_update: merge` regardless of what was declared in the job file.
- Nesting job cannot be used together with any of the these properties: `key`, `args`. The key is automatically calculated based on the following template: `<parent_table>:<parent_key>:<parent_key.value | child_key.value>`.
- When `expire` is specified, it will be applied to the **parent** key. Therefore all nested objects will expire together with the parent key.

**Example**

[code example]

> Note: In the example above `child_key` is not needed, because the FK in the child table `InvoiceLine` is defined using the same field name `InvoiceId` as in the parent table `Invoice`. If instead a FK was defined differently (e.g. InvoiceLine.ParentInvoivceId = Invoice.InvoiceId), then `child_key` parameter would be required to describe this relationship in the child job.

source:
  server_name: chinook
  schema: public
  table: invoice
output:
  # this block will use the default connection: target - since no explicit connection is specified,
  # the data will be written in a JSON format as the data_type: json is specified for the block
  - uses: redis.write
    with:
      data_type: json
      key:
        expression: concat(['invoice_id:', InvoiceId])
        language: jmespath
      mapping: # only the fields listed below will be written to a JSON document
        - InvoiceId: id # this will create an element with a different name
        - InvoiceDate: date
        - BillingAddress: address.primary.street # this will create a nested element in the JSON document
        - BillingCity: "address.primary.city name" # this will create a nested element with a space in the name
        - BillingState: address.primary.state
        - BillingPostalCode: "address.primary.zip\\.code" # this will create a nested element with a dot in the name
        - Total # this will create an element with the same name as the original field
      on_update: merge
  # this block will use the explicitly specified connection: target1 - it must be defined in config.yaml
  # the data will be written to the corresponding Redis set, based on a value of the key expression
  - uses: redis.write
    with:
      connection: target
      data_type: set
      key:
        expression: concat(['invoices:', BillingCountry])
        language: jmespath
      args:
        member: InvoiceId
  # this block will use the explicitly specified connection: target1 - it must be defined in config.yaml
  # the data will be written to the Redis sorted set named invoices:sorted as specified in the key expression
  - uses: redis.write
    with:
      connection: target1
      data_type: sorted_set
      key:
        expression: "`invoices:sorted`"
        language: jmespath
      args:
        score: Total
        member: InvoiceId
  # this block will use the specified connection: target2 - this, again, has to be defined in config.yaml
  # the data will be written to a Redis stream named invoice:events as specified in the key expression
  - uses: redis.write
    with:
      connection: target2
      data_type: stream
      key:
        expression: "`invoice:events`"
        language: jmespath
      mapping: # only the fields listed below will be written to a stream message, with two of them renamed as message_id and country
        - InvoiceId: message_id
        - BillingCountry: country
        - Total
  # this block will use the default connection: target - since no explicit connection is specified,
  # the data will be written to a Redis string as the data_type: string is specified for the block
  - uses: redis.write
    with:
      data_type: string
      key:
        expression: concat(['Invoice:', InvoiceId])
        language: jmespath
      args:
        value: Total # only the Total field will be written to a string
      expire: 100 # the key will expire in 100 seconds

source:
  server_name: chinook
  schema: public
  table: InvoiceLine
output:
  - uses: redis.write
    with:
      nest:
        parent:
          # server_name: chinook
          # schema: public
          table: Invoice
        nesting_key: InvoiceLineId
        parent_key: InvoiceId
        # child_key: ParentInvoiceId
        path: $.InvoiceLineItems
        # structure: map


relational.write

Summary: Write into a SQL-compatible data store

Overview

Write into a SQL-compatible data store

**Properties**

| Name                                                                                | Type     | Description                                                                                | Required |
| --------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- |
| **connection**
(The connection to use for loading)                                                                                | `string` | Logical connection name as defined in the connections.yaml
                                                                                | yes      |
| **schema**
(The table schema of the target table)                                                                                | `string` | If left blank, the default schema of this connection will be used as defined in the connections.yaml
                                                            | yes      |
| **table**
(The target table name)                                                                                | `string` | Target table name
                                                                                | yes      |
| [**keys**](#keys)
(Business keys to use in case of \`load_strategy\` is UPSERT or working with \`opcode_field\`)                                      | `array`  |                                                                                | no       |
| [**mapping**](#mapping)
(Fields to write)                                                                                | `array`  |                                                                                | no       |
| **foreach**
(Split a column into multiple records with a JMESPath expression)                                                                         | `string` | Use a JMESPath expression to split a column into multiple records. The expression should be in the format column: expression.
Pattern: `^(?!:).*:.*(?<!:)$`
 | no       |
| **opcode_field**                                                                                | `string` | Name of the field in the payload that holds the operation (c - create, d - delete, u - update) for this record in the DB
                                        | no       |
| **load_strategy**                                                                                | `string` | type of target
Default: `"APPEND"`
Enum: `"APPEND"`, `"REPLACE"`, `"UPSERT"`, `"TYPE2"`
                                                                 | no       |
| **active_record_indicator**                                                                                | `string` | Used for `TYPE2` load_strategy. An SQL expression used to identify which rows are active
                                                                        | no       |
| [**inactive_record_mapping**](#inactive_record_mapping)
(Used for \`TYPE2\` load_strategy\. The columns mapping to use to close out an active record) | `array`  | A list of columns to use. Use any valid SQL expression for the source. If 'target' is omitted, will default to the name of the source column
Default:
      | no       |

**Additional Properties:** not allowed

**No properties.**

**Not [required1]:**
**No properties.**

**Example**

[code example]

keys\[\]: Business keys to use in case of \`load_strategy\` is UPSERT or working with \`opcode_field\`

**Items: name of column**

**No properties.**

**Example**

[code example]

mapping\[\]: Fields to write

inactive_record_mapping\[\]: Used for \`TYPE2\` load_strategy\. The columns mapping to use to close out an active record

A list of columns to use. Use any valid SQL expression for the source. If 'target' is omitted, will default to the name of the source column

**No properties.**

**Example**

[code example]

id: load_snowflake
type: relational.write
properties:
  connection: eu_datalake
  table: employees
  schema: dbo
  load_strategy: APPEND

- fname
- lname: last_name
- address
- gender

- source: CURRENT_DATE
  target: deletedAt
- source: "'Y'"
  target: is_active


remove_field

Summary: Remove fields

Overview

Remove fields

**Option 1 (alternative):**
Remove multiple fields

**Properties**

| Name                         | Type       | Description | Required |
| ---------------------------- | ---------- | ----------- | -------- |
| [**fields**](#option1fields) | `object[]` | Fields
 | yes      |

**Additional Properties:** not allowed

**Example**

[code example]

**Option 2 (alternative):**
Remove one field

**Properties**

| Name      | Type     | Description | Required |
| --------- | -------- | ----------- | -------- |
| **field** | `string` | Field
  | yes      |

**Additional Properties:** not allowed
**Example**

[code example]

Option 1: fields\[\]: array

Fields

**Items**

**Item Properties**

| Name      | Type     | Description | Required |
| --------- | -------- | ----------- | -------- |
| **field** | `string` | Field<br/>  | yes      |

**Item Additional Properties:** not allowed

**Example**

[code example]

source:
  server_name: redislabs
  schema: dbo
  table: emp
transform:
  - uses: remove_field
    with:
      fields:
        - field: credit_card
        - field: name.mname

source:
  server_name: redislabs
  schema: dbo
  table: emp
transform:
  - uses: remove_field
    with:
      field: credit_card


rename_field

Overview

Rename fields. All other fields remain unchanged.

**Option 1 (alternative):**
Rename multiple fields

**Properties**

| Name                         | Type       | Description | Required |
| ---------------------------- | ---------- | ----------- | -------- |
| [**fields**](#option1fields) | `object[]` | Fields
 | yes      |

**Additional Properties:** not allowed
**Example**

[code example]

**Option 2 (alternative):**
Rename one field

**Properties**

| Name           | Type     | Description     | Required |
| -------------- | -------- | --------------- | -------- |
| **from_field** | `string` | From field
 | yes      |
| **to_field**   | `string` | To field
   | yes      |

**Additional Properties:** not allowed

**Example**

[code example]

Option 1: fields\[\]: array

Fields

**Items**

**Item Properties**

| Name           | Type     | Description     | Required |
| -------------- | -------- | --------------- | -------- |
| **from_field** | `string` | From field
 | yes      |
| **to_field**   | `string` | To field
   | yes      |

**Item Additional Properties:** not allowed

**Example**

[code example]

source:
  server_name: redislabs
  schema: dbo
  table: emp
transform:
  - uses: rename_field
    with:
      fields:
        - from_field: name.lname
          to_field: name.last_name
        - from_field: name.fname
          to_field: name.first_name

source:
  server_name: redislabs
  schema: dbo
  table: emp
transform:
  - uses: rename_field
    with:
      from_field: name.lname
      to_field: name.last_name


Data type conversion

Summary: Data type conversion reference

Overview

This document describes default conversions of data types of supported databases into target redis data types.

General ANSI SQL data types

| Source data type                                                                          | Target data type for HASH                                                                                | Example for HASH                                                                                |
| :---------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| array                                                                                | not supported                                                                                |                                                                                |
| bigint                                                                                | string                                                                                | 2147483648 will be saved as '2147483648'                                                                                |
| binary                                                                                | bytes string
bytes or base64-encoded String, or a hex-encoded String, based on the connector configuration property setting `binary.handling.mode`                                                                                | When `binary.handling.mode = bytes` the string `'hello'` will be added to the table as the binary string `'0x68656C6C6F'`,will be converted by Debezium to `'aGVsbG8AAAAAAAAAAAAAAAAAAAA='` and will be stored in Redis target database as: `'aGVsbG8AAAAAAAAAAAAAAAAAAAA='` |
| bit string                                                                                | not supported                                                                                |                                                                                |
| blob                                                                                | bytes string
bytes or base64-encoded String, or a hex-encoded String, based on the connector configuration property setting `binary.handling.mode`                                                                                | When `binary.handling.mode = bytes` the binary image file ,which was loaded into the table ,will be converted by Debezium to bytes and will be stored in Redis target database as bytes string                                                                               |
| boolean                                                                                | string                                                                                | The boolean value: false will be converted by the Applier to 0 and will be saved in Redis target database as the string '0'                                                                                |
| char                                                                                | string                                                                                | When PostgreSQL data type is char(14) 'hello world' will be saved as 'hello world   '                                                                                |
| date                                                                                | string
Mapped to ms.microsec since epoch                                                                                | PG field value: `'2021-01-29'` will be converted by Debezium to the int `18656` (number of dates since epoch), which will be converted to number of ms since epoch and will be stored in Redis target database as: `'1611878400000'`                                         |
| integer                                                                                | string                                                                                | `2147483640` will be saved as `'2147483640'`                                                                                |
| interval                                                                                | not supported                                                                                |                                                                                |
| null                                                                                |                                                                                | The field with `null` value will be sent by Debezium as `null` and will not be stored in Redis target database                                                                                |
| numeric                                                                                | string
Debezium configuration parameter `decimal.handling.mode` determines how the connector maps decimal values.When `decimal.handling.mode = 'precision'` the binary string,recieved by Debezium,will be converted to its corresponding numeric value and will be stored in Redis target database as string | PG field value: `4521398.56` will be converted by Debezium to binary string `'GvMbUA=='`, which will be converted to numeric value and will be stored in Redis target database as: `'4521398.56'`                                                                            |
| smallint                                                                                | string                                                                                | 32767 will be saved as '32767'                                                                                |
| text                                                                                | string                                                                                | 'This is a very long text for the PostgreSQL text column'                                                                                |
| time                                                                                | string
mapped to number of seconds past midnight                                                                                | `'14:23:46'` will be converted to `'51826000000'` sec                                                                                |
| PostgreSQL, Oracle, Cassandra: timestamp
 MySQL, SQL Server: datetime               | string
mapped to ms.microsec since epoch.
 SQL Server datetime format: `YYYY-MM-DD hh:mm:ss[.nnn]`,
range: `1753-01-01` through `9999-12-31`                                                                                | PG field value: `'2018-06-20 15:13:16.945104'` will be converted by Debezium to `'1529507596945104'`(micro seconds) and will be stored in Redis target database as `'1529507596945.104'`                                                                                |
| PosrgreSQL: timestamptz
Oracle: timestamp with local timezone
MySQL: timestamp | string
converted to UTC and stored as number of ms.microsec since epoch                                                                                |                                                                                |
| timetz                                                                                | string
converted to UTC and stored as number of seconds past midnight                                                                                | PG field value: `'14:23:46.12345'` will be converted by Debezium to the string `'12:23:46.12345Z'` and will be stored in Redis target database as: `'44638.345'`                                                                                |
| varbinary                                                                                | bytes string
bytes or base64-encoded String, or a hex-encoded String, based on the connector configuration property setting `binary.handling.mode`                                                                                | When `binary.handling.mode = bytes` the string `'hello'` will be added to the table as the binary string `'0x68656C6C6F'`,will be converted by Debezium to `'aGVsbG8='` and will be stored in Redis target database as: `'aGVsbG8='`                                         |
| varchar                                                                                | string                                                                                | 'hello world' will be saved as 'hello world'                                                                                |
| xml                                                                                | string                                                                                |

| Source data type                                                                          | Target data type for JSON                                                                                | Example for JSON                                                                                |
| :---------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| array                                                                                | not supported                                                                                |                                                                                |
| bigint                                                                                | number                                                                                | 2147483648 will be saved as 2147483648                                                                                |
| binary                                                                                | bytes string                                                                                | When `binary.handling.mode = bytes` the string `'hello'` will be added to the table as the binary string `'0x68656C6C6F'`,will be converted by Debezium to `'aGVsbG8AAAAAAAAAAAAAAAAAAAA='` and will be stored in Redis target database as: `'aGVsbG8AAAAAAAAAAAAAAAAAAAA='` |
| bit string                                                                                | not supported                                                                                |                                                                                |
| blob                                                                                | bytes string                                                                                | When `binary.handling.mode = bytes` the binary image file ,which was loaded into the table ,will be converted by Debezium to bytes and will be stored in Redis target database as bytes string                                                                               |
| boolean                                                                                | boolean                                                                                | The boolean value:true will be saved in Redis target database as boolean data type with value True                                                                                |
| char                                                                                | string                                                                                | When PostgreSQL data type is char(14) 'hello world' will be saved as 'hello world   '                                                                                |
| date                                                                                | number
Mapped to ms.microsec since epoch                                                                                | PG field value: `2021-01-29` will be converted by Debezium to the int 18656 (number of dates since epoch), which will be converted to number of ms since epoch and will be stored in Redis target database as: `1611878400000`                                               |
| integer                                                                                | number                                                                                | `2147483640` will be saved as `2147483640`                                                                                |
| interval                                                                                | not supported                                                                                |                                                                                |
| null                                                                                |                                                                                | The field with `null` value will be sent by Debezium as `null` and will be stored in Redis target database with the value `null`                                                                                |
| numeric                                                                                | Debezium configuration parameter `decimal.handling.mode` determines how the connector maps decimal values. When `decimal.handling.mode = 'precision'` the binary string received by Debezium will be converted to its corresponding numeric value and will be stored in Redis target database db as number. When `decimal.handling.mode = 'string'` the string ,received by Debezium, will be stored in Redis target database as string. When `decimal.handling.mode = 'double'` the double value received by Debezium will be stored in Redis target database as number. | `decimal.handling.mode = string`: PG field value: `4521398.56` will be recieved by Debezium as the string `'4521398.56'`, and will be stored in Redis target database as the string: '`4521398.56`' .                                                                        |
| smallint                                                                                | number                                                                                | 32767 will be saved as 32767                                                                                |
| text                                                                                | string                                                                                |                                                                                |
| time                                                                                | number number
 mapped to the number of seconds past midnight                                                                                | `'14:23:46'` will be converted to `51826000000` sec                                                                                |
| PostgreSQL, Oracle, Cassandra: timestamp
 MySQL, SQL Server: datetime               | decimal
mapped to ms.microsec since epoch.
 SQL Server datetime format: `YYYY-MM-DD hh:mm:ss[.nnn]`,
range: `1753-01-01` through `9999-12-31`                                                                                | PG field value: `'2018-06-20 15:13:16.945104'` will be converted by Debezium to `'1529507596945104'`(micro seconds) and will be stored in Redis target database as `1529507596945.104                                                                                |
| PosrgreSQL: timestamptz
Oracle: timestamp with local timezone
MySQL: timestamp | number
converted to UTC and stored as number of ms.microsec since epoch                                                                                |                                                                                |
| timetz                                                                                | number
converted to UTC and stored as number of ms.microsec since epoch                                                                                | PG field value: `'14:23:46.12345'` will be converted by Debezium to the string `'12:23:46.12345Z'` and will be stored in Redis target database as: `44638.345`                                                                                |
| varbinary                                                                                | bytes string
bytes or base64-encoded String, or a hex-encoded String, based on the connector configuration property setting `binary.handling.mode`                                                                                | When `binary.handling.mode = bytes` the string `'hello'` will be added to the table as the binary string `'0x68656C6C6F'`,will be converted by Debezium to `'aGVsbG8='` and will be stored in Redis target database as: `'aGVsbG8='`                                         |
| varchar                                                                                | string                                                                                | 'hello world' will be saved as 'hello world’                                                                                |
| xml                                                                                | string                                                                                |                                                                                |

Cassandra data types

| Source data type      | Target data type for HASH                    | Example for HASH                                                                                |
| :-------------------- | :------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ascii                 | string                                       |                                                                                |
| counter (64-bit long) | string                                       | `2`                                                                                |
| date                  | not supported                                |                                                                                |
| decimal               | not supported                                |                                                                                |
| double                | string                                       | '1.007308023'                                                                                |
| duration              | not supported                                |                                                                                |
| float                 | string                                       | The number `-3.4E+38` will be received by debezium as `-3.4E38`, will be converted by the Applier to `-340000000000000000000000000000000000000` and will be saved in Redis target database as `'-340000000000000000000000000000000000000'`            |
| frozen                | string                                       | `{'10.10.11.1', '10.10.10.1', '10.10.12.1'}` will be saved in Redis target database as a string: `'{'10.10.11.1', '10.10.10.1', '10.10.12.1'}'`                                                                                |
| frozen udt            | string                                       | `{'city': 'City','street': 'Street','streetno': 2,'zipcode': '02-212'}` will be saved in Redis target database as a string: `'{'city': 'City','street': 'Street','streetno': 2,'zipcode': '02-212'}'`                                                 |
| inet                  | string
IPv4 and IPv6 network addresses | The IP address `4.35.221.243` will be converted by debezium to `'/4.35.221.243'` and will be saved in Redis target database as `'/4.35.221.243'`                                                                                |
| list                  | string                                       | The list `['New York', 'Paris','London','New York']` will be sent by debezium as array of strings: `['New York', 'Paris','London','New York']` and will be saved in Redis target database as the string `"['New York', 'Paris','London','New York']"` |
| map                   | string                                       | `{'fruit' : 'Apple', 'band' : 'Beatles'}` will be saved in Redis target database as a string: `'{'fruit' : 'Apple', 'band' : 'Beatles'}'`                                                                                |
| set                   | string                                       | The set `{'y','n'}` will be saved in Redis target database as the string `'{'y','n'}'                                                                                |
| tinyint               | string                                       |                                                                                |
| uuid                  | string                                       | b9980b96-a85b-411c-b8e7-4be55c123793                                                                                |
| tuple                 | string                                       | The tuple `{ "field1": 1, "field2": "testing tuple", "field3": 2 }` will be saved in Redis target database as a string: `' { "field1": 1, "field2": "testing tuple", "field3": 2 }'`                                                                  |
| varint                | not supported                                |                                                                                |

| Source data type      | Target data type for JSON                    | Example for JSON                                                                                |
| :-------------------- | :------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ascii                 | string                                       |                                                                                |
| counter (64-bit long) | number                                       | 2                                                                                |
| date                  | not supported                                |                                                                                |
| decimal               | not supported                                |                                                                                |
| double                | number                                       | 1.007308023                                                                                |
| duration              | not supported                                |                                                                                |
| float                 | number                                       | The number -3.4E+38 will be received by debezium as `-3.4E38` and will be saved in Redis target database as `-1e+38`                                                                                |
| frozen                | array                                        | `{'10.10.11.1', '10.10.10.1', '10.10.12.1'}` will be saved in Redis target database as an array: `{'/10.10.11.1', '/10.10.10.1', '/10.10.12.1'}`                                                                                |
| frozen udt            | object                                       | `{'city': 'City','street': 'Street','streetno': 2,'zipcode': '02-212'}` will be saved in Redis target database as string will be saved in Redis target database as an object: `{'city': 'City','street': 'Street','streetno': 2,'zipcode': '02-212'}` |
| inet                  | string
IPv4 and IPv6 network addresses | The IP address `4.35.221.243` will be converted by debezium to `'/4.35.221.243'` and will be saved in Redis target database as `'/4.35.221.243'`                                                                                |
| list                  | array                                        | The list `['New York', 'Paris','London','New York']` will be sent by debezium as array of strings: `['New York', 'Paris','London','New York']` and will be saved in Redis target database as an array: `['New York', 'Paris','London','New York']`    |
| map                   | object                                       | `{'fruit' : 'Apple', 'band' : 'Beatles'}` will be saved in Redis target database as an object: `{'fruit' : 'Apple', 'band' : 'Beatles'}`                                                                                |
| set                   | array                                        | The set `{'y','n'}` will be saved in Redis target database as an array: `{'y','n'}`                                                                                |
| tinyint               |                                              |                                                                                |
| uuid                  | string                                       | b9980b96-a85b-411c-b8e7-4be55c123793                                                                                |
| tuple                 | object                                       | The tuple `{ "field1": 1, "field2": "testing tuple", "field3": 2 }` will be saved in Redis target database as an object: `{ "field1": 1, "field2": "testing tuple", "field3": 2 }`                                                                    |
| varint                | not supported                                |                                                                                |

MySQL and MariaDB data types

| Source data type   | Source data type for HASH | Example for Hash                                                                                |
| ------------------ | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| enum               | string                    | MySQL enum value `'cat'` will be stored in Redis target database as `'cat'`                                                                                |
| geometry           | not supported             |                                                                                |
| geometrycollection | not supported             |                                                                                |
| json               | string                    | {"pid": 102, "name": "name2"}                                                                                |
| linestring         | not supported             |                                                                                |
| multilinestring    | not supported             |                                                                                |
| multipoint         | not supported             |                                                                                |
| multypolygon       | not supported             |                                                                                |
| polygon            | not supported             |                                                                                |
| set                | string                    | `'1,2,3'` will be stored in Redis target database as: `'1,2,3'`                                                                                |
| year               | string                    | The value `'55'` will be stored in the database as `2055` and will be sent by Debezium as int32 data type with value `2055`. It will be stored in Redis target database as the string `'2055'` as well |

| Source data type   | Target data type for JSON | Example for JSON                                                                                |
| ------------------ | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| enum               | string                    | MySQL enum value `'cat'` will be stored in Redis target database as `'cat'`                                                                                |
| geometry           | not supported             |                                                                                |
| geometrycollection | not supported             |                                                                                |
| json               | object                    | {"pid": 102, "name": "name2"}                                                                                |
| linestring         | not supported             |                                                                                |
| multilinestring    | not supported             |                                                                                |
| multipoint         | not supported             |                                                                                |
| multypolygon       | not supported             |                                                                                |
| polygon            | not supported             |                                                                                |
| set                | string                    | `'1,2,3'` will be stored in Redis target database as: `'1,2,3'`                                                                                |
| year               | number                    | The value `'55'` will be stored in the database as `2055` and will be sent by Debezium as int32 data type with value `2055`. It will be stored in Redis target database as the number `2055` as well |

Oracle data types

| Source data type                                                                   | Source data type for HASH                                       | Example for Hash                                                                                |
| ---------------------------------------------------------------------------------- | --------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| bfile                                                                              | not supported                                                   |                                                                                |
| binary_double                                                                      | string                                                          | '1.7E+308'                                                                                |
| binary_float                                                                       | string                                                          | '3.40282E38'                                                                                |
| clob                                                                               | string                                                          | large block of text                                                                                |
| float,real,double precision
 real = FLOAT(63),double precision = FLOAT(126). | string                                                          | The value `-3.402E+38` will be saved in Redis target database as the string `'-340200000000000000000000000000000000000'` when Debezium configuration parameter `decimal.handling.mode = 'double'`                    |
| long raw                                                                           | not supported                                                   |                                                                                |
| nchar                                                                              | string - is Unicode data type that can store Unicode characters | The string `'testing hebrew שלום'` will be stored in Redis target database as '`testing hebrew \xd7\xa9\xd7\x9c\xd7\x95\xd7\x9d`         '                                   |
| nclob                                                                              | not supported                                                   |                                                                                |
| number(p,s)                                                                        | string                                                          | '10385274000.32'                                                                                |
| nvarchar                                                                           | string - is Unicode data type that can store Unicode characters | The string `testing hebrew שלום'` will be stored in Redis target database as '`testing hebrew \xd7\xa9\xd7\x9c\xd7\x95\xd7\x9d`'                                                                                |
| raw                                                                                | not supported                                                   |                                                                                |
| rowid                                                                              | string                                                          | AAAR1QAAOAAAACFAAA                                                                                |
| timestamp with tz                                                                  | string                                                          | `'2021-12-30 14:23:46'` will be converted by Debezium to the string `'2021-12-30T14:23:46+02:00'` and will be stored in Redis target database as the string: `'1611878400000'` which is the number of ms since epoch |
| urowid                                                                             | not supported                                                   |

| Source data type                                                                   | Target data type for JSON                                       | Example for JSON                                                                                |
| ---------------------------------------------------------------------------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| bfile                                                                              | not supported                                                   |                                                                                |
| binary_double                                                                      | number                                                          | 1.7E+308                                                                                |
| binary_float                                                                       | number                                                          | 3.40282E38                                                                                |
| clob                                                                               | string                                                          | large block of text                                                                                |
| float,real,double precision
 real = FLOAT(63),double precision = FLOAT(126). | number/string                                                   | the value `-3.402E+38` will be saved in Redis target database as the number `-340200000000000000000000000000000000000` when Debezium configuration parameter `decimal.handling.mode = 'double'`                    |
| long raw                                                                           | not supported                                                   |                                                                                |
| nchar                                                                              | string - is Unicode data type that can store Unicode characters | The string `'testing hebrew שלום'` will be stored in Redis target database as '`testing hebrew \xd7\xa9\xd7\x9c\xd7\x95\xd7\x9d`         '                                 |
| nclob                                                                              | not supported                                                   |                                                                                |
| number(p,s)                                                                        | number                                                          | 10385274000.32                                                                                |
| nvarchar                                                                           | string - is Unicode data type that can store Unicode characters | The string `testing hebrew שלום'` will be stored in Redis target database as '`testing hebrew \xd7\xa9\xd7\x9c\xd7\x95\xd7\x9d`'                                                                                |
| raw                                                                                | not supported                                                   |                                                                                |
| rowid                                                                              | string                                                          | AAAR1QAAOAAAACFAAA                                                                                |
| timestamp with tz                                                                  | number                                                          | `'2021-12-30 14:23:46'` will be converted by Debezium to the string `'2021-12-30T14:23:46+02:00'` and will be stored in Redis target database as the number: `1611878400000` which is the number of ms since epoch |
| urowid                                                                             | not supported                                                   |                                                                                |

PostgreSQL data types

| Source data type | Source data type for HASH                     | Example for Hash                                                                                |
| ---------------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| box              | not supported                                 |                                                                                |
| cidr             | string
IPv4 and IPv6 networks addresses. | '4.35.221.243/32'                                                                                |
| circle           | not supported                                 |                                                                                |
| domain           | string                                        |                                                                                |
| hstore           | string                                        | '{"pages":"368","author":"Katherine Dunn","category":"fiction"}'                                                                                |
| inet             | string
IPv4 and IPv6 network addresses   | '4.35.221.243'                                                                                |
| json             | string                                        | "{"guid":  "9c36adc1-7fb5-4d5b-83b4-90356a46061a",
"name": "Angela Barton",
"is_active": null,
"company": "Magnafone"
"address": \"178 Howard Place, Gulf, Washington,702",
"registered": "2009-11-07T08:53:22 +08:00",
"latitude": 19.793713,
"longitude": 86.513373,
"tags": ["enim","aliquip","qui\" ]}" |
| line             | not supported                                 |                                                                                |
| macaddr          | string
 mac addresses                    | '08:00:2b:01:02:03'                                                                                |
| money            | string                                        | When `decimal.handling.mode = 'double'` the money value `-8793780.01` will be received by Debezium as `-8793780.01` with double data type, and will be stored in Redis target database as the string'`-8793780.01`'                                                                                |
| path             | not supported                                 |                                                                                |
| point            | not supported                                 |                                                                                |
| polygon          | not supported                                 |                                                                                |
| uuid             | string                                        | 'a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11'                                                                                |
|                  |                                               |                                                                                |

| Source data type | Target data type for JSON                     | Example for JSON                                                                                |
| ---------------- | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| box              | not supported                                 |                                                                                |
| cidr             | string
IPv4 and IPv6 networks addresses. | '4.35.221.243/32'                                                                                |
| circle           | not supported                                 |                                                                                |
| domain           | string                                        |                                                                                |
| hstore           | string                                        | '{"pages":"368","author":"Katherine Dunn","category":"fiction"}                                                                                |
| inet             | string
IPv4 and IPv6 network addresses   | '4.35.221.243'                                                                                |
| json             | object                                        | {"guid":  "9c36adc1-7fb5-4d5b-83b4-90356a46061a",
"name": "Angela Barton",
"is_active": null,
"company": "Magnafone"
"address": \"178 Howard Place, Gulf, Washington,702",
"registered": "2009-11-07T08:53:22 +08:00",
"latitude": 19.793713,
"longitude": 86.513373,
"tags": ["enim","aliquip","qui\" ]} |
| line             | not supported                                 |                                                                                |
| macaddr          | string
 mac addresses                   | '08:00:2b:01:02:03'                                                                                |
| money            | string                                        | When `decimal.handling.mode = 'double'` the money value `-8793780.01` will be received by Debezium as `-8793780.01`, with double data type and will be stored in Redis target database as the number `-8793780.01`                                                                                |
| path             | not supported                                 |                                                                                |
| point            | not supported                                 |                                                                                |
| polygon          | not supported                                 |                                                                                |
| uuid             | string                                        | 'a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11'                                                                                |
|                  |                                               |                                                                                |

SQL server data types

| Source data type        | Source data type for HASH                                                                                | Example for Hash                                                                                |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| bit                     | string                                                                                | When value `>0` it will be converted by Debezium to true and will be saved in Redis target database as '`1`' while when value `= 0` it will be converted by Debezium to false and will be saved in Redis target database as '`0`'                                                                                |
| datetime2               | string
Represents the number of milliseconds since the epoch, and does not include timezone information.                                                                        | When Debezium configuration parameter `time.precision.mode='connect'`, the value '`2018-06-20 15:13:16.945104`' will be converted by Debezium to the value '`1529507596945104`' and will be saved in Redis target database database as the string `'1529507596945.104`'                                                   |
| datetimeoffset          | string                                                                                | When Debezium configuration parameter `decimal.handling.mode = 'precision'`, the datetimeoffset datatype value '`12-10-25 12:32:10 +01:00`' will be converted to the string '`2025-12-10T12:32:10+01:00'` and will be saved in Redis target database as `1765366330000`                                                   |
| decimal,float,real      | string
 range of values: decimal-10^38 +1 to 10^38,float-1.79E+308 to -2.23E-308, 0 and 2.23E-308 to 1.79E+308,real:- 3.40E + 38 to -1.18E - 38, 0 and 1.18E - 38 to 3.40E + 38 | When Debezium configuration parameter `decimal.handling.mode = 'precision'` the value '`-3.402E+38`' will be converted by Debezium to the binary string '`/wAP3QCzc/wpiIGe8AAAAAA=`' and will be saved in Redis target database as the string '`-340200000000000000000000000000000000000`'                                |
| image                   | string
 Variable-length binary data from 0 through 2,147,483,647 bytes.                                                                                |                                                                                |
| money                   | string
 range of values: -922,337,203,685,477,5808 to 922,337,203,685,477.5807                                                                                | When Debezium configuration parameter `decimal.handling.mode = 'precision'` the value `922337203685477.5807` will be converted by Debezium to the binary '`f/////////8=`' string and will be saved in Redis target database as the string '`922337203685477.5807`'                                                        |
| nchar                   | string - fixed-size string data , Unicode data type that can store Unicode characters                                                                                |                                                                                |
| nvarchar                | string - variable-size string data, Unicode data type that can store Unicode characters                                                                                |                                                                                |
| numeric                 | string
 range of values - 10^38 +1 to 10^38                                                                                | When Debezium configuration parameter `time.precision.mode = 'connect'` and `decimal.handling.mode = 'precision'` , the value `1.00E +33` will be converted by Debezium to the binary string `'SztMqFqGw1MAAAAAAAAAAA=='` and will be saved in Redis target database as the string `'1000000000000000000000000000000000'` |
| rowversion              | string
data type that exposes automatically generated, unique binary numbers within a database. rowversion is generally used as a mechanism for version-stamping table rows.    | 0x00000000000007D0                                                                                |
| smalldatetime           | string
represents the number of milliseconds past the epoch, and does not include timezone information.                                                                         | `'2018-06-20 15:13:16`' will be converted by Debezium to `1529507580000` ms past the epoch and will be saved in Write-behind as the string '`1529507580000`'.
number of seconds: 16 will be not be included in the convertion and will not be saved in Redis target database                              |
| smallmoney              | string
range of values: - 214,748.3648 to 214,748.3647                                                                                | When Debezium configuration parameter `decimal.handling.mode = 'string'` the value `-214748.3648` will be converted by Debezium to the string `'-214748.3648'` and will be saved in Redis target database as '`-214748.3648`'                                                                                |
| Spatial Geometry Types  | not supported                                                                                |                                                                                |
| Spatial Geography Types | not supported                                                                                |                                                                                |
| table                   | not supported                                                                                |                                                                                |
| text                    | Variable-length Unicode data                                                                                |                                                                                |
| uniqueidentifier        | string                                                                                | 06BEEF00-F859-406B-9A60-0A56AB81A97                                                                                |

| Source data type        | Target data type for JSON                                                                                | Example for JSON                                                                                |
| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| bit                     | boolean                                                                                | When value `>0` it will be converted by Debezium to true and will be saved in Redis target database as `True` while when value `= 0` it will be converted by Debezium to false and will be saved in Redis target database as `False`                                                                                |
| datetime2               | number
 Represents the number of milliseconds since the epoch, and does not include timezone                                                                                | When Debezium configuration parameter `time.precision.mode='connect'`, the value '`2018-06-20 15:13:16.945104`' will be converted by Debezium to the value '`1529507596945104`' and will be saved in Redis target database database as the number `1529507596945.104`                                                  |
| datetimeoffset          | number                                                                                | When Debezium configuration parameter `decimal.handling.mode = 'precision'`, the datetimeoffset datatype value '`12-10-25 12:32:10 +01:00`' will be converted to the string '`2025-12-10T12:32:10+01:00'` and will be saved in Redis target database as the string `1765366330000`                                     |
| decimal,float,real      | number/string                                                                                | When Debezium configuration parameter `decimal.handling.mode = 'precision'` the value '`-3.402E+38`' will be converted by Debezium to the binary string '`/wAP3QCzc/wpiIGe8AAAAAA=`' and will be saved in Redis target database as the number `-340200000000000000000000000000000000000`                               |
| image                   | string                                                                                |                                                                                |
| money                   | number/string depending on the value of decimal.handling.mode                                                                                | When Debezium configuration parameter `decimal.handling.mode = 'precision'` the value `922337203685477.5807` will be converted by Debezium to the binary '`f/////////8=`' string and will be saved in Redis target database as the number `922337203685477.5807`                                                       |
| nchar                   | string                                                                                |                                                                                |
| nvarchar                | string - variable-size string data, Unicode data type that can store Unicode characters                                                                                |                                                                                |
| numeric                 | number                                                                                | When Debezium configuration parameter `time.precision.mode = 'connect'` and `decimal.handling.mode = 'precision'` , the value `1.00E +33` will be converted by Debezium to the binary string `'SztMqFqGw1MAAAAAAAAAAA=='` and will be saved in Redis target database as the number`1000000000000000000000000000000000` |
| rowversion              | string
data type that exposes automatically generated, unique binary numbers within a database. rowversion is generally used as a mechanism for version-stamping table rows. | 0x00000000000007D0                                                                                |
| smalldatetime           | number
represents the number of milliseconds past the epoch, and does not include timezone information.                                                                      | `'2018-06-20 15:13:16`' will be converted by Debezium to `1529507580000` ms past the epoch and will be saved in Write-behind as the number `1529507580000`.
number of seconds: 16 will be not be included in the convertion and will not be saved in Redis target database                             |
| smallmoney              | number                                                                                | When Debezium configuration parameter `decimal.handling.mode = 'string'` the value `-214748.3648` will be converted by Debezium to the string `'-214748.3648'` and will be saved in Redis target database as '`-214748.3648`'                                                                                |
| Spatial Geometry Types  | not supported                                                                                |                                                                                |
| Spatial Geography Types | not supported                                                                                |                                                                                |
| table                   | not supported                                                                                |                                                                                |
| text                    |                                                                                |                                                                                |
| uniqueidentifier        | string                                                                                | 06BEEF00-F859-406B-9A60-0A56AB81A97                                                                                |

\* fields with "not supported" data type will not appear in target hash.


Write-behind configuration for cassandra

Summary: Describes the `application.properties` settings that configure Debezium Server for cassandra

application.properties

[code example]

debezium.sink.type=redis
debezium.sink.redis.message.format=extended
debezium.sink.redis.address=<RDI_HOST>:<RDI_PORT>
# Comment the following line if not using a password for Write-behind.
debezium.sink.redis.password=<RDI_PASSWORD>
debezium.sink.redis.memory.limit.mb=80
# Redis SSL/TLS
#debezium.sink.redis.ssl.enabled=true
# When Redis is configured with a replica shard, these properties allow to verify that the data has been written to the replica.
#debezium.sink.redis.wait.enabled=true
#debezium.sink.redis.wait.timeout.ms=1000
#debezium.sink.redis.wait.retry.enabled=true
#debezium.sink.redis.wait.retry.delay.ms=1000
#debezium.source.database.history.redis.ssl.enabled=true
# Location of the Java keystore file containing an application process' own certificate and private key.
#javax.net.ssl.keyStore=<KEY_STORE_FILE_PATH>
# Password to access the private key from the keystore file specified by javax.net.ssl.keyStore. This password is used twice: To unlock the keystore file (store password), and To decrypt the private key stored in the keystore (key password).
#javax.net.ssl.keyStorePassword=<KEY_STORE_PASSWORD>
# Location of the Java keystore file containing the collection of CA certificates trusted by this application process (trust store).
#javax.net.ssl.trustStore=<TRUSE_STORE_FILE_PATH>
# Password to unlock the keystore file (store password) specified by javax.net.ssl.trustStore.
#javax.net.ssl.trustStorePassword=<TRUST_STORE_PASSWORD>

debezium.source.connector.class=io.debezium.connector.cassandra.Cassandra4Connector
debezium.source.snapshot.consistency=ONE
debezium.source.offset.storage=io.debezium.storage.redis.offset.RedisOffsetBackingStore
debezium.source.topic.prefix=<SOURCE_LOGICAL_SERVER_NAME>
debezium.source.cassandra.node.id=<NAME_SOURCE_NODE>
debezium.source.cassandra.hosts=<SOURCE_DB_HOST>
debezium.source.cassandra.port=<SOURCE_DB_PORT>
debezium.source.cassandra.config=<ABSOLUTE_PATH_CONFIG_FILE>
debezium.source.commit.log.relocation.dir=<RELOCATION_DIR>
debezium.source.commit.log.real.time.processing.enabled=true
debezium.source.commit.marked.complete.poll.interval.ms=1000
debezium.source.http.port=8040

# Whether to include the detailed schema information generated by Debezium in each record written to RDI.
# Note: Including the schema reduces the initial sync throughput and is not recommended for large data sets.
debezium.source.key.converter.schemas.enable=false
debezium.source.value.converter.schemas.enable=false
# When detailed schema information is excluded, handle decimal numeric types as strings.
debezium.source.decimal.handling.mode=string

debezium.transforms=AddPrefix
debezium.transforms.AddPrefix.type=org.apache.kafka.connect.transforms.RegexRouter
debezium.transforms.AddPrefix.regex=.*
debezium.transforms.AddPrefix.replacement=data:$0

# Logging
# Uncomment the following lines if running Debezium Server as a Java standalone process (non-containerized).
#quarkus.log.file.enable=true
#quarkus.log.file.path=<LOG_FILE_PATH>
#quarkus.log.file.rotation.max-file-size=100M
#quarkus.log.file.rotation.rotate-on-boot=true
#quarkus.log.file.rotation.file-suffix=.yyyy-MM-dd.gz
#quarkus.log.file.rotation.max-backup-index=3

# The default minimum log level for every log category, change only quarkus.log.level when needed.
quarkus.log.min-level=TRACE
# The default log level for every log category.
quarkus.log.level=INFO
# Determine whether to enable the JSON console formatting extension, which disables "normal" console formatting.
quarkus.log.console.json=false
# The port on which Debezium exposes Microprofile Health endpoint and other exposed status information.
quarkus.http.port=8088


Debezium Server configuration file

Summary: Application properties settings used to configure Debezim Server for source database servers

Summary: Describes the `application.properties` settings that configure Debezium Server for mongodb

Write-behind configuration for mongodb

Summary: Describes the `application.properties` settings that configure Debezium Server for mysql

Write-behind configuration for mysql

Summary: Describes the `application.properties` settings that configure Debezium Server for oracle

Write-behind configuration for oracle

Summary: Describes the `application.properties` settings that configure Debezium Server for postgresql

Write-behind configuration for postgresql

Summary: Describes the `application.properties` settings that configure Debezium Server for sqlserver

Write-behind configuration for sqlserver

Content

The `application.properties` file configures Debezium Server configuration to support source databases. It contains sections that define the sink connector (Redis) configuration and the source connector configuration.
This file needs to be saved in the host running Debezium Server.

The following topics describe `application.properties` for specific database servers:


Write-behind configuration for mongodb

application.properties

[code example]

debezium.sink.type=redis
debezium.sink.redis.message.format=extended
debezium.sink.redis.address=<RDI_HOST>:<RDI_PORT>
# Comment the following line if not using a password for Write-behind.
debezium.sink.redis.password=<RDI_PASSWORD>
debezium.sink.redis.memory.limit.mb=80
# Redis SSL/TLS
#debezium.sink.redis.ssl.enabled=true
# When Redis is configured with a replica shard, these properties allow to verify that the data has been written to the replica.
#debezium.sink.redis.wait.enabled=true
#debezium.sink.redis.wait.timeout.ms=1000
#debezium.sink.redis.wait.retry.enabled=true
#debezium.sink.redis.wait.retry.delay.ms=1000
#debezium.source.database.history.redis.ssl.enabled=true
# Location of the Java keystore file containing an application process' own certificate and private key.
#javax.net.ssl.keyStore=<KEY_STORE_FILE_PATH>
# Password to access the private key from the keystore file specified by javax.net.ssl.keyStore. This password is used twice: To unlock the keystore file (store password), and To decrypt the private key stored in the keystore (key password).
#javax.net.ssl.keyStorePassword=<KEY_STORE_PASSWORD>
# Location of the Java keystore file containing the collection of CA certificates trusted by this application process (trust store).
#javax.net.ssl.trustStore=<TRUSE_STORE_FILE_PATH>
# Password to unlock the keystore file (store password) specified by javax.net.ssl.trustStore.
#javax.net.ssl.trustStorePassword=<TRUST_STORE_PASSWORD>

debezium.source.connector.class=io.debezium.connector.mongodb.MongoDbConnector
debezium.source.mongodb.hosts=<REPLICATION_SET_NAME>/<MONGO_DB_HOST>:<MONGO_DB_PORT>
debezium.source.mongodb.connection.mode=replica_set
debezium.source.mongodb.user=<MONGO_DB_USER>
debezium.source.mongodb.password=<MONGO_DB_PASSWORD>
debezium.source.offset.storage=io.debezium.storage.redis.offset.RedisOffsetBackingStore
debezium.source.topic.prefix=<SOURCE_LOGICAL_SERVER_NAME>

debezium.source.offset.flush.interval.ms=1000
debezium.source.tombstones.on.delete=false
debezium.source.schema.history.internal=io.debezium.storage.redis.history.RedisSchemaHistory

# Important: Do NOT use `include` and `exclude` database lists at the same time, use either `include` or `exclude`.
# An optional, comma-separated list of regular expressions that match database names to be monitored.
# By default, all databases are monitored.
#debezium.source.database.include.list=<DB_NAME1>,<DB_NAME2>...
# An optional, comma-separated list of regular expressions that match database names for which you do not want to capture changes.
#debezium.source.database.exclude.list=<DB_NAME1>,<DB_NAME2>...
# Important: Do NOT use `include` and `exclude` collection lists at the same time, use either `include` or `exclude`.
# An optional, comma-separated list of regular expressions that match collection names to be monitored.
#debezium.source.collection.include.list=<DB_NAME.COLLECTION_NAME1>,<DB_NAME.COLLECTION_NAME2>...
# An optional, comma-separated list of regular expressions that match collection names for which you do not want to capture changes.
#debezium.source.collection.exclude.list=<DB_NAME.COLLECTION_NAME1>,<DB_NAME.COLLECTION_NAME2>...

#An optional, comma-separated list of regular expressions that match field names for which you do not want to capture changes.
#debezium.source.field_exclude_list=<DB_NAME.COLLECTION_NAME1.FIELD_NAME1>,<DB_NAME.COLLECTION_NAME.FIELD_NAME2>

# Whether to include the detailed schema information generated by Debezium in each record written to RDI.
# Note: Including the schema reduces the initial sync throughput and is not recommended for large data sets.
debezium.source.key.converter.schemas.enable=false
debezium.source.value.converter.schemas.enable=false
# When detailed schema information is excluded, handle decimal numeric types as strings.
debezium.source.decimal.handling.mode=string

debezium.transforms=AddPrefix
debezium.transforms.AddPrefix.type=org.apache.kafka.connect.transforms.RegexRouter
debezium.transforms.AddPrefix.regex=.*
debezium.transforms.AddPrefix.replacement=data:$0

# Logging
# Uncomment the following lines if running Debezium Server as a Java standalone process (non-containerized).
#quarkus.log.file.enable=true
#quarkus.log.file.path=<LOG_FILE_PATH>
#quarkus.log.file.rotation.max-file-size=100M
#quarkus.log.file.rotation.rotate-on-boot=true
#quarkus.log.file.rotation.file-suffix=.yyyy-MM-dd.gz
#quarkus.log.file.rotation.max-backup-index=3

# The default minimum log level for every log category, change only quarkus.log.level when needed.
quarkus.log.min-level=TRACE
# The default log level for every log category.
quarkus.log.level=INFO
# Determine whether to enable the JSON console formatting extension, which disables "normal" console formatting.
quarkus.log.console.json=false
# The port on which Debezium exposes Microprofile Health endpoint and other exposed status information.
quarkus.http.port=8088


Write-behind configuration for mysql

application.properties

[code example]

debezium.sink.type=redis
debezium.sink.redis.message.format=extended
debezium.sink.redis.address=<RDI_HOST>:<RDI_PORT>
# Comment the following line if not using a password for Write-behind.
debezium.sink.redis.password=<RDI_PASSWORD>
debezium.sink.redis.memory.limit.mb=80
# Redis SSL/TLS
#debezium.sink.redis.ssl.enabled=true
# When Redis is configured with a replica shard, these properties allow to verify that the data has been written to the replica.
#debezium.sink.redis.wait.enabled=true
#debezium.sink.redis.wait.timeout.ms=1000
#debezium.sink.redis.wait.retry.enabled=true
#debezium.sink.redis.wait.retry.delay.ms=1000
#debezium.source.database.history.redis.ssl.enabled=true
# Location of the Java keystore file containing an application process' own certificate and private key.
#javax.net.ssl.keyStore=<KEY_STORE_FILE_PATH>
# Password to access the private key from the keystore file specified by javax.net.ssl.keyStore. This password is used twice: To unlock the keystore file (store password), and To decrypt the private key stored in the keystore (key password).
#javax.net.ssl.keyStorePassword=<KEY_STORE_PASSWORD>
# Location of the Java keystore file containing the collection of CA certificates trusted by this application process (trust store).
#javax.net.ssl.trustStore=<TRUSE_STORE_FILE_PATH>
# Password to unlock the keystore file (store password) specified by javax.net.ssl.trustStore.
#javax.net.ssl.trustStorePassword=<TRUST_STORE_PASSWORD>

debezium.source.connector.class=io.debezium.connector.mysql.MySqlConnector
# A numeric ID of this database client, which must be unique across all currently-running database processes in the MySQL cluster.
debezium.source.database.server.id=1
debezium.source.offset.storage=io.debezium.storage.redis.offset.RedisOffsetBackingStore
debezium.source.topic.prefix=<SOURCE_LOGICAL_SERVER_NAME>

debezium.source.database.hostname=<SOURCE_DB_HOST>
debezium.source.database.port=<SOURCE_DB_PORT>
debezium.source.database.user=<SOURCE_DB_USER>
debezium.source.database.password=<SOURCE_DB_PASSWORD>
debezium.source.include.schema.changes=false
# Determines whether the connector should omit publishing change events when there are no modifications in the included columns.
debezium.source.skip.messages.without.change=true
debezium.source.offset.flush.interval.ms=1000
debezium.source.tombstones.on.delete=false
debezium.source.schema.history.internal=io.debezium.storage.redis.history.RedisSchemaHistory

# Important: Do NOT use `include` and `exclude` database lists at the same time, use either `include` or `exclude`.
# An optional, comma-separated list of regular expressions that match database names to be monitored.
# By default, all databases are monitored.
#debezium.source.database.include.list=<DB_NAME1>,<DB_NAME2>...
# An optional, comma-separated list of regular expressions that match database names for which you do not want to capture changes.
#debezium.source.database.exclude.list=<DB_NAME1>,<DB_NAME2>...
# Important: Do NOT use `include` and `exclude` table lists at the same time, use either `include` or `exclude`.
# An optional, comma-separated list of regular expressions that match fully-qualified table identifiers of tables whose changes you want to capture.
#debezium.source.table.include.list=<DB_NAME.TABLE_NAME1>,<DB_NAME.TABLE_NAME2>...
# An optional, comma-separated list of regular expressions that match fully-qualified table identifiers for tables whose changes you do not want to capture.
#debezium.source.table.exclude.list=<DB_NAME.TABLE_NAME1>,<DB_NAME.TABLE_NAME2>...

# Important: Do NOT use include and exclude column lists at the same time, use either include or exclude.
# An optional, comma-separated list of regular expressions that match the fully-qualified names of columns to include in change event record values.
#debezium.source.column.include.list=<DB_NAME.TABLE_NAME.COLUMN_NAME1>,<DB_NAME.TABLE_NAME.COLUMN_NAME2>...
# An optional, comma-separated list of regular expressions that match the fully-qualified names of columns to exclude from change event record values.
#debezium.source.column.exclude.list=<DB_NAME.TABLE_NAME.COLUMN_NAME1>,<DB_NAME.TABLE_NAME.COLUMN_NAME2>...

# Records only DDL statements that are relevant to tables whose changes are being captured by Debezium.
# In case of changing the captured tables, run `redis-di reset`.
debezium.source.schema.history.internal.store.only.captured.tables.ddl=true

# Whether to include the detailed schema information generated by Debezium in each record written to RDI.
# Note: Including the schema reduces the initial sync throughput and is not recommended for large data sets.
debezium.source.key.converter.schemas.enable=false
debezium.source.value.converter.schemas.enable=false
# When detailed schema information is excluded, handle decimal numeric types as strings.
debezium.source.decimal.handling.mode=string

debezium.transforms=AddPrefix
debezium.transforms.AddPrefix.type=org.apache.kafka.connect.transforms.RegexRouter
debezium.transforms.AddPrefix.regex=.*
debezium.transforms.AddPrefix.replacement=data:$0

# Logging
# Uncomment the following lines if running Debezium Server as a Java standalone process (non-containerized).
#quarkus.log.file.enable=true
#quarkus.log.file.path=<LOG_FILE_PATH>
#quarkus.log.file.rotation.max-file-size=100M
#quarkus.log.file.rotation.rotate-on-boot=true
#quarkus.log.file.rotation.file-suffix=.yyyy-MM-dd.gz
#quarkus.log.file.rotation.max-backup-index=3

# The default minimum log level for every log category, change only quarkus.log.level when needed.
quarkus.log.min-level=TRACE
# The default log level for every log category.
quarkus.log.level=INFO
# Determine whether to enable the JSON console formatting extension, which disables "normal" console formatting.
quarkus.log.console.json=false
# The port on which Debezium exposes Microprofile Health endpoint and other exposed status information.
quarkus.http.port=8088