{
  "schema_version": 2,
  "id": "integrate/redis-data-integration/faq",
  "title": "FAQ",
  "url": "https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/faq/",
  "summary": "Find answers to common questions about RDI",
  "content": "\n## Which license does RDI use?\n\nYou must purchase a commercial license for RDI with Redis Enterprise. This includes two extra\nRedis Enterprise shards (primary and replica) for the staging database.\n\n## How does RDI track data changes in the source database?\n\nRDI uses change data capture (CDC) mechanisms that are specific to each of the\nsupported source databases:\n\n- **Oracle**: RDI uses `LogMiner` to read Oracle's `redo logs` and `archive logs`,\n  or, alternatively, `XStream`.\n- **MySQL/MariaDB**: RDI uses `binary log` (binlog) replication to capture all commits.\n- **PostgreSQL**: RDI uses the `pgoutput` logical decoding plugin. The same\n  applies to the PostgreSQL-compatible databases that RDI supports, including\n  Supabase, AlloyDB for PostgreSQL, Amazon Aurora/RDS for PostgreSQL, and Neon.\n- **SQL Server**: RDI uses the database's built-in CDC feature.\n- **MongoDB**: RDI uses `change streams` to read the `oplog`. The source must be\n  a replica set, sharded cluster, or MongoDB Atlas deployment, because a\n  standalone MongoDB server has no oplog.\n- **Google Cloud Spanner**: RDI uses `Spanner change streams` for the streaming\n  phase and the JDBC driver for the initial snapshot. Spanner is supported only\n  when RDI is deployed on Kubernetes with Helm.\n- **Snowflake** (preview): RDI uses `Snowflake Streams`. Snowflake is supported\n  only when RDI is deployed on Kubernetes with Helm.\n\nFor the complete list of supported source databases and versions, see\n[Prepare source databases](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/data-pipelines/prepare-dbs).\n\n## How much data can RDI process?\n\nRDI uses the concept of *processing units*. Each processing unit uses 1 CPU core and can process\nabout 10,000 records per second, assuming the records have a size of about 1KB each. This throughput\nmight change slightly depending on the number of columns, the number of data transformations,\nand the speed of the network. Typically, one processing unit is enough for RDI to deal with the\ntraffic from a relational database.\n\n## Can RDI work with any Redis database?\n\nNo. RDI is designed and tested to work only with Redis Enterprise. The staging database can\nonly use version 6.4 or above. The target Redis database can be of any version and can be a\nreplica of an Active-Active replication setup or an Auto tiering database.\n\n## Can I use Active-Active for the RDI database?\n\nYes, starting with RDI 1.16.0, you can use Active-Active for the RDI database. This is\nsupported whether or not you also run a disaster recovery (DR) setup for RDI.\n\nIf you have two RDI instances sharing a single RDI database then they will use that database for leader election, so\nthey need no other lease mechanism. This is how high availability (HA) works for VM\ninstallations. See\n[Installing with High Availability](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/installation/install-vm#installing-with-high-availability).\n\nIn a DR setup, each site runs its own RDI instance against its local instance of the\nActive-Active RDI database, so leader election needs an external lease. Google Cloud Storage\n(GCS) is currently the only supported lease mechanism, and you can configure it only for\n[Helm based installations](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/installation/install-k8s).\n\n**Important:** Use a DR setup only when both sites capture changes from the same source\ndatabase server. Both RDI instances must point at that same server, not at a replica of it.\n\n## Can I run multiple RDI installations in the same Kubernetes cluster?\n\nNo. Only one RDI installation is supported per Kubernetes cluster, even if\nyou install into different namespaces. If you need more than one RDI\ndeployment, use separate Kubernetes clusters. See\n[Install on Kubernetes](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/installation/install-k8s)\nfor installation details.\n\n## Can RDI automatically track changes to the source database schema?\n\nIf you don't configure RDI to capture a specific set of tables in the schema then it will\ndetect any new tables when they are added. Similarly, RDI will capture new table columns\nand changes to column names unless you configure it for a specific set of columns.\nBear in mind that the Redis keys in the target database will change to reflect the\nnew or renamed tables and columns.\n\n## Should I be concerned when the log says RDI is out of memory? {#rdi-oom}\n\nSometimes the Debezium log will contain a message saying that RDI is out of\nmemory. This is not an error but an informative message to say that RDI\nis applying *backpressure* to Debezium. See\n[Backpressure mechanism](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/architecture#backpressure-mechanism)\nin the Architecture guide for more information.\n\n## What happens when RDI can't write to the target Redis database?\n\nRDI will keep attempting to write the changes to the target and will also attempt\nto reconnect to it, if necessary. While the target is disconnected, RDI\nwill keep capturing change events from the source database and adding them to its\nstreams in the staging database. This continues until the staging database gets\nlow on space to store new events. When RDI detects this, it applies a \"back pressure\"\nmechanism to capture data from the source less frequently, which reduces the risk of running\nout of space altogether. The systems that the source databases use to record changes can\nretain the change data for at least a few hours, and RDI can catch up with the\nchanges as soon as the target connection recovers or the staging database has\nmore space available.\n\n## What does RDI do if the data is corrupted or invalid?\n\nThe collector reports the data to RDI in a structured JSON format. If\nthe structure of the JSON data is invalid or if there is a fatal bug in the transformation\njob then RDI can't transform the data. When this happens, RDI will store the original data\nin a \"dead letter queue\" along with a message to say why it was rejected. The dead letter\nqueue is stored as a capped stream in the RDI staging database. You can see its contents\nwith Redis Insight or with the\n[`redis-di list-dlq-records`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-list-dlq-records)\ncommand from the CLI.\n\nSee [Rejected records](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/data-pipelines/rejected-records) for more information about DLQ.\n\n## Can I use RDI without persistence enabled?\n\nBy default, RDI requires persistence to be enabled on the RDI database. This ensures that RDI can recover both its configuration and the last known state if the cluster crashes.\n\nIf you don't have permissions to use persistence due to compliance or other reasons, you can disable\nthe persistence check on the RDI database (Helm installation only). If you do this, RDI will not be\nable to recover from a crash, and you will have to perform a new deploy to reinitialize the pipeline.\n\nTo disable the persistence check, set the `aofRequired` value to `false` in the `operator.prerequisiteChecks`\nsection of the `values.yaml` file.\n\n```yaml\noperator:\n  prerequisiteChecks:\n    aofRequired: false\n```\n\nThis option is available in RDI 1.16.2 and later.\n\n## Which processor should I use? {#which-processor-should-i-use}\n\nRDI ships with two stream processor implementations: the *classic*\nprocessor and the *Flink* processor. Both are fully supported for\nproduction on VM and Kubernetes installations. The Flink processor\nis generally available as of RDI 1.19.0 and is enabled per pipeline.\n\nThe Flink processor delivers significantly higher snapshot throughput,\nlower end-to-end latency, horizontal scaling, and Flink checkpointing\non top of the same at-least-once delivery guarantees as the classic\nprocessor. It also adds optional expression and `redis.lookup` result\ncaching.\n\n**We strongly recommend using the Flink processor** for new pipelines and\nmigrating existing pipelines to it, to benefit from these improvements. The\n*classic* processor is still the default, so pipelines keep using it until\nyou opt in, and it remains a fully supported choice — for example, when you\nwant to ensure your pipelines continue to work as before until you have\nconsciously migrated them. In a future release, however, the Flink processor\nwill become the default and the classic processor may be deprecated, so adopting\nthe Flink processor now avoids a later migration.\n\nSwitch a pipeline to the Flink processor by setting\n[`processors.type`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config#processors)\nto `flink` (`classic` is the default). You can adopt it per pipeline without\nchanging the others.\n\nSee\n[Differences between the classic and Flink processors](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/architecture/classic-vs-flink)\nfor a side-by-side comparison and\n[Migrate from the classic processor to the Flink processor](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/installation/migration-classic-to-flink)\nfor a step-by-step migration guide.\n",
  "tags": ["docs","integrate","rs","rdi"],
  "last_updated": "2026-09-19T17:55:58-07:00"
}