{ "schema_version": 2, "id": "integrate/redis-data-integration/installation/disaster-recovery", "title": "Disaster recovery (DR)", "url": "https://redis.io/docs/latest/integrate/redis-data-integration/installation/disaster-recovery/", "summary": "Configure disaster recovery for Redis Data Integration with GCS or DynamoDB leader election.", "tags": [ "docs", "integrate", "rs", "rdi" ], "last_updated": "2026-11-13T11:28:40+02:00", "children": [ { "id": "integrate/redis-data-integration/installation/disaster-recovery/gcs", "summary": "Create and configure a Google Cloud Storage lease for RDI disaster recovery.", "title": "Configure a GCS lease", "url": "https://redis.io/docs/latest/integrate/redis-data-integration/installation/disaster-recovery/gcs/" }, { "id": "integrate/redis-data-integration/installation/disaster-recovery/dynamodb", "summary": "Create and configure a strongly consistent DynamoDB lease for RDI disaster recovery.", "title": "Configure a DynamoDB lease", "url": "https://redis.io/docs/latest/integrate/redis-data-integration/installation/disaster-recovery/dynamodb/" } ], "page_type": "content", "content_hash": "35b43cf32cb8a3ec6d52523751d86c1ef3cce45a492874cc3a294810cce5f6c4", "sections": [ { "id": "overview", "title": "Overview", "role": "overview", "text": "Redis Data Integration (RDI) can run across two or more sites so that another site takes over\nwhen the active site fails. Each site runs its own RDI installation, an *instance*, in a\nseparate Kubernetes cluster. The instances use an external lease to elect one of them, the\n*leader*, to run the pipelines. The other instances are *followers* and wait to take over.\nThe examples in this guide use two sites, A and B.\n\nRDI keeps its state, such as pipeline configurations and saved source positions, in the RDI\ndatabase. In a disaster recovery (DR) setup, the RDI database is an Active-Active database with\na database instance at each site, so this state replicates between the sites. Each RDI instance\nconnects to the database instance at its own site.\n\nFor [Kubernetes installations](https://redis.io/docs/latest/integrate/redis-data-integration/installation/install-k8s),\nRDI supports leases in Google Cloud Storage (GCS), since RDI 1.15.0, and in Amazon DynamoDB,\nsince RDI 2.2.0. DR requires a Kubernetes installation and one of these lease stores.\n\nDR differs from the\n[high availability (HA) pair](https://redis.io/docs/latest/integrate/redis-data-integration/installation/install-vm#installing-with-high-availability)\nfor the VM installations, where two instances share one RDI database and use it for leader\nelection. Use an HA pair to survive the failure of one RDI instance, and DR to survive the\nloss of a site." }, { "id": "understand-the-topology", "title": "Understand the topology", "role": "content", "text": "![Site A has the source primary and the RDI leader, which captures from it. Source changes replicate to a source replica at site B, where RDI pipelines are in standby. After a takeover, site B's RDI captures from the source at site B. Both sites share an external lease. RDI database endpoints and example target endpoints use Active-Active replication.](https://redis.io/docs/latest/images/rdi/disaster-recovery.svg)\n\nEach RDI instance is in one of three states:\n\n| State | Takes part in leader election | Runs the pipelines | Typical use |\n| :-- | :-- | :-- | :-- |\n| Leader | Yes, and holds the lease | Yes | The active site. |\n| Follower | Yes | No. Takes over when the lease is released or expires. | Automatic failover. |\n| Standby | No. Its `standby_mode` setting is `true`. | No | Manual failover, or a site under maintenance. |\n\n`redis-di info` reports `Leader Election Mode: follower` for both followers and standby\ninstances. Check `standby_mode` with `redis-di get-setting standby_mode` to tell them apart.\nOn an instance that isn't the leader, RDI reports the pipelines as `standby` and their\ncomponents as `stopped`. An instance that has never been the leader has no pipelines.\n\nEach site's RDI instance captures from the source database named in its pipeline\nconfiguration. Usually that's the closest database that supports snapshots and change data\ncapture (CDC): the active source database or a replica of it. It can also be a shared endpoint,\nsuch as a Domain Name System (DNS) name that resolves to the active source database. Which kinds of\nreplica RDI can capture from, and when, depends on the source database. See\n[Source databases and data after a failover](#source-databases-and-data-after-a-failover).\nManage source replication, promotion, and endpoint routing separately from RDI failover.\n\nThe target in this example also uses Active-Active replication. Configure target availability\nseparately from RDI leader election. If the target at site B is a\n[Replica Of](https://redis.io/docs/latest/operate/rs/databases/import-export/replica-of) destination, use\nmanual failover and stop replication into it before you enable site B. You must not write to a\nReplica Of destination.\n\nThe RDI instances use the lease to elect one leader and let another site take over when a\nwhole site or region fails. See [How automatic failover works](#how-automatic-failover-works)." }, { "id": "rdi-database-replication", "title": "RDI database replication", "role": "content", "text": "A disaster recovery setup requires Active-Active replication for the RDI database." }, { "id": "source-databases-and-data-after-a-failover", "title": "Source databases and data after a failover", "role": "content", "text": "The new leader can resume capture only if its source database still has the changes from the\nposition that RDI saved. The new leader resumes from the state in its local RDI database\ninstance: the collector's saved source position and any change records not yet processed.\n\nRDI saves the position in the source database's own format, such as an Oracle system change number\n(SCN) or a PostgreSQL log sequence number (LSN). The source database at the new leader's site\nmust have the same history as the one RDI captured from. Otherwise, capture can fail, or start\nfrom the wrong place.\n\nThe following table gives the general requirement for each source. Replication and failover\ndetails vary by source database, version, and managed service.\n\n| Source | Saved position | New leader requirements |\n| :-- | :-- | :-- |\n| Oracle | SCN | Sources share the same SCN history, for example a physical standby. |\n| MySQL and MariaDB | Global transaction identifier (GTID) set | GTIDs are enabled, and the source contains every transaction in the saved GTID set, with binary logs (binlogs) kept from that point. A replica writes replicated changes to its own binlog. |\n| PostgreSQL | LSN of the replication slot | The source is a promoted physical standby with a synchronized copy of RDI's replication slot. This requires PostgreSQL 17 with slot synchronization enabled on the standby, and `slot.failover: true` in the source's `advanced.source` before RDI creates the slot. Debezium doesn't change an existing slot, so reset an existing pipeline after you set the option. RDI can't use the slot copy until the standby is promoted. RDI hasn't been tested with the `pg_failover_slots` extension for PostgreSQL 14 to 16. A logical replica needs a pipeline reset. |\n| SQL Server | LSN | The source is a replica of the same database in an Always On availability group, so it has the same transaction log. The CDC capture job runs on the new primary after promotion. |\n| MongoDB | Change stream resume token | The source belongs to the same replica set or sharded cluster, and its oplog still contains the saved position. A separate cluster that another tool copies needs a pipeline reset. |\n| Google Cloud Spanner | Change stream position | Both sites capture from the same Spanner database, and the saved position is within the change stream's retention period. |\n\nBefore you rely on DR, check that RDI can resume capture from its saved position at each\nsite's source database. A replicated copy of the source alone does not guarantee this. Retain\nenough source change history to cover capture lag, snapshots, and the longest outage you plan\nto recover from.\n\nAfter a takeover, one of the following applies:\n\n- **Capture continues from the saved position.** If the source database still has the change\n history from that position, RDI captures every later change. Active-Active replication is\n asynchronous, so the saved state can lag behind the failed site's state. RDI can then apply\n some changes to the target again. If the source replicates asynchronously, a promoted source\n can lack changes that RDI already delivered. These changes remain in the target.\n- **Capture can't continue.** For example, the source database has no compatible change history\n or capture state for the saved position.\n [Reset the pipeline](https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/deploy#reset-a-pipeline)\n to take a new snapshot. The snapshot reloads inserted and updated rows. Rows deleted from the\n source between the failed site's last processed change and the start of the snapshot remain\n in the target. Delete them from the target yourself.\n\nAn Active-Active target replicates separately from the RDI database, so it can lag further\nbehind. RDI can then skip changes that the failed site delivered but that haven't reached the\nnew leader's target instance. They're missing there until the failed site's target instance\nsyncs. If that instance loses them, or you remove it from the Active-Active database, they're\nlost from the target." }, { "id": "prepare-both-sites", "title": "Prepare both sites", "role": "content", "text": "1. Create the RDI database as an\n [Active-Active database](https://redis.io/docs/latest/operate/rs/databases/active-active/create) with an\n instance at each site. It must also meet the\n [RDI database requirements](https://redis.io/docs/latest/integrate/redis-data-integration/installation/install-k8s#create-the-rdi-database).\n1. [Prepare the external lease](#configure-a-lease).\n1. [Prepare the source database](https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/prepare-dbs)\n at each site for CDC, including the source endpoints and any promotion or routing steps.\n Check the requirements in\n [Source databases and data after a failover](#source-databases-and-data-after-a-failover).\n1. Configure target database availability, as described in\n [Understand the topology](#understand-the-topology).\n1. Configure network access from each RDI instance to its site's source database, its target\n database, and the same external lease.\n1. Keep the clocks at both sites synchronized, for example with Network Time Protocol (NTP).\n\nThis guide describes how to set up RDI for disaster recovery. An existing RDI database can't\nbe changed into an Active-Active database. To add DR to an existing installation, export each\npipeline with\n[`redis-di export`](https://redis.io/docs/latest/integrate/redis-data-integration/reference/cli/redis-di-export),\nset up both sites as described here, and deploy the pipelines. RDI then takes a new snapshot." }, { "id": "configure-a-lease", "title": "Configure a lease", "role": "configuration", "text": "RDI needs a lease store that allows at most one leader and stays available when any one site or\nregion fails. The lease is one record that the instances at every site read and write. Both\nsupported stores meet these requirements:\n\n| Requirement | GCS lease | DynamoDB lease |\n| :-- | :-- | :-- |\n| All sites of a DR setup use one record. | The object that `gcsLease.objectName` names. | The item that `dynamoDBLease.keyName` names. |\n| Each write is conditional. It succeeds only if the lease is free, expired, or held by the writing instance, so two sites can't both acquire it. | Conditional object creation and metadata updates. | Conditional `PutItem` and `DeleteItem` requests. |\n| The store checks each condition against the latest write from any site, not a local copy. | Cloud Storage is strongly consistent. It writes object metadata, which holds the lease, synchronously across the bucket's regions. | Only a global table with multi-Region strong consistency (MRSC) does this. In the default eventual consistency mode, each Region checks its own copy. |\n| The lease survives the loss of any one site or region. | Use a dual-region or multi-region bucket. | Use an MRSC table, which spans three Amazon Web Services (AWS) Regions. A replica Region that can reach neither of the other two rejects writes, so its site stops leading. |\n\nThe lease holds the leader's `instanceID` and an expiry time. If no site can reach the store, no\ninstance can become the leader, and the pipelines stop.\n\nChoose one provider and follow its guide to create the lease resource and grant operator\naccess. Return to [Prepare both sites](#prepare-both-sites) before configuring or installing RDI:\n\n- [Create the GCS bucket](https://redis.io/docs/latest/integrate/redis-data-integration/installation/disaster-recovery/gcs#create-the-bucket)\n- [Create the DynamoDB table](https://redis.io/docs/latest/integrate/redis-data-integration/installation/disaster-recovery/dynamodb#create-the-table)\n\nWith more than two sites, sites can share a replica Region of the DynamoDB table.\n\n> [!WARNING]\n> Set `operator.leaderElection.leaseType` to `gcs` or `dynamodb`, as the provider guides show.\n> RDI uses the `redsync` lease for any other value, and `redsync` can't be used with an\n> Active-Active RDI database. After installing, check that the operator log at each site\n> contains `Creating GCS lease` or `Creating DynamoDB lease`.\n\n#### Protect the lease\n\n> [!WARNING]\n> Don't delete the lease while any instance takes part in leader election. A participating\n> instance recreates it on its next attempt, so two sites can briefly lead.\n>\n> For GCS, don't apply an Object Lifecycle Management rule that deletes the lease object,\n> such as a rule that deletes objects by age. RDI creates the object once and then only\n> updates it, so such a rule eventually deletes it.\n\n#### Share a lease store between DR setups\n\nA DR setup uses one lease for all its pipelines. Separate DR setups, such as a production\nsetup and a test setup, can share one GCS bucket or DynamoDB table. Give each setup its own\nlease with `gcsLease.objectName` or `dynamoDBLease.keyName`, and use the same value at every\nsite of that setup. For example, in the DynamoDB values files of the production setup:\n\n[code example]\n\nAnd in the values files of the test setup:\n\n[code example]\n\nSet `keyName` explicitly for every setup, because it defaults to `rdi-lease`. Setups that use\nthe same lease compete for it, so only one of them runs its pipelines. Each setup still needs\nits own Kubernetes clusters and Active-Active RDI database.\n\n#### Lease timing\n\nThe lease timings are Helm values under `operator.leaderElection`:\n\n| Helm value | Default | What it controls |\n| :-- | :-- | :-- |\n| `ttl` | `60s` | How long the lease stays valid after the leader last renewed it. |\n| `refreshInterval` | `10s` | How often each instance tries to acquire or renew the lease and reads its runtime settings. |\n| `timeout` | `30s` | Total time for one attempt to acquire, renew, or release the lease, including retries. |\n| `retryPolicy` | `attempts: 3`, `initialInterval: 1s`, `backoffFactor: 2.0` | Retries within `timeout`. |\n| `checkTimeout` | `15s` | Total time to read the runtime settings from the RDI database. |\n| `checkRetryPolicy` | `attempts: 3`, `initialInterval: 1s`, `backoffFactor: 2.0` | Retries within `checkTimeout`. |\n\n- A leader that enters standby releases the lease within `refreshInterval`. A leader that\n can't read its RDI database after the `checkRetryPolicy` attempts, or within `checkTimeout`,\n also releases it. Another instance acquires it on its next attempt.\n- A leader that fails, restarts, or can't reach the lease store doesn't release the lease.\n Another instance can acquire it after `ttl`. A leader that restarts and renews within `ttl`\n keeps the lease.\n- If you change the timings, keep the sum of `timeout`, `checkTimeout`, and `refreshInterval`\n less than `ttl`, so that a leader that can't renew the lease stops leading before another site\n can acquire it.\n\nAn instance that stops leading removes its pipeline components, which can take a minute or more.\n\nWith the default settings, expect another site to take over within about 2 minutes. Pipeline\nstartup and catch-up add to the recovery time." }, { "id": "install-rdi-at-both-sites", "title": "Install RDI at both sites", "role": "setup", "text": "Install the same RDI version in separate Kubernetes clusters after preparing both sites." }, { "id": "choose-a-failover-mode", "title": "Choose a failover mode", "role": "content", "text": "The failover mode sets whether site B waits as a follower or in standby. See the instance\nstates in [Understand the topology](#understand-the-topology). Both instances take part in\nleader election after installation. Set the mode with the `standby_mode`\n[instance setting](https://redis.io/docs/latest/integrate/redis-data-integration/reference/instance-settings#standby_mode),\nand change it at any time:\n\n| Mode | `standby_mode` | Who starts the takeover? | Use it when |\n| :-- | :-- | :-- | :-- |\n| Automatic | `false` at both sites, the default | The follower, after the leader releases the lease or the lease expires. | RDI can resume capture from the source database at site B as soon as site B takes over. For example, both sites capture from one Spanner database. |\n| Manual | `false` at site A, `true` at site B | You, by enabling participation on the destination instance. | The source database at site B needs a step first, such as promotion or a check of its change history. For example, a PostgreSQL standby must be promoted before RDI can use its slot copy. |\n\nWith manual failover, prepare the source first. Promote the source database at site B or\nconfirm that RDI can resume capture from the saved position, then enable site B. If capture\ncan't resume from the saved position, reset the pipeline after site B takes over. See\n[Source databases and data after a failover](#source-databases-and-data-after-a-failover)." }, { "id": "configure-each-sites-values-file", "title": "Configure each site's values file", "role": "configuration", "text": "Create `rdi-values-site-a.yaml` and `rdi-values-site-b.yaml` from the chart's values, following\n[Install the RDI Helm chart](https://redis.io/docs/latest/integrate/redis-data-integration/installation/install-k8s#install-the-rdi-helm-chart).\nSet the connection to the local RDI database and the instance identifier (ID) in each file.\nFor site A:\n\n[code example]\n\nFor site B:\n\n[code example]\n\n> [!NOTE]\n> The `standby_mode` setting stays in the RDI database when you uninstall RDI. After you\n> install an instance on an RDI database that an earlier installation used, run\n> `redis-di get-setting standby_mode`. If it isn't ``, run\n> `redis-di delete-setting standby_mode`.\n\nSet `operator.leaderElection.instanceID` before you install each instance, and keep it unchanged\nacross upgrades.\n\n> [!WARNING]\n> Use a different `instanceID` at each site. The ID defaults to `cluster-1`. Instances that\n> share an ID both act as the leader, and they share runtime\n> settings such as `standby_mode`.\n\nThese snippets contain only the common, site-specific settings. Add the provider's lease\nsettings in [Configure the lease and install RDI](#configure-the-lease-and-install-rdi) before\ninstalling. Complete the other installation settings, including Transport Layer Security (TLS)\nand API access, as described in\n[The `values.yaml` file](https://redis.io/docs/latest/integrate/redis-data-integration/installation/install-k8s#the-valuesyaml-file)." }, { "id": "configure-the-lease-and-install-rdi", "title": "Configure the lease and install RDI", "role": "configuration", "text": "Complete the lease settings in both values files, then install RDI and check the operators\nusing the guide for your provider:\n\n- [Configure both instances for GCS](https://redis.io/docs/latest/integrate/redis-data-integration/installation/disaster-recovery/gcs#configure-both-instances)\n- [Configure both instances for DynamoDB](https://redis.io/docs/latest/integrate/redis-data-integration/installation/disaster-recovery/dynamodb#configure-both-instances)" }, { "id": "deploy-pipelines-and-secrets", "title": "Deploy pipelines and secrets", "role": "content", "text": "Configure a separate RDI command-line interface (CLI)\n[context](https://redis.io/docs/latest/integrate/redis-data-integration/reference/cli#contexts) for each\ninstance. The examples use `site-a` and `site-b` as context names:\n\n[code example]\n\n1. Check leadership on both instances:\n\n [code example]\n\n One reports `Leader Election Mode: leader` and the other\n `Leader Election Mode: follower`. The examples assume that site A is the leader.\n\n If leadership isn't as expected, check the operator log at each site:\n\n [code example]\n\n1. For manual failover, put site B in standby:\n\n [code example]\n\n1. [Set the secrets for every pipeline](https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/multiple-pipelines#set-secrets-for-each-pipeline)\n on both instances. Secret values of the same pipeline may differ across instances. You can\n set secrets before a pipeline exists. Include the source and target credentials and any\n certificates.\n\n When the sites use different source or target endpoints, set the endpoints as\n [generic secrets](https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/deploy#set-secrets)\n at each site. For a source named `mysql`:\n\n [code example]\n\n Reference `${MYSQL_DB_HOST}` in the source's `connection.host` and `${TARGET_DB_HOST}` in\n the target's `connection.host` in `config.yaml`. RDI then resolves the same pipeline\n configuration to the local endpoints at each site.\n\n1. For a new pipeline, create it empty on the leader, then deploy its configuration. For a\n leader at site A:\n\n [code example]\n\n Don't run `deploy --empty` on an existing pipeline. It clears the pipeline's configuration\n and its saved source positions.\n\n RDI saves the pipeline configuration in the RDI database and restores it when another\n instance becomes the leader. Do not deploy a separate configuration on the follower.\n\n1. Run `redis-di --context site-a describe \"\"` for each pipeline on the leader.\n Check that its components run without errors and that source changes reach the target." }, { "id": "test-a-takeover", "title": "Test a takeover", "role": "content", "text": "Move leadership to site B and back, following\n[Move leadership manually](#move-leadership-manually). Verify pipeline health and data delivery\nat each site before you rely on the setup." }, { "id": "how-automatic-failover-works", "title": "How automatic failover works", "role": "content", "text": "The leader stops leading in these cases:\n\n| Event at the leader's site | What the leader does | When another site can take over |\n| :-- | :-- | :-- |\n| You set `standby_mode` to `true`. | Releases the lease. | On its next attempt. |\n| The operator can't read its RDI database instance after the `checkRetryPolicy` attempts, or within `checkTimeout`. | Releases the lease. | On its next attempt. |\n| The operator can't renew the lease, for example because it can't reach the lease store. | Stops leading before the lease expires. | After the lease expires. |\n| The operator stops, for example because its pod, node, or Kubernetes cluster fails. | Nothing. | After the lease expires. |\n\nAnother site takes over only if its operator is running, it can read its own RDI database\ninstance, it isn't in standby, and it can reach the lease store. If no site can reach the lease\nstore, no site can become the leader, and no pipelines run. See [Lease timing](#lease-timing).\n\nThe new leader starts the pipeline components and resumes from the state in its local RDI\ndatabase instance. See\n[Source databases and data after a failover](#source-databases-and-data-after-a-failover).\n\nAfter a failover, [check the new leader and its pipelines](#verify-a-takeover). When the old\nsite recovers, it remains a follower while the other site holds the lease. To move leadership\nback, see [Move leadership manually](#move-leadership-manually)." }, { "id": "move-leadership-manually", "title": "Move leadership manually", "role": "content", "text": "Use this procedure for a manually controlled takeover. It assumes site A is the leader and\nsite B is the destination.\n\n1. Check that site B is ready to take over:\n\n - Site B reports `Leader Election Mode: follower`:\n\n [code example]\n\n - Site B has the secrets of every pipeline. Compare the `KEY` and `DB` columns at both\n sites:\n\n [code example]\n\n - Site B's instances of the RDI database and the target are in sync. On Redis Software,\n check the replication lag of each database with the\n [database availability API](https://redis.io/docs/latest/operate/rs/monitoring/db-availability#lag-aware):\n\n [code example]\n\n A `200 OK` response means that the lag is within the tolerance, 100 milliseconds by default.\n\n To check that site B can reach the source and target, move leadership to site B and back\n during a maintenance window.\n\n1. On both reachable instances, record the current runtime setting:\n\n [code example]\n\n1. If site B is in standby, enable it. It waits as a follower while site A holds the lease:\n\n [code example]\n\n If site A is unreachable, site B takes over after the lease expires.\n\n1. If site A is reachable, put it in standby. Site A releases the lease, and site B takes over\n within `refreshInterval`:\n\n [code example]\n\n1. [Verify the takeover](#verify-a-takeover).\n1. If site A was unreachable and you use manual failover, set its `standby_mode` to `true` when\n it becomes reachable.\n\nAfter the takeover, set `standby_mode` for the operating mode you want:\n\n| Mode | Site A `standby_mode` | Site B `standby_mode` |\n| :-- | :-- | :-- |\n| Automatic | `false` | `false` |\n| Manual, site A leads | `false` | `true` |\n| Manual, site B leads | `true` | `false` |\n\nTo restore the previous settings instead, use the values you recorded. Run\n`redis-di delete-setting standby_mode` for a recorded value of ``. Don't leave every\ninstance in standby, or no instance runs the pipelines.\n\nTo move leadership back to site A, repeat the procedure with the sites exchanged." }, { "id": "verify-a-takeover", "title": "Verify a takeover", "role": "content", "text": "1. Run `redis-di --context site-a info` and `redis-di --context site-b info` on the reachable\n instances. Confirm that the destination reports `Leader Election Mode: leader` and any\n reachable former leader reports `Leader Election Mode: follower`.\n1. On the new leader, describe each pipeline that should be running. For a new leader at site B:\n\n [code example]\n\n Check its components and errors.\n1. Make a test change in the source and confirm that the expected result reaches the target.\n Leadership alone does not confirm pipeline health or data delivery." }, { "id": "monitor-a-dr-setup", "title": "Monitor a DR setup", "role": "content", "text": "Scrape the operator at both sites and monitor the pipelines of the leader, as described in\n[Observability](https://redis.io/docs/latest/integrate/redis-data-integration/observability). Alert when:\n\n- No instance reports `rdi_operator_is_leader` as `1` for more than 2 minutes. Check the\n standby settings, lease access, and RDI database connectivity.\n- Active-Active replication of the RDI database or the target lags or reports syncer errors.\n- An operator logs `ERROR` messages repeatedly over a longer period.\n\nAfter a takeover, check that the new leader's collector reports `Connected` and that\n`MilliSecondsBehindSource` decreases." }, { "id": "upgrade-or-downgrade-a-dr-setup", "title": "Upgrade or downgrade a DR setup", "role": "content", "text": "Follow the [RDI upgrade instructions](https://redis.io/docs/latest/integrate/redis-data-integration/installation/upgrade).\nUpgrade the leader first. Check leadership, pipeline health, and data delivery before\nupgrading the follower. Finish upgrading both instances before creating a pipeline other\nthan `default`.\n\n> [!WARNING]\n> With a GCS lease, the upgraded leader migrates the lease to a format that versions before\n> 2.2.0 can't read. The follower can take over only after you upgrade it, so upgrade it as\n> soon as you've checked the leader.\n\nDowngrading from RDI 2.0.0 or later to a version earlier than 2.0.0 is not supported. Do not\ndowngrade after creating a pipeline other than `default`. A DR setup with a DynamoDB lease\nrequires RDI 2.2.0 or later.\n\nTo downgrade to RDI 2.0.0 with a GCS lease, plan for downtime:\n\n1. Put both instances in standby, and check that neither reports `Leader Election Mode: leader`:\n\n [code example]\n\n1. Delete the lease object:\n\n [code example]\n\n1. Downgrade both instances with Helm. Versions before 2.2.0 use\n `operator.leaderElection.standbyMode` instead of the `standby_mode` setting. The first\n instance that takes part in leader election recreates the lease in its earlier format.\n1. Check that only one instance is the leader, and verify pipeline health and data delivery.\n\nThe `standby_mode` settings remain in the RDI database. If you upgrade to 2.2.0 again, run\n`redis-di delete-setting standby_mode` on each instance as soon as it runs 2.2.0, or set the\nmode you want." }, { "id": "uninstall-a-dr-setup", "title": "Uninstall a DR setup", "role": "content", "text": "Delete a follower's pipelines with `kubectl delete pipelines`. The RDI API rejects pipeline\nchanges on a follower. When you delete pipelines on a follower, RDI keeps their data in the\nRDI database for the active instance. When you delete pipelines on the active instance, RDI\nremoves their components and cleans up their data in the RDI database.\n\nDelete the pipelines before you run `helm uninstall`. Pipeline resources left behind keep\ntheir finalizers, which stops Kubernetes from deleting the RDI custom resource definitions\n(CRDs) and the namespace." }, { "id": "remove-one-instance", "title": "Remove one instance", "role": "content", "text": "1. If the instance is the leader, follow\n [Move leadership manually](#move-leadership-manually)\n to put it in standby and let the other instance take over.\n1. Check that the other instance reports `Leader Election Mode: leader`, its pipelines run\n without errors, and data reaches the target.\n1. On the instance you want to remove, run the commands in\n [Uninstall RDI](https://redis.io/docs/latest/integrate/redis-data-integration/installation/install-k8s#uninstall-rdi).\n Do not flush the RDI database while another instance uses it." }, { "id": "remove-both-instances", "title": "Remove both instances", "role": "content", "text": "1. Remove the follower first, as described in [Remove one instance](#remove-one-instance).\n1. On the active instance, run the commands in\n [Uninstall RDI](https://redis.io/docs/latest/integrate/redis-data-integration/installation/install-k8s#uninstall-rdi).\n1. After both instances are removed, optionally remove any remaining data as described in\n [Uninstall RDI](https://redis.io/docs/latest/integrate/redis-data-integration/installation/install-k8s#uninstall-rdi)." } ], "examples": [ { "id": "configure-a-lease-ex0", "language": "yaml", "code": "operator:\n leaderElection:\n instanceID: prod-site-a\n dynamoDBLease:\n tableName: \n keyName: rdi-prod-lease", "section_id": "configure-a-lease" }, { "id": "configure-a-lease-ex1", "language": "yaml", "code": "operator:\n leaderElection:\n instanceID: test-site-a\n dynamoDBLease:\n tableName: \n keyName: rdi-test-lease", "section_id": "configure-a-lease" }, { "id": "configure-each-sites-values-file-ex0", "language": "yaml", "code": "connection:\n host: \n port: \n username: \n password: \n\napi:\n jwtKey: \n\noperator:\n leaderElection:\n enabled: true\n instanceID: site-a", "section_id": "configure-each-sites-values-file" }, { "id": "configure-each-sites-values-file-ex1", "language": "yaml", "code": "connection:\n host: \n port: \n username: \n password: \n\napi:\n jwtKey: \n\noperator:\n leaderElection:\n enabled: true\n instanceID: site-b", "section_id": "configure-each-sites-values-file" }, { "id": "deploy-pipelines-and-secrets-ex0", "language": "bash", "code": "redis-di set-context site-a --api-url \"https://\" \\\n --user \"\" --cacert \"\"\nredis-di set-context site-b --api-url \"https://\" \\\n --user \"\" --cacert \"\"", "section_id": "deploy-pipelines-and-secrets" }, { "id": "deploy-pipelines-and-secrets-ex1", "language": "bash", "code": "redis-di --context site-a info\n redis-di --context site-b info", "section_id": "deploy-pipelines-and-secrets" }, { "id": "deploy-pipelines-and-secrets-ex2", "language": "bash", "code": "kubectl --context \"\" --namespace rdi logs deployment/rdi-operator", "section_id": "deploy-pipelines-and-secrets" }, { "id": "deploy-pipelines-and-secrets-ex3", "language": "bash", "code": "redis-di --context site-b set-setting standby_mode true", "section_id": "deploy-pipelines-and-secrets" }, { "id": "deploy-pipelines-and-secrets-ex4", "language": "bash", "code": "redis-di --context site-a set-secret HOST \"\" \\\n --db mysql --pipeline \"\"\n redis-di --context site-a set-secret HOST \"\" \\\n --db target --pipeline \"\"\n redis-di --context site-b set-secret HOST \"\" \\\n --db mysql --pipeline \"\"\n redis-di --context site-b set-secret HOST \"\" \\\n --db target --pipeline \"\"", "section_id": "deploy-pipelines-and-secrets" }, { "id": "deploy-pipelines-and-secrets-ex5", "language": "bash", "code": "redis-di --context site-a deploy \"\" --empty\n redis-di --context site-a deploy \"\" --dir \"\"", "section_id": "deploy-pipelines-and-secrets" }, { "id": "move-leadership-manually-ex0", "language": "bash", "code": "redis-di --context site-b info", "section_id": "move-leadership-manually" }, { "id": "move-leadership-manually-ex1", "language": "bash", "code": "redis-di --context site-a list-secrets --pipeline \"\"\n redis-di --context site-b list-secrets --pipeline \"\"", "section_id": "move-leadership-manually" }, { "id": "move-leadership-manually-ex2", "language": "bash", "code": "curl -u \":\" \\\n \"https://:9443/v1/bdbs//availability?extend_check=lag\"", "section_id": "move-leadership-manually" }, { "id": "move-leadership-manually-ex3", "language": "bash", "code": "redis-di --context site-a get-setting standby_mode\n redis-di --context site-b get-setting standby_mode", "section_id": "move-leadership-manually" }, { "id": "move-leadership-manually-ex4", "language": "bash", "code": "redis-di --context site-b set-setting standby_mode false", "section_id": "move-leadership-manually" }, { "id": "move-leadership-manually-ex5", "language": "bash", "code": "redis-di --context site-a set-setting standby_mode true", "section_id": "move-leadership-manually" }, { "id": "verify-a-takeover-ex0", "language": "bash", "code": "redis-di --context site-b describe \"\"", "section_id": "verify-a-takeover" }, { "id": "upgrade-or-downgrade-a-dr-setup-ex0", "language": "bash", "code": "redis-di --context site-a set-setting standby_mode true\n redis-di --context site-b set-setting standby_mode true", "section_id": "upgrade-or-downgrade-a-dr-setup" }, { "id": "upgrade-or-downgrade-a-dr-setup-ex1", "language": "bash", "code": "gcloud storage rm \"gs:///\"", "section_id": "upgrade-or-downgrade-a-dr-setup" } ] }