Configure a DynamoDB lease

Create and configure a strongly consistent DynamoDB lease for RDI disaster recovery.

Use an Amazon DynamoDB global table with multi-Region strong consistency (MRSC) to coordinate leader election between Redis Data Integration (RDI) disaster recovery (DR) sites. All sites must use the same table and key name, with each site using a replica Region it can reach.

Before you begin, create the RDI database at both sites. The commands use the Amazon Web Services (AWS) command-line interface (CLI), authenticated as an identity that can create an MRSC global table. Use a current AWS CLI version 2 that supports --global-table-witness-updates and --multi-region-consistency. If either option is unknown, update the AWS CLI. The RDI operator needs separate runtime permissions.

To prepare the lease store with Terraform, use the DynamoDB lease Terraform example. It creates the MRSC global table and operator access policy, and outputs lease settings for each replica Region. After applying it, complete Prepare both sites, then continue with Configure both instances.

Create the table

The table must have a single string partition key named id, with no sort key. Do not enable Time to Live (TTL), which MRSC tables do not support. See DynamoDB consistency modes.

  1. Choose three supported AWS Regions: two for replicas and one for a witness. Each RDI site uses a replica endpoint it can reach. The witness cannot serve reads or writes. For example, you can use us-east-1 and us-east-2 for replicas and us-west-2 for the witness. Check MRSC Region availability before choosing your Regions.

  2. Set the table name and Regions in your shell:

    DR_LEASE_TABLE="<lease-table>"
DR_LEASE_REGION_A="<aws-replica-region-a>"
DR_LEASE_REGION_B="<aws-replica-region-b>"
DR_LEASE_WITNESS_REGION="<aws-witness-region>"

  3. Create the first replica table:

    aws dynamodb create-table --region "$DR_LEASE_REGION_A" \
  --table-name "$DR_LEASE_TABLE" \
  --attribute-definitions AttributeName=id,AttributeType=S \
  --key-schema AttributeName=id,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST

aws dynamodb wait table-exists --region "$DR_LEASE_REGION_A" \
  --table-name "$DR_LEASE_TABLE"

  4. Convert the empty table to an MRSC global table with a second replica and a witness:

    aws dynamodb update-table --region "$DR_LEASE_REGION_A" \
  --table-name "$DR_LEASE_TABLE" \
  --replica-updates "[{\"Create\":{\"RegionName\":\"$DR_LEASE_REGION_B\"}}]" \
  --global-table-witness-updates "[{\"Create\":{\"RegionName\":\"$DR_LEASE_WITNESS_REGION\"}}]" \
  --multi-region-consistency STRONG

    Keep the table empty until the conversion finishes. Do not start RDI against it yet. You choose the consistency mode when creating the global table; you cannot change it on an existing global table. See Create an MRSC global table.

  5. Check readiness from both replica Regions:

    for DR_LEASE_REGION in "$DR_LEASE_REGION_A" "$DR_LEASE_REGION_B"; do
  aws dynamodb describe-table --region "$DR_LEASE_REGION" \
    --table-name "$DR_LEASE_TABLE" \
    --query 'Table.{Status:TableStatus,Consistency:MultiRegionConsistency,KeySchema:KeySchema,Attributes:AttributeDefinitions,Replicas:Replicas[*].[RegionName,ReplicaStatus],Witnesses:GlobalTableWitnesses[*].[RegionName,WitnessStatus]}'
done

    Wait until the output shows ACTIVE for the table, replicas, and witness, and STRONG for the consistency. Confirm the single id partition key with string type S. A newly created replica can take time to become visible.

Grant the operator access

Give each operator's AWS identity the dynamodb:DescribeTable, dynamodb:PutItem, and dynamodb:DeleteItem permissions on the replica it uses. The operator does not need access to the witness. Creating the table requires separate permissions.

For an identity used at both sites, the following Identity and Access Management (IAM) policy grants runtime access to the two replica table Amazon Resource Names (ARNs):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:DescribeTable",
        "dynamodb:PutItem",
        "dynamodb:DeleteItem"
      ],
      "Resource": [
        "arn:aws:dynamodb:<aws-replica-region-a>:<aws-account-id>:table/<lease-table>",
        "arn:aws:dynamodb:<aws-replica-region-b>:<aws-account-id>:table/<lease-table>"
      ]
    }
  ]
}

Attach this policy to the operator's role or user using your IAM administration process. If the sites use different identities, restrict each identity to the replica it uses. Don't restrict dynamodb:ReturnValues to NONE for these actions. The operator reads the current lease owner from a failed conditional write.

Allow outbound Hypertext Transfer Protocol Secure (HTTPS) access from each operator to dynamodb.<aws-replica-region>.amazonaws.com.

Choose one of these identity options for each operator.

Default credential provider chain

RDI uses the AWS default credential provider chain when you omit a credentials file. Configure the identity in the operator's environment, not only in your local shell. On Amazon Elastic Kubernetes Service (EKS), use IAM roles for service accounts or EKS Pod Identity. The operator runs as the Kubernetes service account rdi-operator in the RDI namespace. For IAM roles for service accounts, set the role annotation with operator.serviceAccount.annotations in each values file:

operator:
  serviceAccount:
    annotations:
      eks.amazonaws.com/role-arn: arn:aws:iam::<aws-account-id>:role/<operator-role>

Configure both instances

Merge the following settings into the operator.leaderElection section of each site's values file. Keep the local RDI database connection and other installation settings from Configure each site's values file.

In rdi-values-site-a.yaml:

operator:
  leaderElection:
    enabled: true
    instanceID: site-a
    leaseType: dynamodb
    dynamoDBLease:
      region: <aws-replica-region-a>
      tableName: <lease-table>
      keyName: <lease-item-key>

In rdi-values-site-b.yaml:

operator:
  leaderElection:
    enabled: true
    instanceID: site-b
    leaseType: dynamodb
    dynamoDBLease:
      region: <aws-replica-region-b>
      tableName: <lease-table>
      keyName: <lease-item-key>

Use identical tableName and keyName values at both sites, distinct instanceID values, and each site's selected replica Region. keyName defaults to rdi-lease. To use one table for several DR setups, see Share a lease store between DR setups. RDI creates the lease item.

Install RDI

With the AWS identity configured for both operators, install RDI in each Kubernetes cluster:

helm upgrade --install rdi "rdi-<tag>.tgz" \
  --kube-context "<site-a-kube-context>" --namespace rdi --create-namespace \
  --values rdi-values-site-a.yaml

helm upgrade --install rdi "rdi-<tag>.tgz" \
  --kube-context "<site-b-kube-context>" --namespace rdi --create-namespace \
  --values rdi-values-site-b.yaml

An operator running outside AWS can use an AWS shared credentials file. If you use a shared credentials file, you must pass it to the helm upgrade commands shown above using the --set-file option:

--set-file operator.leaderElection.dynamoDBLease.credentials="<aws-creds-file>"

For example, the file can contain:

[default]
aws_access_key_id = <access-key-id>
aws_secret_access_key = <secret-access-key>

Don't use temporary credentials in this file. RDI reads it only at startup, so they stop working when they expire.

After creating the lease resource and granting access, return to Prepare both sites to complete the source, target, and network preparation.

Continue with Deploy pipelines and secrets to configure RDI CLI contexts. One leader, one follower, and no access errors in the operator logs confirm that both operators can access the table.

RATE THIS PAGE
Back to top ↑