{
  "id": "quick-start",
  "title": "RDI on Redis Cloud quick start",
  "url": "https://redis.io/docs/latest/operate/rc/rdi/quick-start/",
  "summary": "Learn how to create a data pipeline between a PostgreSQL source database created with Terraform and a Redis Cloud target database.",
  "tags": [
    "docs",
    "operate",
    "rc"
  ],
  "last_updated": "2026-06-08T15:47:16+02:00",
  "page_type": "content",
  "content_hash": "888f939430401ad77bf115250d55f2daeba3e17bddb3fdf7cb55e67e36cc8321",
  "sections": [
    {
      "id": "overview",
      "title": "Overview",
      "role": "overview",
      "text": "The [`rdi-cloud-automation` GitHub repository](https://github.com/redis/rdi-cloud-automation) contains a Terraform script that quickly sets up a PostgreSQL source database on an EC2 instance and all required permissions and network setup to connect it to a Redis Cloud target database.\n\n\nThis guide is for demonstration purposes only. It is not recommended for production use."
    },
    {
      "id": "prerequisites",
      "title": "Prerequisites",
      "role": "content",
      "text": "To follow this guide, you need to:\n\n1. Create a [Redis Cloud Pro database](https://redis.io/docs/latest/operate/rc/databases/create-database/create-pro-database-new) hosted on Amazon Web Services (AWS).\n\n    Turn on Multi-AZ replication and [manually select the availability zones](https://redis.io/docs/latest/operate/rc/databases/configuration/high-availability#availability-zones) when creating the database.\n\n1. Install the [AWS CLI](https://aws.amazon.com/cli/) and set up [credentials for the CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html).\n\n1. Install [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli)."
    },
    {
      "id": "create-a-data-integration-workspace",
      "title": "Create a data integration workspace",
      "role": "content",
      "text": "Before you can create your first Data Integration pipeline for a Redis Cloud subscription, you must first deploy the cloud infrastructure needed to host the pipeline and run the workers associated with the pipeline. In Redis Cloud, this is called a **Workspace**. See [Create and manage Data Integration workspace](https://redis.io/docs/latest/operate/rc/rdi/create-workspace) for more information.\n\nTo create a Data Integration workspace for an existing [Pro subscription](https://redis.io/docs/latest/operate/rc/databases/create-database/create-pro-database-new):\n\n1. From the Redis Cloud console, select **Data Integration** from the left-hand menu. If you don't have any workspaces yet, select **Create workspace** to go to the **Create workspace** page.\n\n    ![images/rc/rdi/rdi-create-workspace-button.png](https://redis.io/docs/latest/images/rc/rdi/rdi-create-workspace-button.png)\n\n    If you already have a workspace deployed, you'll see your current workspaces. Select **New workspace** to go to the **Create workspace** page.\n\n    ![images/rc/rdi/rdi-new-workspace-button.png](https://redis.io/docs/latest/images/rc/rdi/rdi-new-workspace-button.png)\n\n    You can also go to the **Data Integration** tab from your subscription or database page and select **Create workspace** to go to the **Create workspace** page for your subscription.\r\n\n    ![images/rc/rdi/rdi-create-workspace-button.png](https://redis.io/docs/latest/images/rc/rdi/rdi-create-workspace-button.png)\n\n2. Select your Pro subscription from the list if it's not already selected.\n\n    ![images/rc/rdi/rdi-create-workspace-select-subscription.png](https://redis.io/docs/latest/images/rc/rdi/rdi-create-workspace-select-subscription.png)\n\n3. A **Data Integration subnet (CIDR)** is automatically generated for you. If, for any reason, a CIDR is not generated, enter a valid CIDR that does not conflict with your applications or other databases.\n\n    ![images/rc/rdi/rdi-create-workspace-cidr.png](https://redis.io/docs/latest/images/rc/rdi/rdi-create-workspace-cidr.png)\n\n4. Select **Create workspace** to create your workspace.\n\n    ![images/rc/rdi/rdi-create-workspace-button.png](https://redis.io/docs/latest/images/rc/rdi/rdi-create-workspace-button.png)\n\nYour workspace will be created in the background. You can select **Create pipeline** to [create your pipeline](https://redis.io/docs/latest/operate/rc/rdi/define) while the workspace is provisioning, or you can select **Create pipeline later** to go back to the Redis Cloud console."
    },
    {
      "id": "get-required-arns",
      "title": "Get required ARNs",
      "role": "content",
      "text": "1. On the [Redis Cloud console](https://cloud.redis.io/), go to your target database and select the **Data Integration** tab.\n1. Select **Add pipeline**.\n    ![images/rc/rdi/rdi-workspace-add-pipeline.png](https://redis.io/docs/latest/images/rc/rdi/rdi-workspace-add-pipeline.png)\n1. Select **PostgreSQL** as the source database type.\n    ![images/rc/rdi/rdi-select-source-db.png](https://redis.io/docs/latest/images/rc/rdi/rdi-select-source-db.png)\n1. Enter a name for your source database in the **Source name** field. This is a name for the source database that will appear on Redis Cloud.\n1. Select **Continue to source** to move to the **Source configuration** step.\n\n    ![images/rc/rdi/rdi-continue-to-source-button.png](https://redis.io/docs/latest/images/rc/rdi/rdi-continue-to-source-button.png)\n1. Under **Source connectivity**, save the provided ARN. This will be the `redis_privatelink_arn` you will need later.\n\n    ![images/rc/rdi/rdi-setup-connectivity-arn.png](https://redis.io/docs/latest/images/rc/rdi/rdi-setup-connectivity-arn.png)\n\n1. Under **Secrets**, save the provided ARN. This will be the `redis_secrets_arn` you will need later.\n\n    ![images/rc/rdi/rdi-credentials-arn.png](https://redis.io/docs/latest/images/rc/rdi/rdi-credentials-arn.png)"
    },
    {
      "id": "create-the-source-database-and-network-resources",
      "title": "Create the source database and network resources",
      "role": "content",
      "text": "1. Clone or download the [`rdi-cloud-automation` GitHub repository](https://github.com/redis/rdi-cloud-automation).\n\n1. In a terminal window, go to the `examples/aws-ec2-privatelink` directory.\n\n1. Run `terraform init` to initialize the Terraform working directory.\n\n1. Open the `example.tfvars` file and edit the following variables:\n\n    - `region`: The AWS region where your Redis Cloud database is deployed.\n    - `azs`: The availability zone IDs where your Redis Cloud database is deployed.\n    - `port`: The port number for the new PostgreSQL source database.\n    - `name`: A prefix for all of the created AWS resources.\n    - `redis_secrets_arn`: The source database credentials and certificates ARN from the Redis Cloud console.\n    - `redis_privatelink_arn`: The PrivateLink ARN from the Redis Cloud console.\n\n1. To view the configuration, run:\n\n    [code example]\n\n1. To create the AWS resources, run:\n\n    [code example]\n\n    This example creates the following resources on your AWS account:\n\n    - An AWS KMS key with the required permissions for RDI\n    - A VPC with a public and private subnet and all necessary route tables\n    - An EC2 instance running a PostgreSQL database with a security group that allows access from Redis Cloud\n    - An AWS Secrets Manager secret for the PostgreSQL database credentials\n    - A Network Load Balancer (NLB), a listener, and target group to route traffic to the EC2 instance with AWS PrivateLink\n    - An AWS PrivateLink endpoint service for the PostgreSQL database\n\nCreating the AWS resources will take some time. After the resources are created, you'll be able to view them in the AWS management console.\n\nSave the following outputs:\n\n- `database`: The name of the PostgreSQL database.\n- `port`: The port number for the PostgreSQL database.\n- `secret_arn`: The ARN of the AWS Secrets Manager secret for the PostgreSQL database credentials.\n- `vpc_endpoint_service_name`: The name of the AWS PrivateLink endpoint service for the PostgreSQL database.\n\nIf you lose any outputs, run `terraform output` to view them again."
    },
    {
      "id": "resume-pipeline-setup",
      "title": "Resume pipeline setup",
      "role": "content",
      "text": "1. Return to the [Redis Cloud console](https://cloud.redis.io/). Go to your target database and select the **Data Integration** tab.\n1. You'll see a draft pipeline in the workspace you created. Select **More actions > Resume pipeline setup** to continue with pipeline setup.\n\n    ![images/rc/rdi/rdi-workspace-resume-setup.png](https://redis.io/docs/latest/images/rc/rdi/rdi-workspace-resume-setup.png)\n\n1. Continue to the **Source configuration** step.\n\n1. In the **Source connectivity** section, enter the `vpc_endpoint_service_name` output in the **PrivateLink service name** field.\n\n    ![images/rc/rdi/rdi-source-configuration-source-connectivity-privatelink.png](https://redis.io/docs/latest/images/rc/rdi/rdi-source-configuration-source-connectivity-privatelink.png)\n\n1. Select **Connect to Private Link** to test your Private Link connectivity. This will take a few minutes, but you can continue while it's testing.\n\n1. In the **Secrets** section, enter the `secret_arn` output in the **Credentials secret ARN** field.\n\n    ![images/rc/rdi/rdi-source-configuration-secrets.png](https://redis.io/docs/latest/images/rc/rdi/rdi-source-configuration-secrets.png)\n\n1. Select **Validate** to check that Redis Cloud can access your secrets. \n\n1. In the **Source configuration** section, enter the terraform outputs in the following fields.\n    - **Database**: `database`\n    - **Port**: `port`\n\n1. Select **Test source** to test Redis Cloud's connection with the source database. After the test completes, select **Continue to dataset**.\n\n    ![images/rc/rdi/rdi-continue-to-dataset-button.png](https://redis.io/docs/latest/images/rc/rdi/rdi-continue-to-dataset-button.png)\n\n1. In the **Schemas** section, select the schema(s) you want to migrate to the target database from the list.\n\n    ![images/rc/rdi/rdi-dataset-schema-selected.png](https://redis.io/docs/latest/images/rc/rdi/rdi-dataset-schema-selected.png)\n\n1. When you select a schema, you will see its tables in the **Tables** section. Redis Cloud will automatically select all tables for import. You can de-select any columns you do not wish to import to your Redis database.\n\n1. Select a table to view its columns in the **Columns** section. You can de-select any columns you do not wish to import.\n\n    ![images/rc/rdi/rdi-select-columns.png](https://redis.io/docs/latest/images/rc/rdi/rdi-select-columns.png)\n\n1. Select **Continue to transformations** to move to the **Transformations** step.\n\n    ![images/rc/rdi/rdi-continue-to-transformations-button.png](https://redis.io/docs/latest/images/rc/rdi/rdi-continue-to-transformations-button.png)\n\n1. Select how your records will be stored in Redis. You can choose **Hash** or **JSON**.\n\n    ![images/rc/rdi/rdi-transformations.png](https://redis.io/docs/latest/images/rc/rdi/rdi-transformations.png)\n\n1. Review the tables you selected in the **Review and deploy** step. If everything looks correct, select **Deploy pipeline** to start ingesting data from your source database.\n\n    ![images/rc/rdi/rdi-confirm-deploy.png](https://redis.io/docs/latest/images/rc/rdi/rdi-confirm-deploy.png)\n\nAt this point, the data pipeline will ingest data from the source database to your target Redis database. This process will take time, especially if you have a lot of records in your source database. \n\nAfter this initial sync is complete, the data pipeline enters the *change streaming* phase, where changes are captured as they happen. Changes in the source database are added to the target within a few seconds of capture. \n\nYou can view the status of your data pipeline in the **Data pipeline** tab of your database. See [View and edit data pipeline](https://redis.io/docs/latest/operate/rc/rdi/view-edit) to learn more."
    },
    {
      "id": "delete-sample-resources",
      "title": "Delete sample resources",
      "role": "content",
      "text": "Make sure to [delete your data pipeline](https://redis.io/docs/latest/operate/rc/rdi/view-edit#delete-pipeline) before deleting the sample resources.\n\n\nTo delete the sample resources created by Terraform, run:\n\n[code example]"
    }
  ],
  "examples": [
    {
      "id": "create-the-source-database-and-network-resources-ex0",
      "language": "sh",
      "code": "terraform plan -var-file=example.tfvars",
      "section_id": "create-the-source-database-and-network-resources"
    },
    {
      "id": "create-the-source-database-and-network-resources-ex1",
      "language": "sh",
      "code": "terraform apply -var-file=example.tfvars",
      "section_id": "create-the-source-database-and-network-resources"
    },
    {
      "id": "delete-sample-resources-ex0",
      "language": "sh",
      "code": "terraform destroy -var-file=example.tfvars",
      "section_id": "delete-sample-resources"
    }
  ]
}