{
  "id": "view-edit",
  "title": "View and edit data pipeline",
  "url": "https://redis.io/docs/latest/operate/rc/rdi/view-edit/",
  "summary": "Observe and change your data pipeline.",
  "tags": [
    "docs",
    "operate",
    "rc"
  ],
  "last_updated": "2026-06-08T15:47:16+02:00",
  "page_type": "content",
  "content_hash": "8140864e40615a1d96251775a13193e00650fe1d9d1b3b727e3d725903caa1eb",
  "sections": [
    {
      "id": "overview",
      "title": "Overview",
      "role": "overview",
      "text": "To view or manage your data pipeline, select it from your workspace from the **Data Integration** page or from the **Data Integration** tab in your subscription or database.\n\nThe pipeline page has the following tabs:\n\n- [Dashboard](#dashboard)\n- [Metrics](#metrics)\n- [Configuration](#configuration)\n- [Dataset](#dataset)\n- [Transformations](#transformations)\n\nThe following sections describe each of these tabs, as well as any actions that you can perform."
    },
    {
      "id": "dashboard",
      "title": "Dashboard",
      "role": "content",
      "text": "The **Dashboard** tab shows an overview and high-level statistics for the data pipeline.\n\n![images/rc/rdi/rdi-view-dashboard.png](https://redis.io/docs/latest/images/rc/rdi/rdi-view-dashboard.png)\n\n- **Pipeline health**: The status of the data pipeline. Possible statuses include:\n    | Status | Description |\n    |--------|-------------|\n    | **Streaming** | The data pipeline is capturing new changes from the source database as they happen. Changes in the source database are added to the target database within a few seconds. |\n    | **Stopped**| The data pipeline has been [stopped](#stop-and-restart-data-pipeline). |\n    | **Error** | There is an error in the data pipeline. [Reset the pipeline](#reset-data-pipeline) and contact support if the issue persists. |\n- **Total ingested**: Total number of records ingested from the source database.\n- **Pending**: Total number of records that are being processed and have not been inserted into the target database.\n- **Record health**: The number of records that were rejected from the database, and the number of records that were filtered from being inserted into the database.\n- **Data latency**: How long it takes for a new record to be ingested from the source database."
    },
    {
      "id": "change-target-database",
      "title": "Change target database",
      "role": "content",
      "text": "You can change the target database for your pipeline if the new target database is in the same subscription of the current target database. To do this:\n\n1. From the **Dashboard** tab of your pipeline, select **More Actions > Change pipeline target** next to the target database of the pipeline.\n\n![images/rc/rdi/rdi-view-target-more-actions.png](https://redis.io/docs/latest/images/rc/rdi/rdi-view-target-more-actions.png)\n\n2. Select the new target database from the list.\n\n![images/rc/rdi/rdi-view-change-pipeline-target.png](https://redis.io/docs/latest/images/rc/rdi/rdi-view-change-pipeline-target.png)\n\n3. Select **Change target** to continue.\n\nChanging the target database restarts the pipeline and re-ingests all data into the new target database."
    },
    {
      "id": "metrics",
      "title": "Metrics",
      "role": "content",
      "text": "The **Metrics** tab shows the following metrics for each data stream:\n\n![images/rc/rdi/rdi-view-metrics.png](https://redis.io/docs/latest/images/rc/rdi/rdi-view-metrics.png)\n\n| Metric | Description |\n|--------|-------------|\n| **Name** | Name of the data stream. Each stream corresponds to a table from the source database.  |\n| **Total** | Total number of records that arrived from the source table. |\n| **Pending** | Number of records from the source table that are waiting to be processed. |\n| **Inserted** | Number of new records from the source table that have been written to the target database. |\n| **Updated** | Number of updated records from the source table that have been updated in the target database. |\n| **Deleted** | Number of deleted records from the source table that have been deleted in the target database. |\n| **Filtered** | Number of records from the source table that were filtered from being inserted into the target database. |\n| **Rejected** | Number of records from the source table that could not be parsed or inserted into the target database. |\n\n<!--"
    },
    {
      "id": "view-metrics-endpoints",
      "title": "View metrics endpoints",
      "role": "content",
      "text": "You can use [Prometheus and Grafana](https://redis.io/docs/latest/integrate/prometheus-with-redis-cloud/) to track and display metrics for the data pipeline. \n\nTo view the metrics endpoints for the source collector and pipeline processor, select **More actions**, and then **Show metrics**. You can add these endpoints as Prometheus targets to start tracking your RDI metrics.\n\nPrometheus endpoints are exposed on Redis Cloud's internal network. To access this network, enable [VPC peering](https://redis.io/docs/latest/operate/rc/security/vpc-peering) or [AWS Transit Gateway](https://redis.io/docs/latest/operate/rc/security/aws-transit-gateway). See [Prometheus and Grafana with Redis Cloud](https://redis.io/docs/latest/integrate/prometheus-with-redis-cloud/) for more information.\n\nFor more information about available RDI metrics, see [Observability](https://redis.io/docs/latest/integrate/redis-data-integration/observability).\n-->"
    },
    {
      "id": "configuration",
      "title": "Configuration",
      "role": "configuration",
      "text": "The **Configuration** tab shows your source connectivity, secrets, and collector configuration properties. \n\n![images/rc/rdi/rdi-view-configuration.png](https://redis.io/docs/latest/images/rc/rdi/rdi-view-configuration.png)\n\nIf you accidentally overwrite the access for your load balancer or secrets, you can find the required ARNs in the **Source connectivity** or **Secrets** section of this tab."
    },
    {
      "id": "edit-collector-properties",
      "title": "Edit collector properties",
      "role": "content",
      "text": "From the **Configuration** tab of your pipeline, select **Edit collector properties** to change your collector properties.\n\n![images/rc/rdi/rdi-advanced-properties.png](https://redis.io/docs/latest/images/rc/rdi/rdi-advanced-properties.png)\n\nYou can add any [Debezium source property](https://debezium.io/documentation/reference/stable/connectors/) for your source database type in the **Collector source properties** section and any [Redis server Debezium sink property](https://debezium.io/documentation/reference/stable/operations/debezium-server.html#_redis_stream) in the **Collector sink properties** section. Select **Save properties** to save the collector properties."
    },
    {
      "id": "dataset",
      "title": "Dataset",
      "role": "content",
      "text": "The **Dataset** tab shows the data from your source database that is ingested into your target database.\n\n![images/rc/rdi/rdi-view-dataset.png](https://redis.io/docs/latest/images/rc/rdi/rdi-view-dataset.png)"
    },
    {
      "id": "edit-dataset",
      "title": "Edit dataset",
      "role": "content",
      "text": "To change the dataset for your data pipeline:\n\n1. From the **Dataset** tab of your pipeline, select **Edit**.\n\n    ![images/rc/rdi/rdi-view-edit-button.png](https://redis.io/docs/latest/images/rc/rdi/rdi-view-edit-button.png)\n\n1. In the **Schemas** section, select the schema(s) you want to migrate to the target database from the list.\n\n    ![images/rc/rdi/rdi-dataset-schema-selected.png](https://redis.io/docs/latest/images/rc/rdi/rdi-dataset-schema-selected.png)\n\n1. When you select a schema, you will see its tables in the **Tables** section. Redis Cloud will automatically select all tables for import. You can de-select any columns you do not wish to import to your Redis database.\n\n1. Select a table to view its columns in the **Columns** section. You can de-select any columns you do not wish to import.\n\n    ![images/rc/rdi/rdi-select-columns.png](https://redis.io/docs/latest/images/rc/rdi/rdi-select-columns.png)\n\n    If any tables are missing a unique key, a warning will appear in the **Data modeling** section. Select **Show affected** to filter the **Tables** section to the tables without a unique key.\n    \n    ![images/rc/rdi/rdi-dataset-missing-unique-key.png](https://redis.io/docs/latest/images/rc/rdi/rdi-dataset-missing-unique-key.png)\n\n    For these tables, select the key icon next to the column that defines a unique key. \n\n    ![images/rc/rdi/rdi-unique-key-selected.png](https://redis.io/docs/latest/images/rc/rdi/rdi-unique-key-selected.png)\n\n1. Select **Save changes** to save your changes.\n\n1. Redis Cloud will display a warning that the data pipeline will restart. Select **Apply and restart** to proceed.\n\nAt this point, the data pipeline will apply the changes. The data pipeline will re-ingest data from the source database to the target database. After this initial sync is complete, the data pipeline enters the *change streaming* phase, where changes are captured as they happen."
    },
    {
      "id": "transformations",
      "title": "Transformations",
      "role": "content",
      "text": "The **Transformations** tab shows the default data structure for records in your target database and any transformation jobs or processor properties you've set for your pipeline.\n\n![images/rc/rdi/rdi-view-transformations.png](https://redis.io/docs/latest/images/rc/rdi/rdi-view-transformations.png)"
    },
    {
      "id": "edit-transformations",
      "title": "Edit transformations",
      "role": "content",
      "text": "To edit any of the information in the **Transformations** tab:\n\n1. From the **Transformations** tab of your pipeline, select **Edit**.\n\n1. Select how your records will be stored in Redis. You can choose **Hash** or **JSON**.\n\n1. Under **Transformation jobs**, you can supply one or more [transformation job files](https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/transform-examples) that specify how you want to transform the captured data before writing it to the target. Select **Upload jobs** to upload your job files. When you upload job files, Redis Cloud will validate the job files to check for errors. \n\n1. Select **Edit advanced properties** to add any processor properties to control how the data is processed. \n\n    ![images/rc/rdi/rdi-processor-advanced-properties.png](https://redis.io/docs/latest/images/rc/rdi/rdi-processor-advanced-properties.png)\n\n    See the [RDI configuration file reference](https://redis.io/docs/latest/integrate/redis-data-integration/reference/config-yaml-reference#processors) for all available processor properties.\n\n1. Select **Save changes**.\n\n1. Redis Cloud will display a warning that the data pipeline will restart. Select **Apply and restart** to proceed.\n\nAt this point, the data pipeline will apply the changes. The data pipeline will re-ingest data from the source database to the target database. After this initial sync is complete, the data pipeline enters the *change streaming* phase, where changes are captured as they happen."
    },
    {
      "id": "reset-data-pipeline",
      "title": "Reset data pipeline",
      "role": "content",
      "text": "Resetting the data pipeline creates a new baseline snapshot from the current state of your source database, and re-processes the data from the source database to the target Redis database. You may want to reset the pipeline if the source and target databases were disconnected or you made large changes to the data pipeline.\n\nTo reset the data pipeline and restart the ingest process:\n\n1. From your pipeline, select **More actions**, and then **Reset pipeline**.\n\n1. If you want to flush the database, check **Flush target database**. \n\n1. Select **Reset data pipeline**.\n\nAt this point, the data pipeline will re-ingest data from the source database to your target Redis database."
    },
    {
      "id": "stop-and-restart-data-pipeline",
      "title": "Stop and restart data pipeline",
      "role": "content",
      "text": "To stop the data pipeline from synchronizing new data:\n\n1. From your pipeline, select **More actions**, and then **Stop pipeline**.\n\n1. Select **Stop data pipeline** to confirm.\n\nStopping the data pipeline will suspend data processing. To restart the pipeline from the **Data pipeline** tab, select **More actions**, and then **Start pipeline**."
    },
    {
      "id": "delete-pipeline",
      "title": "Delete pipeline",
      "role": "content",
      "text": "To delete the data pipeline:\n\n1. From your pipeline, select **More actions**, and then **Delete pipeline**.\n\n1. Select **Delete data pipeline** to confirm.\n\nDeleted data pipelines cannot be recovered. You may also want to [delete your workspace](https://redis.io/docs/latest/operate/rc/rdi/create-workspace#delete-workspace)."
    }
  ],
  "examples": []
}