{
  "schema_version": 2,
  "id": "integrate/redis-data-integration/data-pipelines/deploy",
  "title": "Deploy a pipeline",
  "url": "https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/data-pipelines/deploy/",
  "summary": "Learn how to deploy an RDI pipeline",
  "content": "\nThe sections below explain how to deploy a pipeline after you have created the required\n[configuration](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/data-pipelines).\n\n## Set secrets\n\nBefore you deploy your pipeline, you must set the authentication secrets for the\nsource and target databases. Each secret has a name that you pass to the\n[`redis-di set-secret`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-set-secret)\ncommand to set the secret value.\nYou can then refer to these secrets in the `config.yaml` file using the syntax \"`${SECRET_NAME}`\"\n(the sample\n[config.yaml file](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config#example)\nshows these secrets in use).\n\nThe table below lists all valid secret names. Note that the\nusername and password are required for the source and target, but the other\nsecrets are only relevant for TLS/mTLS connections.\n\n| Secret name | Description |\n| :-- | :-- |\n| `SOURCE_DB_USERNAME` | Username for the source database |\n| `SOURCE_DB_PASSWORD` | Password for the source database |\n| `SOURCE_DB_CACERT` | (For TLS only) Source database CA certificate |\n| `SOURCE_DB_CERT` | (For mTLS only) Source database client certificate |\n| `SOURCE_DB_KEY` | (For mTLS only) Source database private key |\n| `SOURCE_DB_KEY_PASSWORD` | (For mTLS only) Source database private key password |\n| `TARGET_DB_USERNAME` | Username for the target database |\n| `TARGET_DB_PASSWORD` | Password for the target database |\n| `TARGET_DB_CACERT` | (For TLS only) Target database CA certificate |\n| `TARGET_DB_CERT` | (For mTLS only) Target database client certificate |\n| `TARGET_DB_KEY` | (For mTLS only) Target database private key |\n| `TARGET_DB_KEY_PASSWORD` | (For mTLS only) Target database private key password |\n\n\u003e [!NOTE]\n\u003e When creating secrets for TLS or mTLS, ensure that all certificates and keys are in `PEM` format. The only exception to this is that for PostgreSQL, the source's private key secret must be in `DER` format. If you have a key in `PEM` format, you must convert it to `DER` before creating the secret using the command:\n\n```bash\nopenssl pkcs8 -topk8 -inform PEM -outform DER \\\n    -in /path/to/myclient.pem \\\n    -out /path/to/myclient.pk8 -nocrypt\n```\n\nThis command assumes that the private key is not encrypted. See the [`openssl` documentation](https://docs.openssl.org/master/) to learn how to convert an encrypted private key.\n\n  \n### Set secrets with the CLI\n\nUse [`redis-di set-secret`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-set-secret)\nto set secrets for any installation type (VM, Kubernetes, or Redis Cloud).\n\nThe specific command lines for source secrets are as follows:\n\n```bash\n# For username and password\nredis-di set-secret SOURCE_DB_USERNAME yourUsername\nredis-di set-secret SOURCE_DB_PASSWORD yourPassword\n\n# With source TLS, in addition to the above\nredis-di set-secret SOURCE_DB_CACERT /path/to/myca.crt\n\n# With source mTLS, in addition to the above\nredis-di set-secret SOURCE_DB_CERT /path/to/myclient.crt\nredis-di set-secret SOURCE_DB_KEY /path/to/myclient.key\n# Use this only if SOURCE_DB_KEY is password-protected\nredis-di set-secret SOURCE_DB_KEY_PASSWORD yourKeyPassword \n```\n\nThe corresponding command lines for target secrets are:\n\n```bash\n# For username and password\nredis-di set-secret TARGET_DB_USERNAME yourUsername\nredis-di set-secret TARGET_DB_PASSWORD yourPassword\n\n# With target TLS, in addition to the above\nredis-di set-secret TARGET_DB_CACERT /path/to/myca.crt\n\n# With target mTLS, in addition to the above\nredis-di set-secret TARGET_DB_CERT /path/to/myclient.crt\nredis-di set-secret TARGET_DB_KEY /path/to/myclient.key\n# Use this only if TARGET_DB_KEY is password-protected\nredis-di set-secret TARGET_DB_KEY_PASSWORD yourKeyPassword\n```\n\nBy default, `set-secret` waits for the pipeline to apply the change before returning. When you set\nseveral secrets at once, set all but the last one with `--wait=false` to avoid a timeout while the\npipeline is only partially updated. See [Wait for changes to complete](#wait) below for details.\n\n### Manage secrets with the CLI\n\nAlong with `set-secret`, the CLI has commands to list, inspect, and delete secrets. Because the API\nnever returns secret values, these commands show only the secret keys and whether they are set, not\nthe stored values.\n\n```bash\n# List all the secrets of a pipeline and whether each one is set\nredis-di list-secrets\n\n# Show a single secret and whether it is set\nredis-di describe-secret SOURCE_DB_PASSWORD\n\n# Delete a secret (prompts for confirmation unless you add --force)\nredis-di delete-secret SOURCE_DB_CACERT\n```\n\nSee the reference pages for\n[`list-secrets`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-list-secrets),\n[`get-secret`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-get-secret),\n[`describe-secret`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-describe-secret),\nand [`delete-secret`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-delete-secret)\nfor the full list of options.\n\n### Set secrets for K8s/Helm deployment using Kubectl command\n\n\u003e [!NOTE]\n\u003e It is strongly recommended to manage secrets with the `redis-di` CLI rather than with\n\u003e `kubectl` directly. The CLI applies the correct labels automatically, validates the secret keys, and\n\u003e works the same way across all installation types.\n\nFor a Kubernetes/Helm deployment, you can also use [`kubectl create secret generic`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_generic/)\nto set secrets instead of the CLI. The general pattern of the commands is:\n\n```bash\nkubectl create secret generic \u003cDB\u003e \\\n--namespace=rdi \\\n--from-literal=\u003cSECRET-NAME\u003e=\u003cSECRET-VALUE\u003e\n```\n\nWhere `\u003cDB\u003e` is either `source-db` for source secrets or `target-db` for target secrets.\n\nIf you use TLS or mTLS for either the source or target databases, you also need to create the `source-db-ssl` and/or `target-db-ssl` K8s secrets that contain the certificates used to establish secure connections. The general pattern of the commands is:\n\n```bash\nkubectl create secret generic \u003cDB\u003e-ssl \\\n--namespace=rdi \\\n--from-file=\u003cFILE-NAME\u003e=\u003cFILE-PATH\u003e\n```\n\nThe specific command lines for source secrets are as follows:\n\n```bash\n# Without source TLS\n# Create or update source-db secret\nkubectl create secret generic source-db --namespace=rdi \\\n--from-literal=SOURCE_DB_USERNAME=yourUsername \\\n--from-literal=SOURCE_DB_PASSWORD=yourPassword \\\n--save-config --dry-run=client -o yaml | kubectl apply -f -\n\n# With source TLS\n# Create of update source-db secret\nkubectl create secret generic source-db --namespace=rdi \\\n--from-literal=SOURCE_DB_USERNAME=yourUsername \\\n--from-literal=SOURCE_DB_PASSWORD=yourPassword \\\n--from-literal=SOURCE_DB_CACERT=/etc/certificates/source_db/ca.crt \\\n--save-config --dry-run=client -o yaml | kubectl apply -f -\n# Create or update source-db-ssl secret\nkubectl create secret generic source-db-ssl --namespace=rdi \\\n--from-file=ca.crt=/path/to/myca.crt \\\n--save-config --dry-run=client -o yaml | kubectl apply -f -\n\n# With source mTLS\n# Create or update source-db secret\nkubectl create secret generic source-db --namespace=rdi \\\n--from-literal=SOURCE_DB_USERNAME=yourUsername \\\n--from-literal=SOURCE_DB_PASSWORD=yourPassword \\\n--from-literal=SOURCE_DB_CACERT=/etc/certificates/source_db/ca.crt \\\n--from-literal=SOURCE_DB_CERT=/etc/certificates/source_db/client.crt \\\n--from-literal=SOURCE_DB_KEY=/etc/certificates/source_db/client.key \\\n--from-literal=SOURCE_DB_KEY_PASSWORD=yourKeyPassword \\ # add this only if SOURCE_DB_KEY is password-protected\n--save-config --dry-run=client -o yaml | kubectl apply -f -\n# Create or update source-db-ssl secret\nkubectl create secret generic source-db-ssl --namespace=rdi \\\n--from-file=ca.crt=/path/to/myca.crt \\\n--from-file=client.crt=/path/to/myclient.crt \\\n--from-file=client.key=/path/to/myclient.key \\\n--save-config --dry-run=client -o yaml | kubectl apply -f -\n```\n\nThe corresponding command lines for target secrets are:\n\n```bash\n# Without target TLS\n# Create or update target-db secret\nkubectl create secret generic target-db --namespace=rdi \\\n--from-literal=TARGET_DB_USERNAME=yourUsername \\\n--from-literal=TARGET_DB_PASSWORD=yourPassword \\\n--save-config --dry-run=client -o yaml | kubectl apply -f -\n\n# With target TLS\n# Create of update target-db secret\nkubectl create secret generic target-db --namespace=rdi \\\n--from-literal=TARGET_DB_USERNAME=yourUsername \\\n--from-literal=TARGET_DB_PASSWORD=yourPassword \\\n--from-literal=TARGET_DB_CACERT=/etc/certificates/target_db/ca.crt \\\n--save-config --dry-run=client -o yaml | kubectl apply -f -\n# Create or update target-db-ssl secret\nkubectl create secret generic target-db-ssl --namespace=rdi \\\n--from-file=ca.crt=/path/to/myca.crt \\\n--save-config --dry-run=client -o yaml | kubectl apply -f -\n\n# With target mTLS\n# Create or update target-db secret\nkubectl create secret generic target-db --namespace=rdi \\\n--from-literal=TARGET_DB_USERNAME=yourUsername \\\n--from-literal=TARGET_DB_PASSWORD=yourPassword \\\n--from-literal=TARGET_DB_CACERT=/etc/certificates/target_db/ca.crt \\\n--from-literal=TARGET_DB_CERT=/etc/certificates/target_db/client.crt \\\n--from-literal=TARGET_DB_KEY=/etc/certificates/target_db/client.key \\\n--from-literal=TARGET_DB_KEY_PASSWORD=yourKeyPassword \\ # add this only if TARGET_DB_KEY is password-protected\n--save-config --dry-run=client -o yaml | kubectl apply -f -\n# Create or update target-db-ssl secret\nkubectl create secret generic target-db-ssl --namespace=rdi \\\n--from-file=ca.crt=/path/to/myca.crt \\\n--from-file=client.crt=/path/to/myclient.crt \\\n--from-file=client.key=/path/to/myclient.key \\\n--save-config --dry-run=client -o yaml | kubectl apply -f -\n```\n\nNote that the certificate paths contained in the secrets `SOURCE_DB_CACERT`, `SOURCE_DB_CERT`, and `SOURCE_DB_KEY` (for the source database) and `TARGET_DB_CACERT`, `TARGET_DB_CERT`, and `TARGET_DB_KEY` (for the target database) are internal to RDI, so you *must* use the values shown in the example above. You should only change the certificate paths when you create the `source-db-ssl` and `target-db-ssl` secrets.\n\nSecrets that you create directly with `kubectl` must also be labeled so that the RDI operator\ndiscovers them as pipeline secrets. Each secret needs the following labels, where the\n`app.kubernetes.io/instance` label is the pipeline name (`default` for the default pipeline):\n\n| Label | Value |\n| :-- | :-- |\n| `app.kubernetes.io/name` | `pipeline` |\n| `app.kubernetes.io/instance` | `default` |\n| `product` | `rdi` |\n\nApply the labels to each secret with [`kubectl label`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_label/):\n\n```bash\nkubectl label secret source-db --namespace=rdi --overwrite \\\n  app.kubernetes.io/name=pipeline \\\n  app.kubernetes.io/instance=default \\\n  product=rdi\nkubectl label secret target-db --namespace=rdi --overwrite \\\n  app.kubernetes.io/name=pipeline \\\n  app.kubernetes.io/instance=default \\\n  product=rdi\n\n# With source TLS or mTLS\nkubectl label secret source-db-ssl --namespace=rdi --overwrite \\\n  app.kubernetes.io/name=pipeline \\\n  app.kubernetes.io/instance=default \\\n  product=rdi\n\n# With target TLS or mTLS\nkubectl label secret target-db-ssl --namespace=rdi --overwrite \\\n  app.kubernetes.io/name=pipeline \\\n  app.kubernetes.io/instance=default \\\n  product=rdi\n```\n\n## Deploy a pipeline\n\nWhen you have created your configuration, including the [jobs](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/data-pipelines/transform-examples), you are\nready to deploy. Use the\n[`redis-di deploy`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-deploy)\ncommand to deploy a pipeline:\n\n```bash\nredis-di deploy --dir \u003cpath to pipeline folder\u003e\n```\n\nRDI first validates the configuration and then deploys it if it is correct. You can control the\nvalidation and what happens after deployment with the following options:\n\n- `--dry-run`: Validate the configuration without deploying it. Off by default.\n- `--validate-tables`: Validate the configuration against the source and target databases, for\n  example that the tables it references exist. On by default; pass `--validate-tables=false` to skip\n  this check, which is useful when the databases are not reachable at deploy time.\n- `--validate-cdc`: Additionally validate that the source database is correctly configured for\n  [change data capture (CDC)](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/architecture#overview).\n  Off by default; enable it with `--validate-cdc`.\n- `--start`: Start the pipeline as soon as it is deployed. On by default; pass `--start=false` to\n  deploy the pipeline without starting it, then start it later with\n  [`redis-di start`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-start).\n\nSee the [`redis-di deploy`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-deploy)\nreference page for the full list of options.\n\nYou can also use [Redis Insight](https://redis.io/docs/latest/develop/tools/insight/rdi-connector)\nto configure and deploy pipelines for both VM and K8s installations.\n\n## Display the pipeline status\n\nOnce a pipeline is deployed, use the\n[`redis-di describe`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-describe)\ncommand (also available as `redis-di status`) to display its status. This combines the pipeline\nconfiguration with its runtime status, showing its overall state, its sources and targets, its jobs\nand components, and its per-stream statistics and performance metrics.\n\n```bash\nredis-di describe\n```\n\nTo watch the status update live, pair the command with `watch`:\n\n```bash\nwatch -n 1 redis-di describe\n```\n\nFor a shorter overview, [`redis-di list`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-list)\nprints a one-line summary of the pipeline, and\n[`redis-di get`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-get)\ndoes the same for a single pipeline. See the\n[`redis-di describe`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-describe)\nreference page for details.\n\n## Start and stop a pipeline\n\nUse [`redis-di stop`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-stop)\nto pause a running pipeline and\n[`redis-di start`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-start)\nto resume it. Stopping a pipeline halts data processing without deleting the pipeline or its\nconfiguration, so you can start it again later from where it left off.\n\n```bash\nredis-di stop\nredis-di start\n```\n\n## Reset a pipeline\n\nUse [`redis-di reset`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-reset)\nto return a pipeline to initial full-sync mode. This reloads a fresh\n[snapshot](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/architecture#overview) of the source\ndata and then resumes change data capture (CDC), which is useful when the source and target have\ndrifted out of sync.\n\n```bash\nredis-di reset\n```\n\n## Undeploy a pipeline\n\nTo remove a pipeline, use the\n[`redis-di delete`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-delete)\ncommand. This stops the pipeline and deletes it, along with its configuration and status, from RDI.\nThe secrets you set for the pipeline are not affected.\n\n```bash\nredis-di delete \u003cpipeline\u003e\n```\n\nBecause deleting a pipeline is destructive, the command asks for confirmation unless you add the\n`--force` option. If you omit the pipeline name, the `default` pipeline is deleted.\n\n## Wait for changes to complete {#wait}\n\nThe commands that change a pipeline's state, namely `deploy`, `delete`, `start`, `stop`, `reset`,\n`set-secret`, and `delete-secret`, do not return as soon as the API accepts the request. By default,\nthey wait for the pipeline to finish transitioning to the expected state, polling its status until it\nsucceeds, reaches an error, or the `--timeout` (2 minutes by default) elapses. This is usually what\nyou want: the command reflects the real outcome, so a script can rely on the change having taken\neffect and can fail fast if it did not.\n\nIn some cases, though, a pipeline needs *several* changes before it can transition to a healthy state,\nand waiting after each individual change would time out. The clearest example is rotating both the\nusername and the password of a database: if you set only the username with the default `--wait=true`,\nthe pipeline tries to reconnect with the new username and the old password, fails, and the command\ntimes out after two minutes with the pipeline in a broken state.\n\nTo avoid this, set all the related secrets, or at least all of them except the last, with\n`--wait=false`, so the pipeline applies them together and only the final command waits for it to\nbecome healthy:\n\n```bash\nredis-di set-secret SOURCE_DB_USERNAME newUsername --wait=false\nredis-di set-secret SOURCE_DB_PASSWORD newPassword\n```\n\nThe same applies to any set of changes that are only valid together.\n",
  "tags": ["docs","integrate","rs","rdi"],
  "last_updated": "2026-09-19T17:55:58-07:00"
}