Install on Kubernetes

Learn how to install RDI on Kubernetes

This guide explains how to use the RDI Helm chart to install on Kubernetes (K8s). You can also Install RDI on VMs.

The installation creates the following K8s objects:

A K8s namespace named rdi.
Deployments for the RDI operator, metrics exporter, and API server.
A service account along with a role and role binding for the RDI operator.
A Configmap for the different components with RDI Redis database details.
Secrets with the RDI Redis database credentials and TLS certificates.

You can use this installation on OpenShift and other K8s distributions including cloud providers' K8s managed clusters.

You can pull the RDI images from Docker Hub or from your own private image registry.

Before you install

Complete the following steps before running Helm:

Create the RDI database on your Redis Enterprise cluster.
Create a user for the RDI database if you prefer not to use the default password (see Access control for more information).
Download the RDI helm chart tar file from the download center.
If you want to use a private image registry, prepare it with the RDI images.

Create the RDI database

RDI uses a database on your Redis Enterprise cluster to store its state information. This requires Redis Enterprise v6.4 or greater.

Use the Redis console to create a database with 250MB RAM with one primary and one replica.
If you are deploying RDI for a production environment then secure this database with a password and TLS.
Set the database's eviction policy to noeviction and set data persistence to AOF - fsync every 1 sec.
Ensure that the RDI database is not clustered. RDI will not work correctly if the RDI database is clustered, but it is OK for the target database to be clustered.

You should then provide the details of this database in the values.yaml file as described below.

Using a private image registry

Add the RDI images from Docker Hub to your local registry. The example below shows how to specify the registry and image pull secret in the values.yaml file for the Helm chart:

global:
 imagePullSecrets: []
 # - name: "image-pull-secret"

 image:
   registry: docker.io
   repository: redis

To pull images from a local registry, you must provide the image pull secret and in some cases also set the permissions. Follow the links below to learn how to use a private registry with:

Install the RDI Helm chart

Scaffold the default values.yaml file from the chart into a local rdi-values.yaml file:
```
helm show values rdi-<rdi-tag>.tar.gz > rdi-values.yaml
```
Open the rdi-values.yaml file you just created and set the appropriate values for your installation (see The values.yaml file below for the full set of available values).

Start the installation:

helm install rdi rdi-<rdi-tag>.tar.gz -f rdi-values.yaml

The `values.yaml` file

The annotated values.yaml file below describes the values you can set for the RDI Helm installation.

At a minimum, you must set the values of RDI_REDIS_HOST and RDI_REDIS_PORT in the global.rdiSysConfig section and also RDI_REDIS_PASSWORD and JWT_SECRET_KEY in global.rdiSysSecret to enable the basic connection to the RDI database. RDI uses the value in JWT_SECRET_KEY to encrypt the JSON web token (JWT) token used by RDI API. Best practice is to generate a value containing 32 random bytes of data (equivalent to 256 bits) and then encode this value as ASCII characters. Use the following command to generate the random key from the urandom special file:

head -c 32 /dev/urandom | base64

Note:

If you want to use Redis Insight to connect to your RDI deployment from outside the K8s cluster, you must enable TLS.

The full values.yaml file is shown below:

# Default RDI values in YAML format.
# Variables to template configuration.

global:
  # Set this property when using a private image repository.
  # Provide an array of image pull secrets.
  # Example:
  # imagePullSecrets:
  #   - name: pullSecret1
  #   - name: pullSecret2
  imagePullSecrets:
    - name: docker-config-jfrog

  # DO NOT modify this value.
  vmMode: false

  # Indicates whether the deployment is intended for an OpenShift environment.
  openShift: false

  image:
    # Overrides the image tag for all RDI components.
    # tag: 0.0.0

    # If using a private repository, update the default values accordingly.
    # Docker registry.
    registry: docker.io

    # Docker image repository.
    repository: redis

  # Configuration for the RDI ConfigMap.
  rdiSysConfig:
    # Log level for all RDI components. Valid options: DEBUG, INFO, ERROR.
    # If specific component log levels are not set, this value will be used.
    RDI_LOG_LEVEL: INFO

    # Log level for the RDI API. Valid options: DEBUG, INFO, ERROR.
    # If not set, RDI_LOG_LEVEL will be used.
    # RDI_LOG_LEVEL_API: INFO

    # Log level for the RDI Operator. Valid options: DEBUG, INFO, ERROR.
    # If not set, RDI_LOG_LEVEL will be used.
    # RDI_LOG_LEVEL_OPERATOR: INFO

    # Log level for the RDI processor. Valid options: DEBUG, INFO, ERROR.
    # If not set, RDI_LOG_LEVEL will be used.
    # RDI_LOG_LEVEL_PROCESSOR: INFO

    # Specifies whether the RDI is configured to use TLS.
    RDI_REDIS_SSL: false

    # RDI_IMAGE_REPO: redis

    # This value must be set to the same tag as global.image.tag.
    # RDI_IMAGE_TAG: ""

    # If using a private repository, set this value to the same secret name as in global.imagePullSecrets.
    # RDI_IMAGE_PULL_SECRET: []

    # The service IP of the RDI database.
    # RDI_REDIS_HOST: ""

    # The port for the RDI database.
    # RDI_REDIS_PORT: ""

    # Enable authentication for the RDI API.
    # RDI_API_AUTH_ENABLED: "1"

    # Specifies whether the API Collector should be deployed.
    # RDI_API_COLLECTOR_ENABLED: "0"

  # Configuration for the RDI Secret.
  rdiSysSecret:
    # Username and password for RDI database.
    # If using the default password, keep the username as an empty string.
    # RDI_REDIS_USERNAME: ""
    # RDI_REDIS_PASSWORD: ""

    # Uncomment this property when using a TLS connection from RDI to its Redis database.
    # DO NOT modify this value.
    # RDI_REDIS_CACERT: /etc/certificates/rdi_db/cacert

    # Uncomment these properties when using an mTLS connection from RDI to its Redis database.
    # DO NOT modify these values.
    # RDI_REDIS_CERT: /etc/certificates/rdi_db/cert
    # RDI_REDIS_KEY: /etc/certificates/rdi_db/key

    # The passphrase used to get the private key stored in the secret store when using mTLS.
    # RDI_REDIS_KEY_PASSPHRASE: ""

    # The key used to encrypt the JWT token used by RDI API. Best practice is for this
    # to contain 32 random bytes encoded as ASCII characters (equivalent to 256 bits of
    # data). See `The values.yaml file` section above to learn how to generate the key.
    # JWT_SECRET_KEY: ""

  rdiDbSSLSecret:
    # Set to `true` when using a TLS connection from RDI to its Redis database.
    enabled: false

    # The content of the CA certificate PEM file.
    # Uncomment and set this property when using a TLS connection from RDI to its Redis database.
    # cacert: ""

    # The content of the certificate PEM file.
    # Uncomment and set this property when using an mTLS connection from RDI to its Redis database.
    # cert: ""

    # The content of the private key PEM file.
    # Uncomment and set this property when using an mTLS connection from RDI to its Redis database.
    # key: ""

  # Container default security context.
  # ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    runAsGroup: 1000
    allowPrivilegeEscalation: false

# Set `isOpenshift` to `true` if deploying on OpenShift.
reloader:
  reloader:
    isOpenshift: false
    deployment:
      containerSecurityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop:
            - ALL
      securityContext:
        runAsUser: null

# Configuration of the RDI Operator.
operator:
  image:
    name: rdi-operator

    # Specify an imagePullPolicy.
    # ref: https://kubernetes.io/docs/concepts/containers/images/#pre-pulled-images
    pullPolicy: IfNotPresent

  # Extra optional options for liveness probe.
  # ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
  liveness:
    failureThreshold: 6
    periodSeconds: 10

  # Extra optional options for readiness probe.
  # ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
  readiness:
    failureThreshold: 6
    periodSeconds: 30

  # Extra optional options for startup probe.
  # ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
  startup:
    failureThreshold: 60
    periodSeconds: 5

fluentd:
  image:
    name: rdi-fluentd

    # Specify an imagePullPolicy.
    # ref: https://kubernetes.io/docs/concepts/containers/images/#pre-pulled-images
    pullPolicy: IfNotPresent

  rdiLogsHostPath: "/opt/rdi/logs"
  podLogsHostPath: "/var/log/pods"
  logrotateMinutes: "5"

rdiMetricsExporter:
  image:
    name: rdi-monitor

    # Specify an imagePullPolicy.
    # ref: https://kubernetes.io/docs/concepts/containers/images/#pre-pulled-images
    pullPolicy: IfNotPresent

  # The RDI metrics service is set to ClusterIP, allowing access only from within the cluster.
  # ref: http://kubernetes.io/docs/user-guide/services/
  service:
    protocol: TCP
    port: 9121
    targetPort: 9121
    type: ClusterIP

  # Configure extra options for liveness probe.
  # ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes
  liveness:
    failureThreshold: 6
    periodSeconds: 10

  # Configure extra options for readiness probe.
  # ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes
  readiness:
    failureThreshold: 6
    periodSeconds: 30

  # Configure extra options for startupProbe.
  # ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes
  startup:
    failureThreshold: 60
    periodSeconds: 5

  # Configuration for the ServiceMonitor, which is used to scrape metrics from the RDI metrics service.
  serviceMonitor:
    # Set to `true` to activate the ServiceMonitor.
    enabled: false

    # The endpoint from which Prometheus will scrape metrics.
    path: /metrics

# Configuration of the RDI API.
apiServer:
  image:
    name: rdi-api

    # Specify an imagePullPolicy.
    # ref: https://kubernetes.io/docs/concepts/containers/images/#pre-pulled-images
    pullPolicy: IfNotPresent

  # The RDI API service is set to ClusterIP, allowing access only from within the cluster.
  # ref: http://kubernetes.io/docs/user-guide/services/
  service:
    type: ClusterIP
    name: rdi-api
    port: 8080
    targetPort: 8081

  # Configure extra options for liveness probe.
  # ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes
  liveness:
    failureThreshold: 6
    periodSeconds: 10

  # Configure extra options for readiness probe.
  # ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes
  readiness:
    failureThreshold: 6
    periodSeconds: 30

  # Configure extra options for startupProbe.
  # ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes
  startup:
    failureThreshold: 60
    periodSeconds: 5

# Expose the RDI API service to be accessed from outside the cluster.
# ref: http://kubernetes.io/docs/user-guide/services/
ingress:
  # When `enabled` is set to `true`, RDI API Ingress will be created.
  enabled: false

  # ref: https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#specifying-the-class-of-an-ingress
  # className: ""

  # Ingress resources configure routes based on the requested host.
  # The optional Host field defines the hosts for routing. If omitted, it matches all hosts.
  # Example:
  # hosts:
  #   - example.com
  #   - another-example.com

  # Annotations to be added to the IngressClass resource.
  # Example:
  # annotations:
  #   kubernetes.io/ingress.class: "nginx"
  #   nginx.ingress.kubernetes.io/rewrite-target: /

  tls:
    # Specifies whether the Ingress should be configured to use TLS.
    enabled: false

    # When `enabled` is set to `true`, set this property to the content of the crt file.
    # crt: ""

    # When `enabled` is set to `true`, set this property to the content of the key file.
    # key: ""

# When `openShift` is set to `true`, Route will be created automatically.
# Route exposes RDI API outside the cluster.
route:
  tls:
    # Specifies whether the Route should be configured to use TLS.
    enabled: false

    # When `enabled` is set to `true`, set this property to the content of the crt file.
    # crt: ""

    # When `enabled` is set to `true`, set this property to the content of the key file.
    # key: ""

collectorSourceMetricsExporter:
  # The collector-source metrics service is set to ClusterIP, allowing access only from within the cluster.
  # ref: http://kubernetes.io/docs/user-guide/services/
  service:
    type: ClusterIP
    port: 9092
    targetPort: 19000

  # Configuration for the ServiceMonitor, which is used to scrape metrics from the collector-source metrics service.
  serviceMonitor:
    # Set to `true` to activate the ServiceMonitor.
    enabled: false

    # The endpoint from which Prometheus will scrape metrics.
    path: /metrics

Check the installation

To verify the status of the K8s deployment, run the following command:

helm list -n monitoring -n rdi

The output looks like the following. Check that <logical_chart_name> is listed.

NAME 	             NAMESPACE    REVISION    UPDATED                STATUS    CHART   	 APP VERSION
<logical_chart_name>    rdi 		   1      2024-10-10 16:53... +0300   IDT    deployed    rdi-1.0.0

Also, check that the following pods have Running status:

kubectl get pod -n rdi

NAME                              READY  STATUS  	RESTARTS   AGE
rdi-api-<id>                       1/1 	 Running 	   0      	29m
rdi-metric-<id>l                   1/1    Running 	   0      	29m
rdi-operator-<id>                  1/1 	 Running 	   0      	29m
<logical_chart_name>-reloader-<id> 1/1 	 Running 	   0      	29m
collector-api-<id>                 1/1    Running       0        29m

You can verify that the RDI API works by adding the server in Redis Insight.

Using ingress controllers

If you want to expose the RDI API service via the K8s Ingress resource, you must ensure that an appropriate ingress controller is available in your K8s cluster. Follow the documentation of your cloud provider or of the ingress controller to install the controller correctly.

Using the `nginx` ingress controller on AKS

On AKS, if you want to use the open source nginx ingress controller rather than the AKS application routing add-on, follow the AKS documentation for creating an unmanaged ingress controller. Specifically, ensure that one or both of the following Helm chart values is set:

controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz
controller.service.externalTrafficPolicy=Local

Prepare your source database

You must also configure your source database to use the CDC connector. See the Prepare source databases section to learn how to do this.

Deploy a pipeline

When the Helm installation is complete, and you have prepared the source database for CDC, you are ready to start using RDI. See the guides to configuring and deploying RDI pipelines for more information. You can also configure and deploy a pipeline using Redis Insight.

Products

Tools

Key Features

See how it works

Get Redis

Use cases

Industries

Customer case studies

Expert services

About

Learn

Connect

Vector search

Products

Tools

Key Features

See how it works

Get Redis

Use cases

Industries

Customer case studies

Expert services

About

Learn

Connect

Vector search

Install on Kubernetes

Before you install

Create the RDI database

Using a private image registry

Install the RDI Helm chart

The `values.yaml` file

Check the installation

Using ingress controllers

Using the `nginx` ingress controller on AKS

Prepare your source database

Deploy a pipeline

On this page

Install on Kubernetes

Before you install

Create the RDI database

Using a private image registry

Install the RDI Helm chart

The values.yaml file

Check the installation

Using ingress controllers

Using the nginx ingress controller on AKS

Prepare your source database

Deploy a pipeline

On this page

The `values.yaml` file

Using the `nginx` ingress controller on AKS