title: Configure Ingress for external routing

title: Prerequisites

text:
Before creating an Ingress, you'll need:

 - A RedisEnterpriseDatabase (REDB) with TLS enabled for client connections
 - A supported Ingress controller with `ssl-passthrough` enabled
    - [Ingress-NGINX Controller](https://kubernetes.github.io/ingress-nginx/deploy/)
        - Be sure to use the `kubernetes/ingress-nginx` controller and NOT the `nginxinc/kubernetes-ingress` controller.
    - [HAProxy Ingress](https://haproxy-ingress.github.io/docs/getting-started/)
    - To use Istio for your Ingress resources, see [Configure Istio for external routing]()

Make sure your Ingress controller has `ssl-passthrough`enabled. This is enabled by default for HAProxy, but disabled by default for NGINX. See the [NGINX User Guide](https://kubernetes.github.io/ingress-nginx/user-guide/tls/#ssl-passthrough) for details.

title: Create an Ingress resource

text:
1. Retrieve the hostname of your Ingress controller's `LoadBalancer` service.

    [code example]

    Below is example output for an HAProxy running on a K8s cluster hosted by AWS.

    [code example]

1. Choose the hostname you will use to access your database (this value will be represented in this article with `<my-db-hostname>`).

1. Create a DNS entry that resolves your chosen database hostname to the IP address for the Ingress controller's LoadBalancer.

1. Create the Ingress resource YAML file.

    [code example]

    For HAProxy, insert the following into the `annotations` section:

    [code example]

    For NGINX, insert the following into the `annotations` section:

    [code example]

    The `ssl-passthrough` annotation is required to allow access to the database. The specific format changes depending on your Ingress controller and any additional customizations. See [NGINX Configuration annotations](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/) and [HAProxy Ingress Options](https://www.haproxy.com/documentation/kubernetes/latest/configuration/ingress/) for updated annotation formats.

title: Test your external access

text:
To test your external access to the database, you need a client that supports [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security) and [SNI](https://en.wikipedia.org/wiki/Server_Name_Indication).

#### Test your access with Openssl

1. Get the default CA certificate from the `redis-enterprise-node` container on any of the Redis Enterprise pods.

    [code example]

1. Run the following `openssl` command, substituting your own values for `<my-db-hostname>`.

    [code example]

    If you are connected to the database, you will receive `PONG` back, as shown below:

    [code example]

#### Test your access with Python

You can use the code below to test your access with Python, substituting your own values for `<my-db-hostname>` and `<file-path>`.

[code example]

Your output should look something like this:

[code example]

code:
$ kubectl get svc <haproxy-ingress | ingress-ngnix-controller> \
                        -n <ingress-ctrl-namespace>

section_id: create-an-ingress-resource

code:
NAME              TYPE           CLUSTER-IP    EXTERNAL-IP                                                              PORT(S)                      AGE
    haproxy-ingress   LoadBalancer   10.43.62.53   a56e24df8c6173b79a63d5da54fd9cff-676486416.us-east-1.elb.amazonaws.com   80:30610/TCP,443:31597/TCP   21m

code:
apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: rec-ingress
      annotations:
        <controller-specific-annotations-below>
    spec:
      rules:
      - host: <my-db-hostname>
        http:
          paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              service:
                name: <db-name>
                port:
                  name: redis

code:
kubernetes.io/ingress.class: haproxy
     ingress.kubernetes.io/ssl-passthrough: "true"

code:
kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"

code:
$ kubectl exec -it <pod-name> -c redis-enterprise-node \
                    -- cat /etc/opt/redislabs/proxy_cert.pem

section_id: test-your-external-access

code:
$ openssl s_client \
      -connect <my-db-hostname>:443 \
      -crlf -CAfile ./proxy_cert.pem \
      -servername <my-db-hostname>

code:
...
     Verify return code: 0 (ok)
    ---

    PING
    +PONG

code:
import redis

r = redis.Redis(host='<my-db-hostname>',
              port=443, db=0, ssl=True,
              ssl_ca_certs='/<file-path>/proxy_cert.pem')

print(r.info())

code:
$ /Users/example-user/Documents/Projects/test_client/venv/bin/python \
    /Users/example-user/Documents/Projects/test_client/test_ssl.py
{
    'redis_version': '5.0.5',
    'redis_git_sha1': 0,
    'redis_git_dirty': 0,
    'redis_build_id': 0,
    'redis_mode': 'standalone',
    'os': 'Linux 4.14.154-128.181.amzn2.x86_64 x86_64',
    'arch_bits': 64,
    'multiplexing_api': 'epoll',
    'gcc_version': '7.4.0',
    'process_id': 1,
    'run_id': '3ce7721b096517057d28791aab555ed8ac02e1de',
    'tcp_port': 10811,
    'uptime_in_seconds': 316467,
    'uptime_in_days': 3,
    'hz': 10,
    'lru_clock': 0,
    'config_file': '',
    'connected_clients': 1,
    'client_longest_output_list': 0,
    'client_biggest_input_buf': 0,
    'blocked_clients': 0,
    'used_memory': 12680016,
    'used_memory_human': '12.9M',
    'used_memory_rss': 12680016,
    'used_memory_peak': 13452496,
    'used_memory_peak_human': '12.82M',
    'used_memory_lua': 151552,
    'mem_fragmentation_ratio': 1,
    'mem_allocator': 'jemalloc-5.1.0',
    'loading': 0,
    'rdb_changes_since_last_save': 0,
    'rdb_bgsave_in_progress': 0,
    'rdb_last_save_time': 1577753916,
    'rdb_last_bgsave_status': 'ok',
    'rdb_last_bgsave_time_sec': 0,
    'rdb_current_bgsave_time_sec': -1,
    'aof_enabled': 0,
    'aof_rewrite_in_progress': 0,
    'aof_rewrite_scheduled': 0,
    'aof_last_rewrite_time_sec': -1,
    'aof_current_rewrite_time_sec': -1,
    'aof_last_bgrewrite_status': 'ok',
    'aof_last_write_status': 'ok',
    'total_connections_received': 4,
    'total_commands_processed': 6,
    'instantaneous_ops_per_sec': 14,
    'total_net_input_bytes': 0,
    'total_net_output_bytes': 0,
    'instantaneous_input_kbps': 0.0,
    'instantaneous_output_kbps': 0.0,
    'rejected_connections': 0,
    'sync_full': 1,
    'sync_partial_ok': 0,
    'sync_partial_err': 0,
    'expired_keys': 0,
    'evicted_keys': 0,
    'keyspace_hits': 0,
    'keyspace_misses': 0,
    'pubsub_channels': 0,
    'pubsub_patterns': 0,
    'latest_fork_usec': 0,
    'migrate_cached_sockets': 0,
    'role': 'master',
    'connected_slaves': 1,
    'slave0': {
        'ip': '0.0.0.0',
        'port': 0,
        'state': 'online',
        'offset': 0,
        'lag': 0
    },
    'master_repl_offset': 0,
    'repl_backlog_active': 0,
    'repl_backlog_size': 1048576,
    'repl_backlog_first_byte_offset': 0,
    'repl_backlog_histlen': 0,
    'used_cpu_sys': 0.0,
    'used_cpu_user': 0.0,
    'used_cpu_sys_children': 0.0,
    'used_cpu_user_children': 0.0,
    'cluster_enabled': 0
}

Process finished with exit code 0


title: Establish external routing on the RedisEnterpriseCluster

page_type: content

title: Overview

role: overview

text: An Ingress is an API resource that provides a standardized and flexible way to manage external access to services running within a Kubernetes cluster.

title: Install Ingress controller

role: setup

text:
Redis Enterprise for Kubernetes supports the Ingress controllers below:
* [HAProxy](https://haproxy-ingress.github.io/)
* [NGINX](https://kubernetes.github.io/ingress-nginx/)
* [Istio](https://istio.io/latest/docs/setup/getting-started/)

OpenShift users can use [routes]() instead of an Ingress.

Install your chosen Ingress controller, making sure `ssl-passthrough` is enabled. `ssl-passthrough` is turned off by default for NGINX but enabled by default for HAProxy.

title: Configure DNS

role: configuration

text:
1. Choose the hostname (FQDN) you will use to access your database according to the recommended naming conventions below, replacing `<placeholders>` with your own values.

     REC API hostname: `api-<rec-name>-<rec-namespace>.<subdomain>`
     REAADB hostname: `-db-<rec-name>-<rec-namespace>.<subdomain>`

     We recommend using a wildcard (`*`) in place of the database name, followed by the hostname suffix.

1. Retrieve the `EXTERNAL-IP` of your Ingress controller's `LoadBalancer` service.

     [code example]

     Below is example output for an HAProxy ingress controller running on a K8s cluster hosted by AWS.

     [code example]

1. Create DNS records to resolve your chosen REC API hostname and database hostname to the `EXTERNAL-IP` found in the previous step.

title: Edit the REC spec

role: content

text: Edit the RedisEnterpriseCluster (REC) spec to add the `ingressOrRouteSpec` field, replacing `<placeholders>` below with your own values.

title: NGINX or HAproxy ingress controllers

text:
* Define the REC API hostname (`apiFqdnUrl`) and database hostname suffix (`dbFqdnSuffix`) you chose when configuring DNS.
* Set `method` to `ingress`.
* Set `ssl-passthrough` to "true".
* Add any additional annotations required for your ingress controller. See [NGINX docs](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/) or [HAproxy docs](https://haproxy-ingress.github.io/docs/configuration/keys/) for more information.

[code example]

title: OpenShift routes

text:
* Define the REC API hostname (`apiFqdnUrl`) and database hostname suffix (`dbFqdnSuffix`) you chose when configuring DNS.
* Set `method` to `openShiftRoute`.

[code example]

OpenShift routes do not require any `ingressAnnotations` in the `ingressOrRouteSpec`.

code:
$ kubectl get svc <haproxy-ingress | ingress-ngnix-controller> \
                         -n <ingress-ctrl-namespace>

section_id: configure-dns

code:
NAME              TYPE           CLUSTER-IP    EXTERNAL-IP                                                              PORT(S)                      AGE
     haproxy-ingress   LoadBalancer   10.43.62.53   a56e24df8c6173b79a63d5da54fd9cff-676486416.us-east-1.elb.amazonaws.com   80:30610/TCP,443:31597/TCP   21m

code:
kubectl patch rec  <rec-name> --type merge --patch "{\"spec\": \
    {\"ingressOrRouteSpec\": \
      {\"apiFqdnUrl\": \"api-<rec-name>-<rec-namespace>.example.com\", \
      \"dbFqdnSuffix\": \"-db-<rec-name>-<rec-namespace>.example.com\", \
      \"ingressAnnotations\": \
       {\"<kubernetes | github>.io/ingress.class\": \
       \"<ingress-controller>\", \
       \"<ingress-controller-annotation>/ssl-passthrough\": \ \"true\"}, \
      \"method\": \"ingress\"}}}"

section_id: nginx-or-haproxy-ingress-controllers

code:
kubectl patch rec <rec-name> --type merge --patch "{\"spec\": \
     {\"ingressOrRouteSpec\": \
     {\"apiFqdnUrl\": \"api-<rec-name>-<rec-namespace>.example.com\" \
     \"dbFqdnSuffix\": \"-db-<rec-name>-<rec-namespace>.example.com\", \
     \"method\": \"openShiftRoute\"}}}"

section_id: openshift-routes


title: Configure Istio for external routing

summary: Configure Istio as an ingress controller for access to your Redis Enterprise databases from outside the Kubernetes cluster.

title: Overview

text:
Redis Enterprise for Kubernetes has the ability to use an Istio Ingress gateway as an alternative to NGINX or HaProxy Ingress controllers.

Istio can also understand Ingress resources, but using that mechanism takes away the advantages and options that the native Istio resources provide. Istio offers its own configuration methods using custom resources.

To configure Istio to work with the Redis Kubernetes operator, we will use two custom resources: a `Gateway` and a `VirtualService`. Then you'll be able to establish external access to your database.

When using Istio ingress for Redis on Kubernetes, avoid partial wildcard hostnames in your REC configuration. Use only valid FQDNs or leftmost wildcards for Istio compatibility and stable external routing. For more information, see [Istio Ingress Fails for Redis on Kubernetes with Partial Wildcard Hostnames](https://support.redislabs.com/hc/en-us/articles/30446436267666-Istio-Ingress-Fails-for-Redis-on-Kubernetes-with-Partial-Wildcard-Hostnames).

title: Install and configure Istio for Redis Enterprise

text:
1. [Download](https://istio.io/latest/docs/setup/getting-started/) and [install](https://istio.io/latest/docs/setup/getting-started/) Istio (see instructions from Istio's [Getting Started](https://istio.io/latest/docs/setup/getting-started/) guide).

    Once the installation is complete, all the deployments, pods, and services will be deployed in a namespace called `istio-system`. This namespace contains a `LoadBalancer` type service called `service/istio-ingressgateway` that exposes the external IP address.

1. Find the `EXTERNAL-IP` for the `istio-ingressgateway` service.

    [code example]

1. Create a DNS entry that resolves your chosen database hostname (or a wildcard `*` followed by your domain) to the Istio `EXTERNAL-IP`. Use this hostname to access your database from outside the cluster.

    In this example, any hostname that ends with `.istio.k8s.my.example.com` will resolve to the Istio LoadBalancer's external IP of `10.145.78.91`. Substitute your own values accordingly.

1. Verify the record was created successfully.

    [code example]

    Look in the `ANSWER SECTION` for the record you just created.

    [code example]

title: Create custom resources

title: `Gateway` custom resource

text:
1. On a different namespace from `istio-system`, create a `Gateway` custom resource file (`redis-gateway.yaml` in this example).

    - Replace `.istio.k8s.my.example.com` with the domain that matches your DNS record.
    - Replace `<selector-label>` with the label set on your Istio ingress gateway pod (most common is `istio: ingress`).
    - TLS passthrough mode is required to allow secure access to the database.

    [code example]

1. Apply the `Gateway` custom resource file to create the Ingress gateway.

    [code example]

1. Verify the gateway was created successfully.

      [code example]

title: `VirtualService` custom resource

text:
1. On a different namespace than `istio-system`, create the `VirtualService` custom resource file (`redis-vs.yaml` in this example).

    [code example]

    This creates both a route to contact the API server on the REC (`rec1`) and a route to contact one of the databases (`db1`).

    - Replace `.istio.k8s.my.example.com` with the domain that matches your DNS record.
    - The gateway's metadata name must be similar to the gateway's spec name (`redis-gateway` in this example).
1. Apply `VirtualService` custom resource file to create the virtual service.

    [code example]

1. Verify the virtual service was created successfully.

    [code example]

1. [Deploy the operator](), Redis Enterprise Cluster (REC), and Redis Enterprise Database (REDB) on the same namespace as the gateway and virtual service.

title: Test your external access to the database

text:
To [test your external access]() to the database, you need a client that supports [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security) and [SNI](https://en.wikipedia.org/wiki/Server_Name_Indication).

See [Test your access with Openssl]() or [Test your access with Python]() for more info.

code:
kubectl get svc istio-ingressgateway -n istio-system

    NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)                                                                      AGE
    istio-ingressgateway   LoadBalancer   10.34.67.89   10.145.78.91   15021:12345/TCP,80:67891/TCP,443:23456/TCP,31400:78901/TCP,15443:10112/TCP   3h8m

code: dig api.istio.k8s.my.example.com

code:
;; ANSWER SECTION:
    api.istio.k8s.my.example.com 0 IN    A       10.145.78.91

code:
apiVersion: networking.istio.io/v1beta1
    kind: Gateway
    metadata:
      name: redis-gateway
    spec:
      selector:
        istio: <selector-label>
      servers:
      - hosts:
        - '*.istio.k8s.my.example.com'
        port:
          name: https
          number: 443
          protocol: HTTPS
        tls:
          mode: PASSTHROUGH

section_id: gateway-custom-resource

code: kubectl apply -f redis-gateway.yaml

code:
kubectl get gateway

      NAME            AGE
      redis-gateway   3h33m

code:
apiVersion: networking.istio.io/v1beta1
    kind: VirtualService
    metadata:
      name: redis-vs
    spec:
      gateways:
      - redis-gateway
      hosts:
      - "*.istio.k8s.my.example.com"
      tls:
      - match:
        - port: 443
          sniHosts:
          - api.istio.k8s.my.example.com
        route:
        - destination:
            host: rec1
            port:
              number: 9443
      - match:
        - port: 443
          sniHosts:
          - db1.istio.k8s.my.example.com
        route:
        - destination:
            host: db1

section_id: virtualservice-custom-resource

code: kubectl apply -f redis-vs.yaml

code:
kubectl get vs

    NAME       GATEWAYS            HOSTS                              AGE
    redis-vs   ["redis-gateway"]   ["*.istio.k8s.my.example.com"]   3h33m


title: Use OpenShift routes for external database access

title: Overview

text: OpenShift routes allow requests to be routed to the database or cluster API from outside the cluster. For more information about routes, see [OpenShift documentation](https://docs.openshift.com/container-platform/4.13/networking/routes/route-configuration.html).

title: Prerequisites

text:
* Before you can connect to your database from outside the cluster, you'll need the root CA certificate of the DMC Proxy server to validate the server certificate.

  By default, the DMC Proxy uses a self-signed certificate.  You can retrieve it from the Redis Enterprise admin console, or from the machine at `/etc/opt/redislabs/proxy_cert.pem` (default location), and save it as a file (for example, named "ca.pem") on the client machine.

* Your database also needs TLS encryption enabled.

title: Create OpenShift route

text:
1. Select the **Networking/Routes** section of the OpenShift web console.

1. Select **Create route** and fill out the following fields:

   * **Name**: Choose any name you want as the first part of your generated hostname
   * **Hostname**: Leave blank
   * **Path**: Leave as is ("/")
   * **Service**: Select the service for the database you want to access
   * **TLS Termination**: Choose "passthrough"
   * **Insecure Traffic**: Select "None"

1. Select **Create**.

1. Find the hostname for your new route. After route creation, it appears in the "Host" field.

1. Verify you have a DNS entry to resolve the hostname for your new route to the cluster's load balancer.

title: Access database

text:
Access the database from outside the cluster using `redis-cli` or `openssl`.

To connect with `redis-cli`:

   [code example]

Replace the `<hostname>` value with the hostname for your new route.

To connect with `openssl`:

   [code example]

code: redis-cli -h <hostname> -p 443 --tls --cacert ./ca.pem --sni <hostname>

section_id: access-database

code: openssl s_client -connect <hostname>:443 -crlf -CAfile ./ca.pem -servername <hostname>


title: Recover a Redis Enterprise cluster on Kubernetes

summary: This task describes how to recover a Redis Enterprise cluster on Kubernetes.

title: Overview

text:
When a Redis Enterprise cluster loses contact with more than half of its nodes either because of failed nodes or network split,
the cluster stops responding to client connections.
When this happens, you must recover the cluster to restore the connections.

You can also perform cluster recovery to reset cluster nodes, to troubleshoot issues, or in a case of active/passive failover.

The Redis Enterprise for Kubernetes automates these recovery steps:

1. Recreates a fresh Redis Enterprise cluster
1. Mounts the persistent storage with the recovery files from the original cluster to the nodes of the new cluster
1. Recovers the cluster configuration on the first node in the new cluster
1. Joins the remaining nodes to the new cluster.

Redis Enterprise for Kubernetes 7.2.4-2 introduces a new limitation. You cannot recover or upgrade your cluster if there are databases with old module versions or manually uploaded modules. See the [Redis Enterprise Software 7.2.4 known limitations]() for more details.

If your cluster uses user-defined modules, the recovery process doesn't block on module validation errors (such as URL or credential issues). The cluster can recover successfully, and you can resolve any module configuration issues after recovery is complete. See [User-defined modules]() for more information.

title: Prerequisites

text: - For cluster recovery, the cluster must be [deployed with persistence]().

title: Recover a cluster

text:
1. Edit the REC resource to set the `clusterRecovery` flag to `true`.

    [code example]

1. Wait for the cluster to recover until it is in the "Running" state.

    To see the state of the cluster, run:

    [code example]

1. To recover the database, see [Recover a failed database]().

code: kubectl patch rec <cluster-name> -n <rec_namespace> --type merge --patch '{"spec":{"clusterRecovery":true}}'

section_id: recover-a-cluster

code: watch "kubectl describe rec | grep State"


title: Connect the Prometheus operator to Redis Enterprise for Kubernetes

summary: This article describes how to configure a Prometheus operator custom resource to allow it to export metrics from Redis Enterprise for Kubernetes.

title: Overview

text: To collect  metrics data from your databases and Redis Enterprise cluster (REC), you can connect your [Prometheus](https://prometheus.io/) server to an endpoint exposed on your REC. Redis Enterprise for Kubernetes creates a dedicated service to expose the `prometheus` port (8070) for data collection. A custom resource called `ServiceMonitor` allows the [Prometheus operator](https://github.com/prometheus-operator/prometheus-operator/tree/main/Documentation) to connect to this port and collect data from Redis Enterprise.

title: Prerequisites

text:
Before connecting Redis Enterprise to Prometheus on your Kubernetes cluster, make sure you've done the following:

- [Deploy Redis Enterprise for Kubernetes]() (version 6.2.10-4 or newer)
- [Deploy the Prometheus operator](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/user-guides/getting-started.md) (version 0.19.0 or newer)
- [Create a Redis Enterprise cluster]()

title: Create a `ServiceMonitor` custom resource

text:
Below is an example `ServiceMonitor` custom resource file. By specifying the service label (`app: redis.io/service=prom-metrics`) in the `selector.matchLabels` section, you can point the Prometheus operator to the correct Redis Enterprise service (`<rec_name>-prom`).

You'll need to configure the following fields to connect Prometheus to Redis Enterprise:

| Section | Field | Value |
|---|---|---|
| `spec.endpoints` | `port` | Name of exposed port (`prometheus`) |
| `spec.namespaceSelector` | `matchNames` | Namespace for your REC |
| `spec.selector` | `matchLabels` | REC service label (`app: redis.io/service=prom-metrics`) |

Apply the file in the same namespace as your Redis Enterprise cluster (REC).
    If Redis Enterprise and Prometheus are deployed in different namespaces, you'll also need to add the [`serviceMonitorNamespaceSelector`](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api-reference/api.md) field to your Prometheus resource. See the [Prometheus operator documentation](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/user-guides/running-exporters.md) for more details on cross-namespace `ServiceMonitor` configuration.

[code example]

For more info about configuring the `ServiceMonitor` resource, see the [`ServiceMonitorSpec` API documentation](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api-reference/api.md#monitoring.coreos.com/v1.ServiceMonitorSpec).

title: More info

text:
- github.com/prometheus-operator
  - [Getting started](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/getting-started/introduction.md)
  - [Running exporters](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/user-guides/running-exporters.md)
  - [Troubleshooting ServiceMonitor changes](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/platform/troubleshooting.md#troubleshooting-servicemonitor-changes)
- redis.io/docs
  - [Metrics in Prometheus]()
  - [Monitoring and metrics]()

code:
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: redis-enterprise
spec:
  endpoints:
  - interval: 15s
    port: prometheus
    scheme: https
    tlsConfig:
      insecureSkipVerify: true
  namespaceSelector:
    matchNames:
    - <your_REC_namespace>
  selector:
    matchLabels:
      redis.io/service: prom-metrics

section_id: create-a-servicemonitor-custom-resource


title: Connect to the admin console

summary: Connect to the Redis Enterprise admin console to manage your Redis Enterprise cluster.

title: Content

text:
The username and password for the Redis Enterprise Software [admin console]() are stored in a Kubernetes [secret](https://kubernetes.io/docs/concepts/configuration/secret/). After retrieving your credentials, you can use port forwarding to connect to the admin console.

There are several methods for accessing the admin console. Port forwarding is the simplest, but not the most efficient method for long-term use. You could also use a load balancer service or Ingress.

1. Switch to the namespace with your Redis Enterprise cluster (REC).

    [code example]

1. Find your cluster name from your list of secrets.

    [code example]

    In this example, the cluster name is `rec`.

1. Extract and decode your credentials from the secret.

    [code example]

1. Find the port for the REC UI service in the `spec:ports` section of the service definition file.

    [code example]

    The default port is 8443.

1. Use `kubectl port-forward` to forward your local port to the service port.

    [code example]

1. View the admin console from a web browser on your local machine at `https://localhost:8443`.

code: kubectl config set-context --current --namespace=<namespace-of-rec>

section_id: content

code: kubectl get secret

code:
kubectl get secret <cluster-name> -o jsonpath='{.data.username}' | base64 --decode
    kubectl get secret <cluster-name> -o jsonpath='{.data.password}' | base64 --decode

code: kubectl get service/<cluster-name>-ui -o yaml

code: kubectl port-forward service/<cluster-name>-ui <local-port>:<service-port>


title: Delete custom resources

summary: This article explains how to delete Redis Enterprise clusters and Redis Enterprise databases from your Kubernetes environment.

title: Multi-namespace management

text: Before you can delete a namespace, you must remove the Redis custom resources ([REDB](#delete-a-database-redb), [REC](#delete-a-redis-enterprise-cluster-rec), and [operator](#delete-operator-from-one-namespace)) residing within a namespace before you can delete that namespace. Deleting namespace containing an REDB may cause errors or data loss.

title: Delete a database (REDB)

text:
To delete a database managed by the Redis Enterprise Kubernetes operator, run `kubectl delete redb <your-db-name>` from your K8s cluster.

When you delete a database, your data and the REDB custom resource are also deleted.

title: Delete a Redis Enterprise cluster (REC)

text:
To delete a Redis Enterprise cluster managed by the operator:

1. Delete all the databases in your cluster.

2. Run `kubectl delete rec <your-rec-name>` from your K8s cluster.

When you delete your cluster, your databases and the REC custom resource are also deleted. However, persistent volume claims (PVCs) for your cluster are not deleted in the process. If you want to delete your PVCs, you'll have to delete them manually.

title: Delete the operator

text:
There are two ways to delete the operator. You can delete the entire operator bundle, but this will also delete any cluster-wide resources created by the operator. To only delete resources within a specific namespace and leave other cluster-wide resources intact, you need to delete each resource manually.

Whether you decide to delete the entire bundle or just resources from a specific namespace, it's important to delete the operator ConfigMap (`operator-environment-config`).

[code example]

title: Delete operator bundle

text:
To delete the operator from your K8s cluster, you can delete the operator bundle with:

- `kubectl delete -f bundle.yaml` for non-OpenShift K8s deployments
- `kubectl delete -f openshift.bundle.yaml` for OpenShift deployments

This will remove the operator and its custom resource definitions (CRDs) from your K8s cluster.

 The Redis Enterprise CRDs are non-namespaced resources, meaning they are shared across your entire K8s cluster. Deleting CRDs in one namespace will delete custom resources in every other namespace across the K8s cluster.

title: Delete operator from one namespace

text:
If you have Redis Enterprise clusters running in different namespaces on the same K8s cluster, deleting the entire operator bundle might cause data loss.

To safely delete the operator from one namespace without affecting the others, delete the operator files individually, excluding the CRD files:

[code example]

You will also need to remove [the `namespaceSelector` section from the validating webhook]().

title: Delete an Active-Active database (REAADB)

text:
1. On one of the existing participating clusters, delete the REEADB (substituting `<reaadb-name>` with your database name).

  [code example]

2. Verify the REAADB no longer exists.

  [code example]

title: Delete a remote cluster (RERC)

text:
1. Verify the RERC you want to delete isn't listed as a participating cluster in any REAADB resources.

    If an RERC is still listed as a participating cluster in any database, the deletion will be blocked.

2. On one of the existing participating clusters, delete the RERC (substituting `<rerc-name>` with your database name).

  [code example]

3. Verify the RERC no longer exists.

  [code example]

title: Troubleshoot REDB deletion

role: errors

text:
The operator attaches a finalizer to the Redis Enterprise database (REDB) object. This makes sure the database is deleted before the REDB custom resource is removed from the K8s cluster.

If the operator isn't running, or some other fatal error occurs, the finalizer isn't removed automatically by the operator. In this case, you won't be able to delete your REDB resource.

If this happens, you can remove the finalizer manually.

 If you remove the finalizer manually, there is no guarantee that the underlying REDB has been deleted. This may cause resource issues and require manual intervention.

[code example]

title: Troubleshoot REC deletion

text:
The operator attaches a finalizer to the Redis Enterprise cluster (REC) object. This makes sure the Redis cluster is deleted before the REC custom resource is removed from the K8s cluster.

If the operator isn't running, or some other fatal error occurs, the finalizer isn't removed automatically by the operator. In this case, you won't be able to delete your REC resource.

If this happens, you can remove the finalizer manually.

 If you remove the finalizer manually, there is no guarantee that the underlying REC has been deleted. This may cause resource issues and require manual intervention.

[code example]

code: kubectl delete cm operator-environment-config

section_id: delete-the-operator

code:
kubectl delete -f role.yaml
kubectl delete -f role_binding.yaml
kubectl delete -f service_account.yaml
kubectl delete -f admission-service.yaml
kubectl delete -f operator.yaml

section_id: delete-operator-from-one-namespace

code: kubectl delete reaadb <reaadb-name>

section_id: delete-an-active-active-database-reaadb

code: kubectl get reaadb -o=jsonpath='{range .items[*]}{.metadata.name}'

code: kubectl delete rerc <rerc-name>

section_id: delete-a-remote-cluster-rerc

code: kubectl get rerc -o=jsonpath='{range .items[*]}{.metadata.name}'

code:
kubectl patch redb <your-db-name> --type=json -p \
    '[{"op":"remove","path":"/metadata/finalizers","value":"finalizer.redisenterprisedatabases.app.redislabs.com"}]'

section_id: troubleshoot-redb-deletion

code:
kubectl patch rec <your-rec-name> --type=json -p \
    '[{"op":"remove","path":"/metadata/finalizers","value":"redbfinalizer.redisenterpriseclusters.app.redislabs.com"}]'

section_id: troubleshoot-rec-deletion


title: Expand PersistentVolumeClaim (PVC)

summary: Expand your persistent volume claim by editing the REC.

title: Overview

text:
This article outlines steps to increase the size of the persistent volume claim for your Redis Enterprise cluster (REC).

 This feature is only supported in versions 7.4.2-12 and above.

[PersistentVolumeClaims (PVC)](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims) are created by the Redis Enterprise operator and used by the RedisEnterpriseCluster (REC). PVCs are created with a specific size and [can be expanded](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims) with the following steps, if the underlying [storage class](https://kubernetes.io/docs/concepts/storage/storage-classes/) supports it.

This process involves deleting and recreating the REC StatefulSet with a larger persistent volume size. The pods owned by the StatefulSet are not restarted or affected by the deletion and recreation process, except when they are left without an owner momentarily.

Shrinking (reducing the size) of your PVC is not allowed. This process only allows you to expand (size up) your PVC.

title: Default PVC size

text:
By default, if you omit [`spec.persistentSpec.volumeSize`](), the operator allocates a persistent volume that is five times (5x) the Redis Enterprise node memory request defined in [`spec.redisEnterpriseNodeResources.requests.memory`]() (per node). This 5x ratio is the recommended minimum capacity.

- If you set [`spec.persistentSpec.volumeSize`]() explicitly, that exact size is used and the 5x default does not apply.
- Changing node memory requests does not automatically resize existing PVCs. Use the procedure below to expand the PVC if you want to maintain the 5x ratio after changing memory.
- If you omit [`spec.redisEnterpriseNodeResources.requests.memory`](), the operator uses its default memory request (4Gi). With `volumeSize` omitted, this results in a default PVC size of approximately 20Gi per node (5 × 4Gi). See [sizing on Kubernetes]() for defaults.

- See the [volume size recommendations]() and [hardware requirements]() for guidance.

title: Prerequisites

text:
Do not change any other REC fields related to the StatefulSet while resizing is in progress.

- PVC expansion must be supported and enabled by the StorageClass and underlying storage driver of the REC PVCs.
  - The relevant StorageClass is the one associated with the REC PVCs. The StorageClass for existing PVCs cannot be changed.
- The StorageClass must be configured with `allowVolumeExpansion: true`.
- Your storage driver must support online expansion.
- We highly recommend you backup your databases before beginning this PVC expansion process.

 OpenShift users should be aware that (`ClusterResourceQuota`) can limit the PVC expansion. Check your quota before resizing using `oc describe clusterresourcequota <quota-name>`.

 PVC expansion is not supported when using Redis Flex (previously Redis on Flash). Do not enable `enablePersistentVolumeResize` if your REC uses `redisOnFlashSpec` as this will result in conflicts.

title: Expand REC PVC

text:
1. Enable the REC persistent volume resize flag.

    [code example]

1. Set the value of `volumeSize` to your desired size.

    [code example]

1. Apply the changes to the REC, replacing `<your-rec.yaml>` with the name of your REC.

  [code example]

After applying the REC changes, the PVCs will begin to expand to the new size.

Once all the PVCs finish the resizing process, the operator will delete and recreate the StatefulSet with the new volume size.

title: Track progress

text:
You can track the progress by monitoring the status of the REC and PersistentVolumeClaim objects.

The REC status will correspond to the status of one or more PVCs, and will reflect if the resizing is successful or failed.

While the resizing is in progress, the status will be:

[code example]

When the resizing is complete, the status becomes Provisioned and the new volume size is available for use by the REC pods.

[code example]

title: Troubleshooting

text:
If an error occurs during this process:

- Examine the status and events of the REC and PVC objects.

  [code example]

  [code example]

- Examine the logs of the operator pods.

  [code example]

section_id: expand-rec-pvc

code:
spec:
        persistentSpec:
          enablePersistentVolumeResize: true
          volumeSize: <new-size>Gi

code: kubectl apply -f <your-rec.yaml>

code:
status:
  persistenceStatus:
    status: Resizing
    succeeded: 2/3

section_id: track-progress

code:
status:
  persistenceStatus:
    status: Provisioned
    succeeded: 3/3

code: kubectl describe pvc

section_id: troubleshooting

code: kubectl get events

code: kubectl logs <operator_pod_name>


title: Redis Enterprise clusters (REC)

summary: Create and manage Redis Enterprise clusters (REC) on Kubernetes using the Redis Enterprise operator.

title: Connect to the admin console

summary: Redis Enterprise for Kubernetes allows you to deploy to multiple namespaces within your Kubernetes cluster. This article shows you how to configure your Redis Enterprise cluster to connect to databases in multiple namespaces

title: Manage databases in multiple namespaces

title: Delete custom resources

title: Expand PersistentVolumeClaim (PVC)

title: Overview

text:
A Redis Enterprise cluster (REC) is a custom Kubernetes resource that represents a Redis Enterprise cluster deployment. The Redis Enterprise operator manages the lifecycle of REC resources, including deployment, scaling, upgrades, and recovery operations.

REC resources define the cluster configuration, including node specifications, storage requirements, security settings, and networking configuration. After you deploy the cluster, it provides a foundation for creating and managing Redis Enterprise databases (REDB).

title: Cluster management

text:
Manage your Redis Enterprise cluster lifecycle and configuration:

- [Connect to admin console]() - Access the Redis Enterprise web UI for cluster management
- [Multi-namespace deployment]() - Deploy clusters across multiple Kubernetes namespaces
- [Delete custom resources]() - Safely remove REC and related resources

title: Storage and performance

text:
Optimize storage and performance for your Redis Enterprise cluster:

- [Redis Flex]() - Configure automatic data tiering between RAM and flash storage
- [Expand PVC]() - Expand persistent volume claims for additional storage

title: Monitoring and observability

text:
Monitor cluster health and performance:

- [Connect to Prometheus operator]() - Integrate with Prometheus for metrics collection and monitoring

title: Call home client

text:
The call home client sends health or error data from your deployment(s) back to Redis. You can disable it by adding the following to your REC specification:

[code example]

The REST API approach used for Redis Software deployments will have no effect on Kubernetes deployments. You must use the REC specification method shown above.

title: Recovery and troubleshooting

text:
Handle cluster recovery and troubleshooting scenarios:

- [Cluster recovery]() - Recover from cluster failures and restore operations

title: Related topics

role: related

text:
- [Redis Enterprise databases (REDB)]() - Create and manage databases on your cluster
- [Security]() - Configure security settings for your cluster
- [Networking]() - Set up networking and ingress for cluster access
- [REC API reference]() - Complete API specification for REC resources

code:
spec:
  usageMeter:
    callHomeClient:
      disabled: true

section_id: call-home-client


title: Manage databases in multiple namespaces

title: Overview

text:
Multiple Redis Enterprise database resources (REDBs) can be associated with a single Redis Enterprise cluster resource (REC) even if they reside in different namespaces.

To learn more about designing a multi-namespace Redis Enterprise cluster, see [flexible deployment options]().

Multi-namespace installations now support Active-Active databases (REAADB) with certain configuration requirements. For details, see [Multi-namespace Active-Active databases](#multi-namespace-active-active-databases).

title: Prerequisites

text: Before configuring a multi-namespace deployment, you must have a running [Redis Enterprise cluster (REC)](). See more information in the [deployment]() section.

title: Create role and role binding for managed namespaces

text:
Both the operator and the RedisEnterpriseCluster (REC) resource need access to each namespace the REC will manage. For each **managed** namespace, create a `consumer_role.yaml` and `consumer_role_binding.yaml` file within the managed namespace, as shown in the examples below.

These will need to be reapplied each time you [upgrade]().

Replace `<rec-namespace>` with the namespace the REC resides in.
Replace `<service-account-name>` with your own value (defaults to the REC name).

`consumer_role.yaml` example:

`consumer_role_binding.yaml` example:

**Alternative approach**: Instead of creating individual `Role` objects for each namespace, you can create a single `ClusterRole` and bind it with multiple `RoleBinding` objects. This reduces the number of objects and simplifies role management.

To use this approach:
1. Change `kind: Role` to `kind: ClusterRole` in the role definition above
2. Change `roleRef.kind: Role` to `roleRef.kind: ClusterRole` in the role binding definition above
3. Apply the ClusterRole once globally, then apply a RoleBinding in each managed namespace

Apply the files, replacing `<managed-namespace>` with your own values:

[code example]

If the REC is configured to watch a namespace without setting the role and role binding permissions, or a namespace that is not yet created, the operator will fail and halt normal operations.

title: Update Redis Enterprise operator ConfigMap

text:
There are two methods of updating the operator ConfigMap (`operator-environment-config`) to specify which namespaces to manage.

- Method 1: Configure the operator to watch for a namespace label and add this label to managed namespaces (available in versions 6.4.2-4 or later).
- Method 2: Configure the operator with an explicit list of namespaces to manage.

You can create this ConfigMap manually before deployment, or it will be created automatically after the operator was deployed.

Only configure the operator to watch a namespace after the namespace is created and configured with the role/role_binding as explained above. If configured to watch a namespace without setting those permissions or a namespace that is not created yet, the operator will fail and not perform normal operations.

title: Method 1: Namespace label (available in versions 6.4.2-4 or later)

text:
1. Create the `cluster_role_binding.yaml` and `cluster_role.yaml` files. Replace the `<rec-namespace>` with the namespace the Redis Enterprise cluster (REC) resides in.

  `operator_cluster_role.yaml` example:

  `operator_cluster_role_binding.yaml` example:

2. Apply the files.

  [code example]

3. Patch the ConfigMap in the REC namespace (`<rec-namespace>`) to identify the managed namespaces with your label (`<label-name>`).

  [code example]

4. For each managed namespace, apply the same label. Replace `<managed-namespace>` with the namespace the REC will manage. Replace `<label-name>` with the value used in the previous step. If you specify a value for `<label-value>`, both the label name and value in managed namespaces must match to be detected by the operator. If the `<label-value>` is empty, only the label name needs to match on managed namespaces and the value is disregarded.

  [code example]

The operator restarts when it detects a namespace label was added or removed.

title: Method 2: Explicit namespace list

text:
Patch the `operator-environment-config` in the REC namespace with a new environment variable (`REDB_NAMESPACES`).

[code example]

title: Multi-namespace Active-Active databases

text:
You can also deploy `RedisEnterpriseActiveActiveDatabase` (REAADB) objects in consumer namespaces separate from the Redis Enterprise operator or cluster (REC) namespace.

To do this:

1. Configure each participating cluster’s operator to watch the relevant consumer namespace. See [multi-namespace operator setup](#update-redis-enterprise-operator-configmap).
2. Ensure all Active-Active prerequisites are met as described in [Configure Active-Active]().
3. In your REAADB custom resource, specify the target consumer namespace using `metadata.namespace`. For each participating cluster, use the `namespace` field under `spec.participatingClusters` to indicate the namespace where the REAADB should be deployed.
4. If you are using a [global database secret](), deploy the secret in each consumer namespace.

Apply the REAADB object to only one Kubernetes cluster. Based on the specified participating clusters and namespaces, the operator automatically creates the necessary resources in the other clusters.

For example:

[code example]

Configure the operator to watch a namespace only after the namespace exists and the required `Role` and `RoleBinding` resources have been applied. If the operator is configured to watch a namespace that lacks these permissions or does not exist, it will fail and halt normal operations.

code:
kubectl apply -f consumer_role.yaml -n <managed-namespace>
kubectl apply -f consumer_role_binding.yaml -n <managed-namespace>

code:
kubectl apply -f operator_cluster_role.yaml
  kubectl apply -f operator_cluster_role_binding.yaml

code:
kubectl patch ConfigMap/operator-environment-config \
  -n <rec-namespace> \
  --type merge \
  -p '{"data": {"REDB_NAMESPACES_LABEL": "<label-name>"}}'

code: kubectl label namespace <managed-namespace> <label-name>=<label-value>

code:
kubectl patch ConfigMap/operator-environment-config \
-n <rec-namespace> \
--type merge \
-p '{"data":{"REDB_NAMESPACES": "<comma,separated,list,of,namespaces,to,watch"}}'

code:
apiVersion: app.redislabs.com/v1alpha1
kind: RedisEnterpriseActiveActiveDatabase
metadata:
  name: consumer-reaadb
  namespace: consumer-namespace-main
spec:
  participatingClusters:
    - name: participating-cluster-main
    - name: participating-cluster-peer
      namespace: consumer-namespace-peer
  globalConfigurations:
    <your global configurations>

section_id: multi-namespace-active-active-databases


title: Manage Redis Enterprise databases for Kubernetes

summary: This section describes how the database controller provides the ability to create, manage, and use databases via a database custom resource.

title: Redis Enterprise database (REDB) lifecycle

text:
A Redis Enterprise database (REDB) is created with a custom resource file. The custom resource defines the size, name, and other specifications for the REDB. The database is created when you apply the custom resource file.

The database controller in Redis Enterprise for Kubernetes:

- Discovers the custom resource
- Makes sure the REDB is created in the same namespace as the Redis Enterprise cluster (REC)
- Maintains consistency between the custom resource and the REDB

The database controller recognizes the new custom resource and validates the specification.
If valid, the controller combines the values specified in
the custom resource with default values to create a full specification. It then uses this full specification to create the
database on the specified Redis Enterprise cluster (REC).

The admission controller prevents duplicate database names when databases are created via the Kubernetes operator.

Once the database is created, it is exposed with the same service mechanisms by the service rigger for the Redis Enterprise cluster.
If the database [custom resource is deleted](), the database and its services are deleted from the cluster.

title: Flexible deployment options

text:
Databases in multiple namespaces can be managed by the same operator. To learn more, see [Manage databases in multiple namespaces]().

To learn more about designing a multi-namespace Redis Enterprise cluster, see [flexible deployment options]().

title: Create a database

text:
Your Redis Enterprise database custom resource must be of the `kind: RedisEnterpriseDatabase` and have values for `name` and `memorySize`. All other values are optional and will be defaults if not specified.

1. Create a file (in this example mydb.yaml) that contains your database custom resource.

    [code example]

    To create a REDB in a different namespace from your REC, you need to specify the cluster with `redisEnterpriseCluster` in the `spec` section of your RedisEnterpriseDatabase custom resource.

     [code example]

1. Apply the file in the namespace you want your database to be in.

    [code example]

1. Check the status of your database.

    [code example]

    When the status is `active`, the database is ready to use.

title: Modify a database

text:
The custom resource defines the properties of the database.
To change the database, you can edit your original specification and apply the change or use `kubectl edit`.

To modify the database:

1. Edit the definition:

    [code example]

1. Change the specification (only properties in `spec` section) and save the changes.
    For more details, see [RedisEnterpriseDatabaseSpec](https://github.com/RedisLabs/redis-enterprise-k8s-docs/blob/master/redis_enterprise_database_api.md#redisenterprisedatabasespec) or [Options for Redis Enterprise databases]().

1. Monitor the status to see when the changes take effect:

    [code example]

    When the status is `active`, the database is ready for use.

title: Delete a database

text:
The database exists as long as the custom resource exists.
If you delete the custom resource, the database controller deletes the database.
The database controller removes the database and its services from the cluster.

To delete a database, run:

[code example]

title: Connect to a database

text:
After the database controller creates a database, services for accessing the database are automatically created in the same namespace. Connection information is stored in a Kubernetes [secret](https://kubernetes.io/docs/concepts/configuration/secret/) maintained by the database controller.

For comprehensive information about connecting to your database, including:

- Service types and access patterns
- Retrieving connection information from secrets
- In-cluster and external access methods
- Connection examples and troubleshooting

See [Database connectivity]().

code:
apiVersion: app.redislabs.com/v1alpha1
    kind: RedisEnterpriseDatabase
    metadata:
      name: mydb
    spec:
      memorySize: 1GB

section_id: create-a-database

code:
redisEnterpriseCluster:
       name: rec

code: kubectl apply -f mydb.yaml

code: kubectl get redb mydb -o jsonpath="{.status.status}"

code: kubectl edit redb mydb

section_id: modify-a-database

code: kubectl delete redb mydb

section_id: delete-a-database


title: Redis Enterprise databases (REDB)

summary: Create and manage Redis Enterprise databases (REDB) on Kubernetes using the Redis Enterprise operator.

summary: Deploy Redis Enterprise databases with modules using the Redis Enterprise operator for Kubernetes.

title: Configure modules

summary: How to create and automate database replicas using the database controller

title: Create replica databases on Kubernetes

title: Overview

text:
A Redis Enterprise database (REDB) is a custom Kubernetes resource that represents a Redis database running on a Redis Enterprise cluster. The Redis Enterprise operator manages REDB resources and handles database creation, configuration, scaling, and lifecycle operations.

REDB resources define database specifications including memory limits, persistence settings, security configurations, networking options, and Redis modules. You can deploy databases on existing Redis Enterprise clusters (REC) and manage them by using standard Kubernetes tools and workflows.

title: Database management

text:
Create and manage Redis Enterprise databases on your cluster:

- [Database controller]() - Understand how the database controller manages REDB resources and database lifecycle

title: Replication and high availability

text:
Set up database replication for high availability and disaster recovery:

- [Create replica databases]() - Configure replica databases for read scaling and disaster recovery scenarios

title: Advanced database configurations

text:
Explore advanced database features and configurations:

- [Active-Active databases]() - Set up globally distributed Active-Active databases across multiple Kubernetes clusters

title: Database connectivity

text:
Connect applications to your Redis Enterprise databases:

- [Database connectivity]() - Comprehensive guide to in-cluster and external database access, service discovery, and credentials management.
- [Networking]() - Configure ingress, routes, and service exposure for database access
- [Security]() - Set up TLS, authentication, and access control for secure database connections

title: Monitoring and troubleshooting

text:
Monitor database performance and troubleshoot issues:

- [Logs]() - Collect and analyze database logs for troubleshooting
- [Connect to Prometheus operator]() - Monitor database metrics with Prometheus

title: Related topics

text:
- [Redis Enterprise clusters (REC)]() - Manage the underlying cluster infrastructure
- [REDB API reference]() - Complete API specification for REDB resources
- [Active-Active database API]() - API reference for Active-Active databases
- [Remote cluster API]() - API reference for remote cluster configurations


title: Configure modules

title: Overview

text: Redis Enterprise modules extend Redis functionality with additional data types, commands, and capabilities. Redis Enterprise versions 8.0.2 and later include several bundled modules that are automatically enabled for compatible database types. You can also add user-defined modules for additional functionality.

title: Prerequisites

text:
Before you begin, verify the following:

- [Redis Enterprise operator deployed]() in your Kubernetes cluster
- [Redis Enterprise cluster (REC)]() running and in a healthy state

title: Bundled modules

text: Redis Enterprise includes several bundled modules that extend Redis functionality with additional data types, commands, and capabilities. Starting with Redis Enterprise version 8.0.2, these modules are automatically included and immediately available for use.

title: Available bundled modules

text:
| Module | Name | Description | Capabilities |
|--------|------|-------------|--------------|
| **[RediSearch]()** | `search` | Full-text search and secondary indexing | Query, aggregation, full-text search, vector similarity search |
| **[RedisJSON]()** | `ReJSON` | JSON data type support | Store, update, and query JSON documents |
| **[RedisTimeSeries]()** | `timeseries` | Time series data structures | Ingest and query time series data with downsampling and aggregation |
| **[RedisBloom]()** | `bf` | Probabilistic data structures | Bloom filters, Cuckoo filters, Count-Min Sketch, Top-K |

When configuring databases with modules, use the `NAME` field (for example, `search` or `ReJSON`) instead of the `DISPLAY_NAME` field.

title: Automatic enablement in Redis 8 and later

text:
For databases created with or upgraded to Redis version 8 or later, bundled modules are automatically enabled based on the database type. You don't need to specify them in the `spec.moduleList` field unless you want to use a specific version.

For databases using Redis versions earlier than 8, explicitly specify bundled modules in the `spec.moduleList` field when you create the database.

title: Check available module versions

text:
To see which bundled module versions are available in your cluster, run the following command:

[code example]

This command displays all modules (both bundled and user-defined) installed in the cluster and their available versions.

title: Bundled vs. user-defined modules

text:
The following table shows the key differences between bundled and user-defined modules:

| Aspect | Bundled modules | User-defined modules |
|--------|----------------|---------------------|
| **Installation** | Pre-installed with Redis Enterprise | Must be added to the REC spec |
| **Availability** | Immediately available | Available after you add them to the REC |
| **Versions** | Bundled with the Redis Enterprise version | Specified by URL in the REC spec |
| **Examples** | RediSearch, RedisJSON, RedisTimeSeries, RedisBloom | RedisGears, custom modules |
| **Redis 8 and later behavior** | Automatically enabled for compatible database types | Must be explicitly specified |

title: User-defined modules

text:
User-defined modules are custom Redis modules that extend Redis functionality beyond the bundled modules. User-defined modules can include third-party modules like RedisGears or custom in-house modules developed for specific use cases.

**Limitations:**

- **Active-Active databases**: User-defined modules are not supported with Active-Active databases. Only bundled modules (RediSearch, RedisJSON, RedisTimeSeries, RedisBloom) can be used with Active-Active databases.

- **Redis on Flash**: User-defined modules are fully supported with Redis on Flash databases.

title: Add user-defined modules to the REC

text:
To use user-defined modules with your databases, first add them to the Redis Enterprise cluster (REC) custom resource. This enables the operator to validate the modules and make them available for database creation.

Add user-defined modules to the REC **before** you create any databases that use them. The admission controller validates that modules exist in the REC before allowing REDB creation.

1. Edit your REC custom resource:

    [code example]

2. Add the `userDefinedModules` section to the `spec`:

    [code example]

3. If your module repository requires authentication, create a secret with your credentials:

    [code example]

If you use [HashiCorp Vault integration](), store the module credentials in Vault instead of creating a Kubernetes secret. The `credentialsSecret` field will reference the secret name in Vault.

title: Module naming requirements

text:
The `name` field in the REC spec must match either the `module_name` or `display_name` value from the module's manifest file (`module.json` inside the module zip file). This enables the operator to validate the module.

For example, if your module manifest contains the following:

[code example]

You can use either `"rg"` or `"RedisGears"` as the `name` value in your REC spec.

If the names don't match, the operator can't validate the module. This can lead to preventable errors during database creation or upgrades.

title: Edit user-defined modules

text:
To modify the user-defined modules list, complete the following steps:

1. Edit the REC custom resource:

    [code example]

1. Update the `userDefinedModules` section:
   - **Add new modules**: Append them to the list.
   - **Update module URLs**: Change the `url` field for existing modules.
   - **Update credentials**: Change the `credentialsSecret` reference.

1. Save your changes. The operator validates and applies the updates.

Don't remove modules that are currently in use by any database. The operator rejects the change and puts the REC into an error state.

Changes to the `userDefinedModules` list trigger a rolling restart of the Redis Enterprise cluster pods. Plan module updates during a maintenance window to minimize potential impact on your databases.

title: Verify user-defined modules

text:
After you add user-defined modules to the REC, verify that they're available:

[code example]

You can also check the REC status for validation errors:

[code example]

Look for events or status messages related to module validation in the output.

title: Upgrade with modules

text: The upgrade process differs depending on whether you use bundled modules or user-defined modules.

title: Module version selection

text:
When multiple versions of a module are available in the cluster, Redis Enterprise selects the appropriate version based on the `compatible_redis_version` field in the module's manifest file (`module.json`). This field must match the Redis OSS version that the database is using.

For example, if your database uses Redis 7.2, Redis Enterprise selects the module version whose `compatible_redis_version` is `7.2`. If no matching version is found, the module cannot be loaded.

title: Upgrade with bundled modules

text:
For databases using bundled modules (RediSearch, RedisJSON, RedisTimeSeries, RedisBloom):

- **Redis 8 and later**: Bundled modules are automatically enabled and upgraded when you upgrade the database to Redis version 8 or later. You don't need to take any additional action. The module version is automatically selected based on the database's Redis version.

- **Redis versions earlier than 8**: Bundled modules are upgraded automatically when you upgrade the Redis Enterprise cluster. The bundled module versions are tied to the Redis Enterprise version, and the appropriate version is selected based on the database's Redis version.

title: Upgrade with user-defined modules

text:
For databases using user-defined modules, you must take additional steps during cluster upgrades:

1. Set `autoUpgradeRedisEnterprise` to `false` in your REC spec before upgrading.

1. Add or update the `userDefinedModules` list in the REC spec with the new module versions before or during the cluster upgrade. Ensure that the new module versions include a `compatible_redis_version` field that matches the Redis version your databases will use after the upgrade.

1. After the cluster upgrade completes, you can re-enable `autoUpgradeRedisEnterprise` if desired.

For detailed upgrade instructions, see the following:

- [Upgrade a Redis Enterprise cluster (REC)]()
- [Upgrade Redis Enterprise on OpenShift]()

title: Troubleshooting

text: This section covers common issues you might encounter when working with user-defined modules.

title: Module validation errors

text:
Module validation errors occur when the operator can't validate a user-defined module. Common causes include incorrect URLs, authentication failures, or invalid module manifests.

**Symptoms:**

- REC status shows validation errors
- Events indicate module download or validation failures
- Databases fail to create with module-related errors

**Diagnosis:**

Check the REC status for validation errors:

[code example]

Look for error messages related to modules in the Events section.

**Resolution:**

1. **Verify the module URL is accessible:**

    [code example]

2. **Check credentials secret exists and has correct values:**

    [code example]

3. **Verify the module manifest (`module.json`) is valid:**

    Download the module zip file and check that it contains a valid `module.json` file with required fields: `module_name`, `display_name`, `semantic_version`, `commands`, and `compatible_redis_version`.

4. **Ensure the `name` field in the REC spec matches the module manifest:**

    The `name` must match either `module_name` or `display_name` from the module's `module.json` file. See [Module naming requirements](#module-naming-requirements) for details.

title: Bootstrap failures

text:
Bootstrap failures occur when the Redis Enterprise cluster fails to start due to module-related issues.

**Symptoms:**

- REC pods fail to reach Running state
- Operator logs show module-related errors during bootstrap
- Cluster remains in a non-ready state

**Diagnosis:**

Check the operator logs:

[code example]

Check the REC pod logs:

[code example]

**Resolution:**

1. **Remove problematic modules from the REC spec:**

    Edit the REC and remove or comment out the problematic module from the `userDefinedModules` list:

    [code example]

2. **Wait for the cluster to recover:**

    [code example]

3. **Fix the module configuration and re-add it:**

    After the cluster is running, correct the module URL, credentials, or manifest issues, then add the module back to the REC spec.

title: Module not found errors

text:
Module not found errors occur when you try to create a database that uses a module that isn't defined in the REC.

**Symptoms:**

- REDB creation fails with admission webhook errors
- Error message indicates the module is not found in the REC
- Database remains in a pending or failed state

**Diagnosis:**

Check the REDB creation error:

[code example]

Verify which modules are defined in the REC:

[code example]

**Resolution:**

1. **Add the missing module to the REC:**

    See [Add user-defined modules to the REC](#add-user-defined-modules-to-the-rec) for detailed instructions.

2. **Wait for the module to be validated:**

    [code example]

    Look for successful validation in the Events section.

3. **Retry database creation:**

    After the module is available in the REC, the database creation should succeed automatically, or you can delete and recreate the REDB.

title: Related information

text:
- [Redis modules documentation]() - Official Redis modules documentation
- [REDB API reference]() - Complete API specification for REDB resources
- [REAADB API reference]() - API reference for Active-Active databases

title: Redis Software documentation

text:
- [Add modules to a cluster]() - Install module packages on Redis Enterprise Software clusters
- [Enable modules for a database]() - Add modules to databases in Redis Enterprise Software
- [Upgrade modules]() - Upgrade module versions in Redis Enterprise Software
- [Module lifecycle]() - Module versioning and end-of-life schedule

code: kubectl get rec <cluster-name> -o jsonpath='{.status.modules}' | jq

section_id: check-available-module-versions

code: kubectl edit rec <cluster-name>

section_id: add-user-defined-modules-to-the-rec

code:
spec:
      userDefinedModules:
        - name: "custom-module"
          source:
            https:
              url: "https://modules.company.com/custom-module-v1.0.zip"
              credentialsSecret: "module-repo-creds"

code:
kubectl create secret generic module-repo-creds \
      --from-literal=username=<your-username> \
      --from-literal=password=<your-password>

module_name: rg

display_name: RedisGears

section_id: module-naming-requirements

section_id: edit-user-defined-modules

code: kubectl get rec <cluster-name> -o jsonpath='{.spec.userDefinedModules}' | jq

section_id: verify-user-defined-modules

code: kubectl describe rec <cluster-name>

section_id: module-validation-errors

code: curl -I <module-url>

code: kubectl get secret <credentials-secret-name> -o yaml

code: kubectl logs -l name=redis-enterprise-operator -n <namespace>

section_id: bootstrap-failures

code: kubectl logs <rec-pod-name> -n <namespace>

code: kubectl get rec <cluster-name> -w

code: kubectl describe redb <database-name>

section_id: module-not-found-errors


title: Create replica databases on Kubernetes

title: Overview

text:
You can configure a replica of a database by creating an item in
the [`replicaSources`]() section of the RedisEnterpriseDatabase (REDB) custom resource.

A secret must be created with the `stringData` section containing the replica source URI as follows:

Create a secret with the replica source URI listed in the `stringData` field as follows:

[code example]

The replica source URL can be retrieved by going to "UI > database > configuration > Press the button Get Replica of source URL"
in the administrative UI. But, this information can also be retrieved directly from
the REST API as well.

A replica of database CR simply uses the secret in the `replicaSources` section:

[code example]

In the above, `name-of-replica` database will be created as a replica of the
source database as long as the source database exists on the source cluster
and the secret contains the correct replica source URL for that database.

title: Retrieving the replica source URL via kubectl

text:
You will need `kubectl`, `curl`, and `jq` installed for this procedure.

1. Set your metadata:

   [code example]

1. Retrieve the cluster authentication:

   [code example]

1. Forward the port of the REST API service for your source cluster:

   [code example]

1. Request the information from the REST API:

   [code example]

   Note: URI now contains the replica source URI.

1. Construct the secret for the replica:

   [code example]

1. Create the replica database:

   [code example]

title: Automating the creation via a job

text:
The following procedure uses a ConfigMap and a Job to construct the replica
source URL secret from the source database and configure the target database.

There are four parameters:

- `source` - the name of the source database
- `cluster` - the name of the cluster for the source database
- `target` - the name of the target database
- `targetCluster` - the name of the cluster for the target database

These parameters can be set by:

[code example]

where "name-of-..." is replaced with the database source, source cluster,
database target, and target cluster names.

The Job and ConfigMap below, when submitted, will create the secret and
replica database:

[code example]

code:
apiVersion: v1
kind: Secret
metadata:
   name: my-replica-source
stringData:
   uri: <replica-source-uri-goes-here>

section_id: overview

code:
apiVersion: app.redislabs.com/v1alpha1
kind: RedisEnterpriseDatabase
metadata:
   name: name-of-replica
spec:
   redisEnterpriseCluster:
      name: name-of-cluster
   replicaSources:
   - replicaSourceType: SECRET
     replicaSourceName: my-replica-source

code:
CLUSTER_NAME=test
   SOURCE_DB=db1
   TARGET_DB=db2
   TARGET_CLUSTER_NAME=test

code: kubectl port-forward pod/${CLUSTER_NAME}-0 9443

code:
JQ='.[] | select(.name=="'
   JQ+="${SOURCE_DB}"
   JQ+='") | ("redis://admin:" + .authentication_admin_pass + "@"+.name+":"+(.endpoints[0].port|tostring))'
   URI=`curl -sf -k -u "$CLUSTER_USER:$CLUSTER_PASSWORD" "https://localhost:9443/v1/bdbs?fields=uid,name,endpoints,authentication_admin_pass" | jq "$JQ" | sed 's/"//g'`

code:
cat << EOF > secret.yaml
   apiVersion: v1
   kind: Secret
   metadata:
     name: ${SOURCE_DB}-url
   stringData:
     uri: ${URI}
   EOF
   kubectl apply -f secret.yaml

code:
cat << EOF > target.yaml
   apiVersion: app.redislabs.com/v1alpha1
   kind: RedisEnterpriseDatabase
   metadata:
     name: ${TARGET_DB}
   spec:
     redisEnterpriseCluster:
       name: ${TARGET_CLUSTER_NAME}
     replicaSources:
     - replicaSourceType: SECRET
       replicaSourceName: ${SOURCE_DB}-url
   EOF
   kubectl apply -f target.yaml

code:
kubectl create configmap replica-of-database-parameters \
--from-literal=source=name-of-source \
--from-literal=cluster=name-of-cluster \
--from-literal=target=name-of-target \
--from-literal=targetCluster=name-of-cluster

section_id: automating-the-creation-via-a-job

code:
apiVersion: batch/v1
kind: Job
metadata:
  name: replica-of-database
spec:
  backoffLimit: 4
  template:
    spec:
      serviceAccountName: redis-enterprise-operator
      restartPolicy: Never
      volumes:
        - name: scripts
          configMap:
            name: replica-of-database
      containers:
        - name: createdb
          image: debian:stable-slim
          env:
            - name: MY_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: SCRIPT
              value: create.sh
            - name: SOURCE_DB
              valueFrom:
                configMapKeyRef:
                  name: replica-of-database-parameters
                  key: source
            - name: TARGET_DB
              valueFrom:
                configMapKeyRef:
                  name: replica-of-database-parameters
                  key: target
            - name: CLUSTER_SERVICE
              value: .svc.cluster.local
            - name: CLUSTER_NAME
              valueFrom:
                configMapKeyRef:
                  name: replica-of-database-parameters
                  key: cluster
            - name: CLUSTER_PORT
              value: "9443"
            - name: TARGET_CLUSTER_NAME
              valueFrom:
                configMapKeyRef:
                  name: replica-of-database-parameters
                  key: targetCluster
          volumeMounts:
            - mountPath: /opt/scripts/
              name: scripts
          command:
            - /bin/bash
            - -c
            - |
              apt-get update; apt-get install -y curl jq apt-transport-https gnupg2
              apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 6A030B21BA07F4FB
              curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
              echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee -a /etc/apt/sources.list.d/kubernetes.list
              apt-get update
              apt-get install -y kubectl
              bash /opt/scripts/$SCRIPT
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: replica-of-database
data:
  create.sh: |
    CLUSTER_USER=`kubectl get secret/${CLUSTER_NAME} -o json | jq -r .data.username | base64 -d`
    CLUSTER_PASSWORD=`kubectl get secret/${CLUSTER_NAME} -o json | jq -r .data.password | base64 -d`
    CLUSTER_HOST=${CLUSTER_NAME}.${MY_NAMESPACE}${CLUSTER_SERVICE}
    JQ='.[] | select(.name=="'
    JQ+="${SOURCE_DB}"
    JQ+='") | ("redis://admin:" +  .authentication_admin_pass + "@"+.endpoints[0].dns_name+":"+(.endpoints[0].port|tostring))'
    URI=`curl -sf -k -u "$CLUSTER_USER:$CLUSTER_PASSWORD" "https://${CLUSTER_HOST}:${CLUSTER_PORT}/v1/bdbs?fields=uid,name,endpoints,authentication_admin_pass" | jq "$JQ" | sed 's/"//g'`
    echo "URL: ${URL}"
    echo ""
    cat << EOF > /tmp/secret.yaml
    apiVersion: v1
    kind: Secret
    metadata:
      name: ${SOURCE_DB}-url
    stringData:
      uri: ${URI}
    EOF
    cat /tmp/secret.yaml
    cat << EOF > /tmp/target.yaml
    apiVersion: app.redislabs.com/v1alpha1
    kind: RedisEnterpriseDatabase
    metadata:
      name: ${TARGET_DB}
    spec:
      redisEnterpriseCluster:
        name: ${TARGET_CLUSTER_NAME}
      replicaSources:
      - replicaSourceType: SECRET
        replicaSourceName: ${SOURCE_DB}-url
    EOF
    echo "---"
    cat /tmp/target.yaml
    echo ""
    kubectl -n ${MY_NAMESPACE} apply -f /tmp/secret.yaml
    kubectl -n ${MY_NAMESPACE} apply -f /tmp/target.yaml


title: Configuration recommendations

summary: Best practices and configuration recommendations for Redis Enterprise on Kubernetes deployments.

summary: Recommendations on managing node resources and setting eviction thresholds with Redis Enterprise for Kubernetes.

title: Manage node resources

summary: This section covers details about how persistent volumes are sized and specified for Redis Enterprise cluster deployments.

title: Use persistent volumes in Redis Enterprise clusters

summary: This section provides information about sizing and scaling Redis Enterprise in a Kubernetes deployment.

title: Size and scale a Redis Enterprise cluster deployment on Kubernetes

summary: This section provides information about how Redis Enterprise cluster pods can be scheduled to only be placed on specific nodes or node pools.

title: Control node selection

summary: This section provides information about how you can use quality of service, priority class, eviction thresholds and resource monitoring to maintain cluster node pod stability.

title: Manage pod stability

title: Overview

text: Follow these best practices and configuration recommendations to optimize your Redis Enterprise deployment on Kubernetes for performance, reliability, and scalability.

title: Infrastructure recommendations

text:
Configure your Kubernetes infrastructure for optimal Redis Enterprise performance:

- [Node resources]() - CPU, memory, and resource allocation recommendations
- [Node selection]() - Best practices for selecting and configuring Kubernetes nodes
- [Persistent volumes]() - Storage configuration and persistent volume recommendations

title: Deployment recommendations

text:
Optimize your Redis Enterprise deployment configuration:

- [Sizing on Kubernetes]() - Guidelines for sizing clusters and databases
- [Pod stability]() - Ensure stable pod operations and prevent disruptions

title: Performance optimization

role: performance

text: Configure your deployment for optimal performance and reliability based on your workload requirements and infrastructure capabilities.


title: Manage node resources

title: Node eviction thresholds

text:
Kubernetes uses [node-pressure eviction](https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/) to terminate pods and free up node resources. Redis Enterprise for Kubernetes works best when you use the recommendations below.

Eviction thresholds are managed by [kubelet arguments](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/).

Redis recommends setting a high [soft eviction threshold](https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/#soft-eviction-thresholds). This changes the node condition early enough to alert the administrator.

We also recommend setting the [`eviction-soft-grace-period`](https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/#soft-eviction-thresholds) high enough for the administrator to scale the cluster.

For more information about configuring eviction thresholds on specific platforms, see [Freeing node resources](https://docs.openshift.com/container-platform/4.9/nodes/nodes/nodes-nodes-garbage-collection.html) for OpenShift or [Cluster architecture](https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-architecture#node_allocatable) for GKE.

title: Monitor node conditions

text:
Redis recommends monitoring the node conditions. If both `MemoryPressure` and `DiskPressure` are true, the eviction threshold was met.

[code example]

For more information about monitoring node conditions, see [Node conditions](https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/#node-conditions) on [kubernetes.io](https://kubernetes.io/docs/home/).

title: Resource quotas

text:
Kubernetes uses the `ResourceQuota` object to limit resource consumption per namespace. This lets you limit the number of objects created by a namespace or the amount of compute resources consumed by a namespace.

The resource settings for Redis Enterprise for Kubernetes are defined in the `operator.yaml` and the [`RedisEnterpriseCluster`](https://github.com/RedisLabs/redis-enterprise-k8s-docs/blob/master/redis_enterprise_cluster_api.md#redisenterpriseclusterspec) custom resource.

The following settings are the minimum workloads for the operator to function.

[code example]
For more details on using resource quotas, see the [Kubernetes documentation](https://kubernetes.io/docs/concepts/policy/resource-quotas/).

code:
> kubectl get nodes -o jsonpath='{range .items[*]}name:{.metadata.name}{"\t"}MemoryPressure:{.status.conditions[?(@.type == "MemoryPressure")].status}{"\t"}DiskPressure:{.status.conditions[?(@.type == "DiskPressure")].status}{"\n"}{end}'
name:gke-55d1ac88-213c	MemoryPressure:False	DiskPressure:False
name:gke-55d1ac88-vrpp	MemoryPressure:False	DiskPressure:False
name:gke-7253cc19-42g0	MemoryPressure:False	DiskPressure:False

section_id: monitor-node-conditions

code:
resources:
    limits:
      cpu: 0.5
      memory: 256Mi
    requests:
      cpu: 0.5
      memory: 256Mi

section_id: resource-quotas


title: Control node selection

title: Overview

text:
Kubernetes clusters often include nodes with different CPU and memory profiles. You control where Redis Enterprise cluster (REC) pods run by setting fields in the REC custom resource (CRD).

A Redis Enterprise cluster (REC) runs as a StatefulSet. The Kubernetes scheduler assigns nodes when you create or resize the cluster, or when a pod restarts.

Use these options to control pod placement:

title: Use node selectors

text:
The [`nodeSelector`]() field matches the Kubernetes [`nodeSelector`](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector) syntax.
Label the nodes you want to target. For example, if nodes 'n1' and 'n2' are labeled with `memory=high`:

[code example]

The Redis Enterprise cluster CRD can request to be scheduled on these nodes:

[code example]

The operator copies [`nodeSelector`]() into the pod spec. The scheduler places pods only on nodes that match the selector.

title: Use node pools

text:
Node pools group similar nodes. Providers label nodes by pool.

Use [`nodeSelector`]() to target a pool by label. For example, on GKE:

[code example]

title: Provider resources

text:
Cloud providers label nodes by pool. See links below for specific documentation.

- GKE:
  - [Create and manage cluster and node pool labels](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-managing-labels)
  - [Update node labels and taints for existing node pools](https://cloud.google.com/kubernetes-engine/docs/how-to/update-existing-nodepools)
- AKS:
  - [Use labels in an AKS cluster](https://learn.microsoft.com/en-us/azure/aks/use-labels)
  - [Manage node pools in AKS](https://learn.microsoft.com/en-us/azure/aks/manage-node-pools)
- EKS:
  - [Create a managed node group with labels (AWS CLI)](https://docs.aws.amazon.com/cli/latest/reference/eks/create-nodegroup.html)
  - [Update a managed node group to add labels (AWS CLI)](https://docs.aws.amazon.com/cli/latest/reference/eks/update-nodegroup-config.html)

title: Use node taints

text:
Use node taints and pod tolerations to control REC pod scheduling. Set tolerations with [`spec.podTolerations`]() (standard [Kubernetes tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#concepts)).

Example: on a single node pool, reserve nodes n1–n3 for REC by adding taints:

[code example]

This blocks pods unless they tolerate the `db=rec` taint.

Then add a matching toleration to the REC:

[code example]

A set of taints can also handle more complex use cases.
For example, a `role=test` or `role=dev` taint can be used to designate a node as dedicated for testing or development workloads via pod tolerations.

title: Use pod anti-affinity

text:
By default, REC node pods are not scheduled on the same node within the same cluster:

[code example]

Each pod has these labels. `redis.io/cluster` is your cluster name.

Modify this rule to widen or narrow placement. For example, remove the `redis.io/cluster` label to prevent pods from different clusters from sharing a node:

[code example]

To avoid co-locating with other database workloads, label those pods `local/role: database` and add anti-affinity to keep one database per node:

[code example]

Kubernetes will not schedule two pods with label `local/role: database` on the same node.

title: Enable rack awareness

text:
Enable rack-zone awareness to improve availability during rack or zone failures.

When creating your rack-zone ID, there are some constraints to consider; see [rack-zone awareness]() for more info.

Configure it with [`spec.rackAwarenessNodeLabel`]() in the REC.

title: Choose a node label

text:
The most common label used for rack-zone awareness is topology.kubernetes.io/zone, a standard Kubernetes label that shows the zone a node runs in. Many Kubernetes platforms add this label to nodes by default, as noted in the [Kubernetes documentation](https://kubernetes.io/docs/setup/best-practices/multiple-zones/#nodes-are-labeled).

If your platform doesn’t set this label automatically, you can use any custom label that describes the node’s topology (such as rack, zone, or region).

title: Label all eligible nodes

text:
All eligible nodes **must** have the label for rack awareness to work. The operator requires every node that might run Redis Enterprise pods to be labeled. If any nodes are missing the label, reconciliation fails.

Eligible nodes are nodes where REC pods can run. By default, this means all worker nodes. You can limit eligibility with [`spec.nodeSelector`]().

Give each eligible node a label value that reflects its rack, zone, or region.

Check node label values:

[code example]

title: Enable the cluster role

text:
Grant the operator read access to node labels with a ClusterRole and ClusterRoleBinding.

ClusterRole:

Bind to the `redis-enterprise-operator` service account:

Apply these files with `kubectl apply`. For example:

[code example]

After you apply the role and binding, you can configure rack awareness.

title: Configure rack awareness

text:
Set [`spec.rackAwarenessNodeLabel`]() to the node label to use:

When you set [`spec.rackAwarenessNodeLabel`](), the operator sets the anti-affinity `topologyKey` to that label unless you define [`spec.podAntiAffinity`](). If you define both, make sure `topologyKey` matches your node label.

title: Rack awareness limitations

text:
**Pod restart distribution maintenance**: When rack awareness is enabled, node pods and shards are initially deployed based on rack constraints to ensure proper distribution across zones. However, Redis Enterprise does not automatically maintain this distribution when node pods are restarted.

After pod restarts, the rack awareness policy may be violated, requiring manual intervention to restore proper shard distribution. While Redis Enterprise provides tools to identify shards that need to be moved to restore correct rack distribution, it does not provide automated orchestration to perform these moves.

**Important considerations for production deployments:**

- At scale, manual shard redistribution can become operationally challenging
- This limitation is particularly important for edge deployments where automated recovery is preferred
- Plan for operational procedures to handle rack awareness policy violations after pod restarts
- Consider monitoring tools to detect when rack awareness constraints are violated

code:
kubectl label nodes n1 memory=high
kubectl label nodes n2 memory=high

section_id: use-node-selectors

code:
apiVersion: app.redislabs.com/v1
kind: RedisEnterpriseCluster
metadata:
  name: rec
spec:
  nodes: 3
  nodeSelector:
     memory: high

code:
apiVersion: app.redislabs.com/v1
kind: RedisEnterpriseCluster
metadata:
  name: rec
spec:
  nodes: 3
  nodeSelector:
     cloud.google.com/gke-nodepool: 'high-memory'

section_id: use-node-pools

code:
kubectl taint nodes n1 db=rec:NoSchedule
kubectl taint nodes n2 db=rec:NoSchedule
kubectl taint nodes n3 db=rec:NoSchedule

section_id: use-node-taints

code:
apiVersion: app.redislabs.com/v1
kind: RedisEnterpriseCluster
metadata:
  name: rec
spec:
  nodes: 3
  podTolerations:
  - key: db
    operator: Equal
    value: rec
    effect: NoSchedule

code:
podAntiAffinity:
  requiredDuringSchedulingIgnoredDuringExecution:
  - labelSelector:
      matchLabels:
        app: redis-enterprise
        redis.io/cluster: rec
        redis.io/role: node
    topologyKey: kubernetes.io/hostname

section_id: use-pod-anti-affinity

code:
apiVersion: app.redislabs.com/v1
kind: RedisEnterpriseCluster
metadata:
  name: rec
spec:
  nodes: 3
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchLabels:
          app: redis-enterprise
          redis.io/role: node
      topologyKey: kubernetes.io/hostname

code:
apiVersion: app.redislabs.com/v1
kind: RedisEnterpriseCluster
metadata:
  name: rec
spec:
  nodes: 3
  extraLabels:
     local/role: database
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchLabels:
          local/role: database
          app: redis-enterprise
          redis.io/cluster: rec
          redis.io/role: node
      topologyKey: kubernetes.io/hostname

code:
kubectl get nodes -o custom-columns="name:metadata.name","rack\\zone:metadata.labels.topology\.kubernetes\.io/zone"

name                                            rack\zone
ip-10-0-x-a.eu-central-1.compute.internal    eu-central-1a
ip-10-0-x-b.eu-central-1.compute.internal    eu-central-1a
ip-10-0-x-c.eu-central-1.compute.internal    eu-central-1b
ip-10-0-x-d.eu-central-1.compute.internal    eu-central-1b

section_id: label-all-eligible-nodes

code:
kubectl apply -f rack-aware-cluster-role.yaml
kubectl apply -f rack-aware-cluster-role-binding.yaml

section_id: enable-the-cluster-role


title: Storage types

text:
Redis Enterprise for Kubernetes can only use storage classes supported by block storage. Block storage is mounted at the Kubernetes node level and utilizes EXT4 or XFS file systems. It can be sourced from enterprise-grade SANs or cloud environments such as EBS, Azure Managed Disks, or GCP persistent disks.

NFS, NFS-like, and multi-read-write/shared storage options are not supported. These types of storage are often slow and can cause locking behaviors that are incompatible with the requirements of database storage.

title: REC `persistentSpec` field

text:
To deploy a Redis Enterprise cluster with Redis Enterprise operator the
spec should include a *persistentSpec* section, in the
*redis-enterprise-cluster.yaml* file:

    spec:
      nodes: 3
      persistentSpec:
       enabled: true
       storageClassName: "standard"
       volumeSize: "23Gi" #optional

Persistence storage is a requirement for production deployments.

For **production deployments** of Redis Enterprise Cluster on Kubernetes,
the Redis Enterprise Cluster (REC) must be deployed with persistence enabled.
The REC deployment files in the [Kubernetes documentation](https://github.com/RedisLabs/redis-enterprise-k8s-docs) contain this declaration by default.

title: Volume size

text:
*volumeSize* is an optional definition. By default, if the definition is
omitted, operator allocates five times (5x) the amount of memory (RAM)
defined for nodes (see example below), which is the recommended
persistent storage size as described in the [Hardware
requirements]() article.

To explicitly specify the persistent storage size, use the *volumeSize*
property as described in the example above.

Persistent volume claims can be expanded, but not reduced after creation. See [Expand PersistentVolumeClaim (PVC)]() for details.

We recommend that you omit the volumeSize definition from the REC declaration
so that the Redis Enterprise Cluster deployment on Kubernetes use the default volume size.

title: Storage class name

text:
*storageClassName* determines the Storage Class resource, which is
defined by the Kubernetes cluster administrator, to be used for
persistent storage.

Different Kubernetes distributions and different deployments use
different Storage Class resources.

In order to determine the Storage Class resources available for your K8s
deployment, use the following command:

    kubectl get StorageClass

Typically, AWS provides "gp2" as the Storage Class name while GKE uses "standard."
Azure provides two Storage Classes: "default" using HDDs, and "managed-premium" using SSDs.

Below is an example of a response to the command.

|                         |                                                         |
| ----------------------- | ------------------------------------------------------- |
| *Name:*                 | *gp2*                                                   |
| *IsDefaultClass:*       | *Yes*                                                   |
| *Annotations:*          | *storageclass.beta.kubernetes.io/is-default-class=true* |
| *Provisioner:*          | *kubernetes.io/aws-ebs*                                 |
| *Parameters:*           | *encrypted=false,kmsKeyId=,type=gp2*                    |
| *AllowVolumeExpansion:* | *\<unset\>*                                             |
| *MountOptions:*         | *\<none\>*                                              |
| *ReclaimPolicy:*        | *Delete*                                                |
| *VolumeBindingMode:*    | *Immediate*                                             |
| *Events:*               | *\<none\>*                                              |

storageClassName must be specified for this deployment type.

The storage class cannot be changed after deployment. Trying to change this value after deployment could result in unexpected and potentially damaging behavior.

Example of the redisEnterpriseNodeResources definition:

    redisEnterpriseNodeResources:
      limits:
        cpu: "4000m"
        memory: 4Gi
      requests:
        cpu: "4000m"
        memory: 4Gi


title: Manage pod stability

title: Overview

text:
Kubernetes clusters manage the allocation of system resources and can evict pods to release system resources.
Here are some ways that you can configure the Redis Enterprise node pods to maintain pod stability:

title: Guaranteed quality of service

text:
A running pod has a quality of service measure assigned to it that is
one of three [quality of service classes](https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/):
Guaranteed, Burstable, and Best Effort.
You can assure the Guaranteed class is assigned to the Redis Enterprise node pods
by following the right guidelines.

To get a Guaranteed quality of service class assigned:

 * Every container in the pod must have a memory limit and a memory request, and they must be the same.
 * Every container in the pod must have a CPU limit and a CPU request, and they must be the same.

If resources limits and requests are not specified in the Redis Enterprise CRD,
these requirements are met in the default version created by the operator. of your Redis Enterprise cluster CRD,
Otherwise, you must set the limits and requests to the same value for memory and CPU in the `redisEnterpriseNodeResources`
section of the CRD.

Sidecar containers also impact the quality of service class assignment for the pod.

To check the quality of service class of any running Redis Enterprise node pod, run:

[code example]

where `rec-0` is the name of one of the pods associated with the Redis Enterprise cluster.

title: Using priority to protect from preemption

text:
When a Redis Enterprise node pod is scheduled, it can be assigned a
[priority class](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/)
with the `priorityClassName` property. This property value is the name of
a priority class that must already exist within the cluster.

A sufficiently high priority will prevent other workloads with a lower
priority from preempting the scheduling of Redis Enterprise Nodes. Similarly,
a high value may also prevent eviction when lower priority workloads
are deployed on the same cluster.

The successful use of this strategy involves first creating a priority class with a very large priority value:

[code example]

Then, you refer to the priority class by name in your Redis Enterprise cluster CRD:

[code example]

Alternatively, you can also [disable preemption entirely](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#how-to-disable-preemption).

title: Managing eviction thresholds

text:
Eviction thresholds are typically managed by kubelet arguments.
You can set the thresholds:

- On OpenShift - In the [config file](https://docs.openshift.com/container-platform/3.11/admin_guide/out_of_resource_handling.html#out-of-resource-create-config).
- On GKE - In the [managed settings](https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-architecture#node_allocatable).

We recommend that you:

 * Set the [soft eviction threshold](https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/#soft-eviction-thresholds)
   to be higher than the [hard eviction threshold](https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/#hard-eviction-thresholds).
   The high soft threshold makes the node condition change earlier, and alerts the administrator.
 * Set `eviction-max-pod-grace-period` high enough to allow the RS pods to migrate the Redis databases before the pods are force killed.
 * Set the `eviction-soft-grace-period` high enough that the administrator (or a k8s auto-scaling mechanism) scales k8s up or out.

title: Monitoring for memory and disk usage

text:
We recommend that you monitor the node for MemoryPressure and DiskPressure.
When both of these conditions are true, then an [eviction threshold](https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/#eviction-thresholds)
is met and the pod is evicted.

To retrieve the flags, run the command:

[code example]

code: kubectl get pod rec-0 --o jsonpath="{.status.qosClass}"

section_id: guaranteed-quality-of-service

code:
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: redis-enterprise-priority
value: 1000000000
globalDefault: false
description: "This priority class should be used for Redis Enterprise pods only."

code:
apiVersion: app.redislabs.com/v1
kind: RedisEnterpriseCluster
metadata:
  name: example-redisenterprisecluster
spec:
  size: 3
  priorityClassName: "redis-enterprise-priority"

code:
$kubectl get nodes -o jsonpath='{range .items[*]}name:{.metadata.name}{"\t"}MemoryPressure:{.status.conditions[?(@.type == "MemoryPressure")].status}{"\t"}DiskPressure:{.status.conditions[?(@.type == "DiskPressure")].status}{"\n"}{end}'

name:gke-55d1ac88-213c	MemoryPressure:False	DiskPressure:False
name:gke-55d1ac88-vrpp	MemoryPressure:False	DiskPressure:False
name:gke-7253cc19-42g0	MemoryPressure:False	DiskPressure:False

section_id: monitoring-for-memory-and-disk-usage