Upgrade Redis Enterprise with OpenShift OperatorHub
This task describes how to upgrade a Redis Enterprise cluster via OpenShift OperatorHub.
Redis Enterprise for Kubernetes |
---|
Redis implements rolling updates for software upgrades in Kubernetes deployments. The upgrade process includes updating three components:
- Upgrade the Redis Enterprise operator
- Upgrade the Redis Enterprise cluster (REC)
- Upgrade Redis Enterprise databases (REDB)
Prerequisites
The following steps ensure you have the minimum versions of all components necessary to upgrade. Without these minimum versions, the upgrade will freeze and require manual recovery.
See the troubleshooting section for details on recovering a failed upgrade.
Kubernetes version
Check Supported Kubernetes distributions to make sure your Kubernetes distribution is supported. If not, upgrade your Kubernetes distribution before upgrading the Redis operator.
Redis operator version
Your Redis Enterprise clusters must be running version 7.4.2-2 or later before upgrading to 7.8.2-6. See the 7.4 upgrade for detailed steps.
Redis database version
Your Redis databases must be running version 7.2 or later before upgrading your cluster version to 7.8.2-6. See upgrade databases for detailed steps. You can find your database version in the REDB spec.redisVersion
field.
RHEL9-compatible modules
Upgrading to Redis operator version 7.8.2-6 or later involves migrating your Redis Enterprise nodes to RHEL9 from either Ubuntu 18 or RHEL8. If your databases use modules, you need to manually install modules compatible with RHEL9.
To see which modules you have installed, run:
curl -k -u <rec_username>:<rec_password> -X GET https://localhost:9443/v1/modules | jq -r 'map([.module_name, .semantic_version, (.platforms | keys)]) | .[] | .[0] as $name | .[1] as $version | .[2][] | $name + "-" + $version + "-" + .' | sort
To see which modules are currently in use, run:
curl -k -u <rec_username>:<rec_password> -X GET https://localhost:9443/v1/bdbs | jq -r '.[].module_list | map(.module_name + "-" + .semantic_version) | .[]'
See Upgrade modules for details on how to upgrade modules with the rladmin
tool.
Valid license
Use kubectl get rec
and verify the LICENSE STATE
is valid on your REC before you start the upgrade process.
Upgrade the Redis Enterprise operator
-
Select the Redis Enterprise Operator from the Operators>Installed Operators page.
-
Select the Subscription tab.
-
Verify your Update approval is set to "Manual".
-
To upgrade to the latest version, set your Update Channel to ‘production’. To upgrade to an older version, specify the desired version for your Update Channel.
-
Select Upgrade available shown under Upgrade status.
-
When the "Review manual InstallPlan" section appears, select Preview installPlan and then Approve after reviewing the details. This will start the operator upgrade.
You can monitor the upgrade from the Installed Operators page. A new Redis Enterprise Operator will appear in the list, with the status "Installing". OpenShift will delete the old operator, showing the "Cannot update" status during deletion.
Reapply the SCC
If you are using OpenShift, you must manually reappply the security context constraints (SCC) file (scc.yaml
) and bind it to your service account.
oc apply -f openshift/scc.yaml
oc adm policy add-scc-to-user redis-enterprise-scc-v2 \
system:serviceaccount:<my-project>:<rec-name>
Upgrade the Redis Enterprise Cluster
Verify your license is valid before upgrading. Invalid licenses will cause the upgrade to fail.
Use oc get rec
and verify the LICENSE STATE
is valid on your REC before you start the upgrade process.
The Redis Enterprise cluster (REC) can be updated automatically or manually. To trigger automatic upgrade of the REC after the operator upgrade completes, specify autoUpgradeRedisEnterprise: true
in your REC spec. If you don't have automatic upgrade enabled, follow the below steps for the manual upgrade.
Before beginning the upgrade of the Redis Enterprise cluster, check the K8s operator release notes to find the Redis Enterprise image tag.
After the operator upgrade is complete, you can upgrade Redis Enterprise cluster (REC).
Edit redisEnterpriseImageSpec
-
Edit the REC custom resource YAML file.
oc edit rec <your-rec.yaml>
-
Replace the
versionTag:
declaration underredisEnterpriseImageSpec
with the new version tag.spec: redisEnterpriseImageSpec: imagePullPolicy: IfNotPresent repository: redislabs/redis versionTag: <new-version-tag>
-
Save the changes to apply.
Reapply roles and role bindings
If your operator is monitoring multiple namespaces, you'll need to reapply your role and role bindings for each managed namespace. See Manage databases in multiple namespaces for more details.
Monitor the upgrade
You can view the state of the REC with oc get rec
.
During the upgrade, the state should be Upgrade
.
When the upgrade is complete and the cluster is ready to use, the state will change to Running
.
If the state is InvalidUpgrade
, there is an error (usually relating to configuration) in the upgrade.
$ oc get rec
NAME NODES VERSION STATE SPEC STATUS LICENSE STATE SHARDS LIMIT LICENSE EXPIRATION DATE AGE
rec 3 6.2.10-107 Upgrade Valid Valid 4 2022-07-16T13:59:00Z 92m
To see the status of the current rolling upgrade, run:
oc rollout status sts <REC_name>
Upgrade databases
After the cluster is upgraded, you can upgrade your databases. Specify your new database version in the spec.redisVersion
field for your REDB and REAADB custom resources. Supported database versions for this operator version include "7.2"
and "7.4"
(note this value is a string).
Note that if your cluster redisUpgradePolicy
or your database redisVersion
are set to major
, you won't be able to upgrade those databases to minor versions. See Redis upgrade policy for more details.
The Redis Enterprise cluster (REC) can be updated automatically or manually. To trigger automatic upgrade of the REC after the operator upgrade completes, specify autoUpgradeRedisEnterprise: true
in your REC spec. If you don't have automatic upgrade enabled, follow the below steps for the manual upgrade.
Troubleshooting
If you start an upgrade without meeting the prerequisites, the operator will freeze the upgrade. Check the operator logs for the source of the error. The REDB reconsilliation doesn't work during an upgrade, so you need to apply a manual fix with the Redis Software API (examples below). The updates will also need to be added to the REDB custom resource.
Invalid module version
If the operator logs show an event related to an unsupported module, download the updated module locally, and install it using the v2/modules
API endpoint.
curl -sfk -u <rec_username>:<rec_password> -X POST -F 'module=@<full path to your module>' https://localhost:9443/v2/modules
After updating the modules with the Redis Software API, update the REDB custom resource to reflect the change.
Invalid database version
If the operator logs show an event related to an incompatible database version, upgrade the database using the Redis Software API.
curl -sfk -u <rec_username>:<rec_password> -X POST -H "Content-Type: application/json" -d '{"redis_version": <target redis version>}' https://localhost:9443/v1/bdbs/<BDB UID>/upgrade
After updating the database with the Redis Software API, update the REDB custom resource to reflect the change.