How to Recover a Cluster of a CRDB Topology?

Redis Enterprise Software
crdb
recovery

Last updated 18, Apr 2024

Question

  • What are the best practices for rebuilding a cluster that is part of a CRDB mesh? e.g. sync from other clusters (s), recover from RDB/AOF, etc.
  • If a cluster loses quorum should it be rejoined to the CRDB mesh or recover from RDB independently?
  • Any additional consideration if a cluster has both CRDB and non-CRDB databases?

Answer

If you have a live/healthy instance in the CRDB and only need to do cluster recovery on the bad cluster, you should recover the DB with rladmin recover db <db_name> only_configuration and let the data sync from the good instance.

If all instances in the CRDB require recovery, you should recover one instance with data and only recover the configuration for the other instances. This is mentioned in our Recover a DB doc:

For Active-Active databases that still have live instances, we recommend that you recover the configuration for the failed instances and let the data update from the other instances.

No side effects are expected when the cluster has CRDB and standalone DBs.