# Stay up when a region goes down: Highly available Redis for Python apps **Tagline:** News & Media | **Authors:** Vladyslav Vildanov | **Categories:** Tech | **Published:** 2026-09-23 | **Updated:** 2026-09-23 ## Active-Active Redis & client-side geographic failover Active-Active Redis distributes data across multiple regions, allowing each regional database instance to serve both reads and writes. What kind of magic allows that? Redis uses conflict-free replicated data types ([CRDTs](https://redis.io/blog/diving-into-crdts/)) to resolve concurrent updates and ensure that the instances eventually converge to a consistent state. ![Stay Up When a Region Goes Down: Highly Available Redis for Python Applications](https://cdn.sanity.io/images/sy1jschh/production/199968a3c6b270a810ddb47a7065a3aa00152270-1549x900.png) A standard redis-py client connects to a single configured endpoint. In order to be able to quickly fail over between instances, in case of a failure, an application could create separate clients for multiple regional database instances, but it must then implement (and maintain!) health monitoring, endpoint selection, failover, and failback. Having countless applications around the world implementing the same logic, some with more success than others, doesn’t really make a lot of engineering sense, so we decided to come up with a single “canonical” implementation which provides a client API that manages these responsibilities, thereby enabling client-side geographic failover. This API is exposed on `MultiDBClient` - a wrapper over the regular single and cluster client instances. The `MultiDBClient` routes traffic to one selected endpoint (active database) - while monitoring the health of all configured endpoints. If the active database is considered unhealthy, the client selects another healthy endpoint according to the configured weights and redirects traffic to it. When automatic failback is enabled, the client periodically evaluates the unavailable endpoints and can return to the highest-weighted healthy endpoint. `MultiDBClient` does not broadcast commands or replicate data across the endpoints. Data replication remains the responsibility of the Active-Active database layer letting the CRDTs shine. ```python from redis.multidb.client import MultiDBClient from redis.multidb.config import DatabaseConfig, MultiDbConfig config = MultiDbConfig( databases_config=[ DatabaseConfig(from_url="redis://db1.example.com:6379", weight=1.0), DatabaseConfig(from_url="redis://db2.example.com:6379", weight=0.5), ], # Auto fallback enabled by default. Set negative interval to disable it. auto_fallback_interval=120, ) client = MultiDBClient(config) client.set("key", "value") ``` ## Inside `MultiDBClient` Each `DatabaseConfig` defines one endpoint and its weight. Create one for every regional database instance. `MultiDbConfig` collects these endpoint configurations and defines behavior that applies to the overall `MultiDBClient` setup, including health checks, retries, failover, and failback. As we’ll see later, there are plenty of knobs to allow for different scenarios and setups. `MultiDBClient` delegates endpoint communication and connection management to an underlying Redis or RedisCluster client. You can select tone or the other globally with `MultiDbConfig.client_class`: use Redis for standard endpoints and RedisCluster for endpoints exposing the OSS Cluster API. Pass endpoint-specific client options through `DatabaseConfig.client_kwargs`. ```python config = MultiDbConfig( client_class=RedisCluster, databases_config=[ DatabaseConfig( weight=1.0, client_kwargs={ "host": "cluster-east.example.com", "port": 6379, "username": "app-user", "password": "secret", "read_from_replicas": True, # cluster-specific option }, ), DatabaseConfig( weight=0.5, client_kwargs={ "host": "cluster-west.example.com", "port": 6379, "username": "app-user", "password": "secret", "read_from_replicas": True, }, ), ], ) ``` ## From failure detection to recovery The `MultiDBClient` uses a circuit-breaker pattern to control traffic to each endpoint. When the active endpoint is considered unhealthy, its circuit opens and the client fails over to the highest-weighted healthy endpoint. The concept comes from electric circles, where circuit breaker elements protect an electric circle from the damage of the excess of what the equipment can actually carry. The circuit breaker is only one of two complementary mechanisms that detect failures. The proactive background health check periodically evaluates every endpoint using the configured health-check policy. The reactive `FailureDetector` we described above, observes command successes and failures within a sliding window and opens the circuit when the configured failure thresholds are reached. Together, they allow the client to detect and respond to failures more quickly. When the active endpoint’s circuit opens, the failover strategy selects the highest-weighted healthy endpoint and routes subsequent commands to it. It’s worth noting that commands already in flight against the previous endpoint cannot be redirected. If an in-flight write succeeds there, its result might not be immediately visible through the new endpoint until Active-Active replication catches up. Eligible command failures are handled by the global retry policy. Before each retry, `MultiDBClient` checks the currently active endpoint, allowing the command to be retried against the newly selected database. Automatic failback periodically checks whether a higher-weighted endpoint has recovered. Once the endpoint is healthy, `MultiDBClient` can switch traffic back to it. Weights can be configured to prioritize the endpoint closest to the application. This system would allow you to always prefer endpoints closest to the application, while being prepared for outages. For more control, you can disable automatic failback by setting `auto_fallback_interval` to `-1` and dynamically selecting a healthy endpoint explicitly with `set_active_database()`. ```python config = MultiDbConfig( ... # Negative value disables automatic fallback — the client stays on # whatever database is active until you switch it manually. auto_fallback_interval=-1, ) client = MultiDBClient(config) client.set("key", "value") # served by db1 (highest weight) # Manual failover: promote db2 explicitly. # get_databases() returns (database, weight) pairs sorted by weight. databases = client.get_databases() db2, _ = databases.get_top_n(2)[1] # second-highest weight = db2 client.set_active_database(db2) client.set("key", "value") # now served by db2 ``` ## Configuring highly-available Python client ### Health-check policies A health check can run several probes before deciding whether an endpoint is healthy. The policy determines how their results are combined, balancing fast failover against tolerance for transient failures: - **HEALTHY_ALL - strict:** The endpoint is healthy only when every probe succeeds. Use it when continuing to send traffic to an unstable endpoint is riskier than an occasional unnecessary failover. Its downside is low tolerance for transient network errors and a longer evaluation time. - **HEALTHY_ANY - permissive:** The endpoint remains healthy when at least one probe succeeds, and probing stops after the first success. Use it when brief connection failures are expected or failover is expensive. It minimizes false-positive failovers but may keep traffic on a degraded endpoint and takes all configured probes to confirm a complete outage. - **HEALTHY_MAJORITY - balanced:** More than half of the probes must succeed. It tolerates occasional failures while still responding to persistent problems. This is the best starting point for most production deployments. In short: choose HEALTHY_ALL to prioritize endpoint quality, HEALTHY_ANY to prioritize stability and avoid unnecessary switching, or HEALTHY_MAJORITY when you need a balance between the two. For example, the following configuration runs five probes and considers an endpoint healthy when at least three succeed: ```python config = MultiDbConfig( databases_config=db_configs, health_check_interval=5.0, health_check_probes=5, health_check_policy=HealthCheckPolicies.HEALTHY_MAJORITY, ) ``` ### How the settings affect performance **Health-check interval:** Frequent checks improve failure detection when application traffic is absent, but every application instance runs them, which sometimes can be too wasteful. For high-traffic systems, let the reactive detector lead and use health checks as a slower safety net. For low-traffic systems, use more frequent health checks because the reactive detector may not receive enough commands. **Failure-detection window:** A short window reacts quickly and forgets old failures sooner, which suits high traffic. Low-traffic applications need a longer window; otherwise they may never collect enough samples to reach min_num_failures. **Failover attempts:** failover_attempts × failover_delay defines the approximate recovery window when every endpoint is temporarily unavailable. A latency-sensitive API should keep this window short and return control to the application. A background worker can wait longer for an endpoint to recover. ### High-throughput scenario ```python from redis.backoff import ExponentialWithJitterBackoff from redis.retry import Retry from redis.asyncio.multidb.healthcheck import HealthCheckPolicies from redis.multidb.config import DatabaseConfig, MultiDbConfig db_configs = [ DatabaseConfig( client_kwargs={ "host": "redis-east.example.com", "port": 14000, "socket_connect_timeout": 0.3, "socket_timeout": 0.75, }, weight=1.0, grace_period=30.0, ), DatabaseConfig( client_kwargs={ "host": "redis-west.example.com", "port": 14000, "socket_connect_timeout": 0.3, "socket_timeout": 0.75, }, weight=0.5, grace_period=30.0, ), ] config = MultiDbConfig( databases_config=db_configs, command_retry=Retry( retries=1, backoff=ExponentialWithJitterBackoff(base=0.02, cap=0.1), ), failures_detection_window=2.0, min_num_failures=200, failure_rate_threshold=0.10, health_check_interval=5.0, health_check_probes=3, health_check_timeout=2.0, health_check_policy=HealthCheckPolicies.HEALTHY_MAJORITY, failover_attempts=3, failover_delay=0.5, auto_fallback_interval=60.0, ) ``` This preset favors throughput and fast failure detection: - Short timeouts prevent requests from occupying connections for too long. - One retry limits traffic amplification during an outage. - Jitter prevents all application instances from retrying simultaneously. - The short failure-detection window reacts quickly to concentrated failures. - A longer health-check interval limits background traffic. ## Observability and application integration `MultiDBClient` provides runtime notifications whenever failover or failback changes the active database. Applications can use custom event listeners to record the transition, update metrics, trigger alerts, or synchronize external state. These listeners are registered through an `EventDispatcher` supplied to `MultiDbConfig`. ```python from redis.event import EventDispatcher, EventListenerInterface from redis.multidb.client import MultiDBClient from redis.multidb.config import DatabaseConfig, MultiDbConfig from redis.multidb.event import ActiveDatabaseChanged class LogDatabaseSwitch(EventListenerInterface): def listen(self, event: ActiveDatabaseChanged) -> None: print( f"Active database changed: " f"{event.old_database.client.get_connection_kwargs()['host']} -> " f"{event.new_database.client.get_connection_kwargs()['host']}" ) event_dispatcher = EventDispatcher() event_dispatcher.register_listeners({ActiveDatabaseChanged: [LogDatabaseSwitch()]}) config = MultiDbConfig( databases_config=[ DatabaseConfig(from_url="redis://db1.example.com:6379", weight=1.0), DatabaseConfig(from_url="redis://db2.example.com:6379", weight=0.5), ], event_dispatcher=event_dispatcher, ) client = MultiDBClient(config) ``` When redis-py observability is enabled, `MultiDBClient` records the `redis.client.geofailover.failovers` counter. It includes the attributes: - `db.client.geofailover.fail_from` - `db.client.geofailover.fail_to` - `db.client.geofailover.reason`, such as `automatic` or `manual` ```python from redis.observability.config import MetricGroup, OTelConfig from redis.observability.providers import get_observability_instance otel = get_observability_instance() otel.init( OTelConfig( metric_groups=[ MetricGroup.RESILIENCY, MetricGroup.COMMAND, ] ) ) ``` ## Regional failover demonstration To see how `MultiDBClient` handles regional failover and failback end to end, explore the [redis-py geographic failover example](https://github.com/redis-developer/redis-py-geo-failover-example). ## Conclusion Active-Active Redis takes care of syncing your data across regions, while `MultiDBClient` helps your Python application stay connected. It brings health monitoring, weighted endpoint selection, failover and failback, retries, and observability together behind a familiar Redis client API. There’s no one-size-fits-all configuration for high availability. The defaults offer a balanced starting point, but you should still tune them for your workload, expected failover speed, and tolerance for false positives. So, when a region goes down, your Python application has a clear path to keep running.