{ "id": "redis-py", "title": "Redis pub/sub with redis-py", "url": "https://redis.io/docs/latest/develop/use-cases/pub-sub/redis-py/", "summary": "Implement Redis pub/sub messaging in Python with redis-py", "tags": [ "docs", "develop", "stack", "oss", "rs", "rc" ], "last_updated": "2026-05-14T08:58:05-05:00", "children": [], "page_type": "content", "content_hash": "491db2923b9b9be5bb43ba84c3fbe6cd5037e1758a3a8e7a967c50b5a6323d9e", "sections": [ { "id": "overview", "title": "Overview", "role": "overview", "text": "This guide shows you how to implement a Redis-backed pub/sub broadcaster in Python with [`redis-py`](https://redis.io/docs/latest/develop/clients/redis-py). It includes a small local web server built with the Python standard library so you can publish messages to named channels, add and remove subscribers live, and watch Redis fan out each message to every interested listener." }, { "id": "overview", "title": "Overview", "role": "overview", "text": "Pub/sub lets your application broadcast events — chat messages, cache invalidation signals, presence updates, notifications — to many consumers without per-pair wiring. The publisher names a *channel*; every client currently subscribed to that channel receives the message, in publish order, with sub-millisecond fan-out.\n\nThat gives you:\n\n* Many-to-many event delivery with no message storage cost in Redis\n* Exact-match subscriptions (`SUBSCRIBE orders:new`) for known topics\n* Pattern subscriptions (`PSUBSCRIBE notifications:*`) for whole topic hierarchies\n* Live server-side introspection through `PUBSUB CHANNELS`, `PUBSUB NUMSUB`, and `PUBSUB NUMPAT`\n* At-most-once delivery: subscribers that are offline when a message is published miss it, so durable state should live in keys or a Stream, not in the pub/sub channel itself\n\nIn this example, the publisher side calls `PUBLISH` with a JSON-encoded body and counts how many subscribers Redis reported delivering to. Each in-process subscriber owns its own Redis connection and a background thread that pumps incoming messages into a callback." }, { "id": "how-it-works", "title": "How it works", "role": "content", "text": "The flow looks like this:\n\n1. The application calls `hub.subscribe(name, channels)` or `hub.psubscribe(name, patterns)`\n2. The helper creates a redis-py `PubSub` object, binds each target to a dispatch callback, and starts a background thread on `run_in_thread()`\n3. The application (or another process) calls `hub.publish(channel, message)`\n4. Redis fans the message out over every subscribing client's open socket\n5. Each subscriber's dispatch thread wraps the raw message as a `ReceivedMessage`, appends it to a per-subscriber ring buffer, and invokes the optional callback\n6. The publisher receives the integer subscriber count back from `PUBLISH`, which is the number of clients Redis delivered to right then\n\nPattern subscriptions match channels by glob (`*`, `?`, `[abc]`). A single message that matches both an exact subscription and a pattern subscription is delivered twice — once as a `message` and once as a `pmessage`." }, { "id": "the-pub-sub-hub-helper", "title": "The pub/sub hub helper", "role": "content", "text": "The `RedisPubSubHub` class wraps the publish, subscribe, and introspection operations\n([source](https://github.com/redis/docs/blob/main/content/develop/use-cases/pub-sub/redis-py/pubsub_hub.py)):\n\n[code example]" }, { "id": "data-model", "title": "Data model", "role": "content", "text": "Pub/sub has no Redis keyspace footprint of its own — channels are server-side routing entries, not stored values. The hub keeps its own bookkeeping in process memory:\n\n[code example]\n\nThe implementation uses:\n\n* [`PUBLISH`](https://redis.io/docs/latest/commands/publish) to fan a JSON-encoded message out to every subscriber of a channel\n* [`SUBSCRIBE`](https://redis.io/docs/latest/commands/subscribe) for exact-match subscribers\n* [`PSUBSCRIBE`](https://redis.io/docs/latest/commands/psubscribe) for glob-style pattern subscribers\n* [`PUBSUB CHANNELS`](https://redis.io/docs/latest/commands/pubsub-channels) to list the channels with at least one active exact-match subscriber\n* [`PUBSUB NUMSUB`](https://redis.io/docs/latest/commands/pubsub-numsub) to count subscribers per channel\n* [`PUBSUB NUMPAT`](https://redis.io/docs/latest/commands/pubsub-numpat) to count active pattern subscriptions server-wide\n* The redis-py `pubsub.run_in_thread()` helper to dispatch messages from each subscriber's own connection without writing the thread loop by hand" }, { "id": "publishing-messages", "title": "Publishing messages", "role": "content", "text": "`publish()` JSON-encodes the message body, calls `PUBLISH`, and updates the per-channel publish counter:\n\n[code example]\n\nThe integer returned by `PUBLISH` is what Redis itself reports — the number of subscribers (direct and pattern) that received the message in that call. It's a useful sanity check that the channel name is actually being listened to: a steady stream of `0`s means you have a typo somewhere or your subscriber crashed." }, { "id": "subscribing-to-channels", "title": "Subscribing to channels", "role": "content", "text": "`subscribe()` creates a named in-process `Subscription` that owns its own `PubSub` object and dispatch thread:\n\n[code example]\n\nInside `Subscription.__init__`, the helper binds each target to a callback and starts a background thread:\n\n[code example]\n\nA few details matter here:\n\n* `pubsub(ignore_subscribe_messages=True)` skips the `subscribe`/`unsubscribe` acknowledgements Redis sends back on the same socket, so the dispatch callback only ever sees real published payloads.\n* Each `Subscription` gets its own `PubSub` (and therefore its own Redis connection). Sharing one connection across subscribers would couple their lifetimes — closing one would close the channel for the others.\n* `run_in_thread` runs a short poll loop (`sleep_time=0.01`) rather than blocking on `listen()`. That's the cleanest way to support `subscription.close()` without leaving a thread stuck inside a blocking socket read." }, { "id": "pattern-subscriptions-with-psubscribe", "title": "Pattern subscriptions with PSUBSCRIBE", "role": "content", "text": "`psubscribe()` works the same way but routes messages through `PSUBSCRIBE` so each binding is a glob, not a literal channel name:\n\n[code example]\n\nWhen a published channel matches a pattern, the dispatch callback receives both the matched channel and the original pattern:\n\n[code example]\n\nThat distinction is useful for routing: a pattern subscriber can do one thing for the whole hierarchy (such as increment a counter) and dispatch on the specific channel within its callback (such as \"invalidate this region's cache\")." }, { "id": "inspecting-active-subscribers", "title": "Inspecting active subscribers", "role": "content", "text": "Redis exposes a small set of pub/sub introspection commands that report on subscriber state without traversing any keyspace:\n\n[code example]\n\n`PUBSUB CHANNELS` only reports channels with at least one exact-match subscriber — pattern subscribers do not appear here. That's a deliberate Redis design choice: a glob like `*` would otherwise show up as a subscriber to every conceivable channel. `PUBSUB NUMPAT` covers the pattern side as a single global count." }, { "id": "stats-and-history", "title": "Stats and history", "role": "content", "text": "`stats()` reports publish and receive counters plus the size of the subscription registry:\n\n[code example]\n\n`delivered_total` is what Redis itself counted; `received_total` is what this process's in-memory subscribers saw. In a single-process demo they should track each other closely — a sustained divergence usually means a callback raised, or a subscriber crashed while a publisher kept publishing. (Pub/sub is at-most-once: if your subscriber wasn't connected at publish time, the message is gone.)" }, { "id": "prerequisites", "title": "Prerequisites", "role": "content", "text": "* Redis 6.2 or later running locally on the default port (6379). Earlier versions still work for plain `PUBLISH`/`SUBSCRIBE`; `PUBSUB NUMPAT` is older than that.\n* Python 3.9 or later.\n* The `redis-py` client. Install it with:\n\n [code example]" }, { "id": "running-the-demo", "title": "Running the demo", "role": "content", "text": "" }, { "id": "get-the-source-files", "title": "Get the source files", "role": "content", "text": "The demo consists of two Python files. Download them from the [`redis-py` source folder](https://github.com/redis/docs/tree/main/content/develop/use-cases/pub-sub/redis-py) on GitHub, or grab them with `curl`:\n\n[code example]" }, { "id": "start-the-demo-server", "title": "Start the demo server", "role": "content", "text": "From that directory:\n\n[code example]\n\nYou should see:\n\n[code example]\n\nOpen [http://127.0.0.1:8095](http://127.0.0.1:8095) in a browser. You can:\n\n* Publish messages of any text to any channel name in any batch size.\n* Add named subscribers that listen on either a specific channel (`orders:new`) or a glob pattern (`notifications:*`). A single subscriber can listen on multiple targets — enter them comma-separated.\n* Watch each subscriber's incoming-message panel update every 800 ms.\n* See the server-side view: `PUBSUB CHANNELS` lists exact-match channels with subscribers, `PUBSUB NUMSUB` gives per-channel counts, and `PUBSUB NUMPAT` counts active pattern subscriptions.\n* Click **Reset** to drop every subscription, zero the counters, and re-seed the three default subscribers.\n\nIf your Redis server is running elsewhere, start the demo with `--redis-host` and `--redis-port`." }, { "id": "production-usage", "title": "Production usage", "role": "content", "text": "" }, { "id": "pub-sub-is-at-most-once-pair-it-with-durable-state-if-you-need-replay", "title": "Pub/sub is at-most-once — pair it with durable state if you need replay", "role": "content", "text": "A subscriber that's offline when a message is published misses it permanently. For events you can't afford to lose, write the durable record (the order row, the cache key version, the audit log entry) to its primary store, then `PUBLISH` a notification so live consumers can pick it up immediately. On reconnect, consumers reconcile by reading the durable store, not by waiting for missed pub/sub messages. If you actually need replay or at-least-once delivery, switch to [Redis Streams](https://redis.io/docs/latest/develop/data-types/streams) with consumer groups." }, { "id": "use-a-separate-connection-or-pubsub-object-per-subscriber", "title": "Use a separate connection (or `PubSub` object) per subscriber", "role": "content", "text": "A redis-py `PubSub` puts its connection into subscribe-only mode: ordinary commands (`GET`, `HSET`, etc.) on the same connection will hang. Always create the `PubSub` from a client whose pool can spare a connection, or — as the helper does — give every subscriber its own `PubSub`. Sharing one `PubSub` across business-logic subscribers couples their lifetimes (closing one closes the channel for the others) and serialises their callbacks on a single dispatch thread." }, { "id": "choose-a-topic-naming-convention-up-front", "title": "Choose a topic naming convention up front", "role": "content", "text": "A flat namespace gets ugly fast — `email`, `email_high_priority`, `email_high_priority_billing`. Pick a colon-separated hierarchy (`notifications:billing:invoice`, `cache:invalidate:products:p-001`) so consumers can subscribe at the right level: a billing service uses `notifications:billing:*`, the audit logger uses `notifications:*`. Glob patterns are evaluated for every published message, so don't go wild with multiple wildcards on hot paths — `*:*:*` matches everything and costs more than a flat `notifications:*` would." }, { "id": "don-t-do-heavy-work-in-the-dispatch-callback", "title": "Don't do heavy work in the dispatch callback", "role": "content", "text": "The dispatch thread reads messages from a single socket. If the callback blocks (synchronous HTTP call, big computation, slow DB write), the next message waits behind it and the subscriber's effective throughput drops to whatever the callback's latency is. For heavier work, the callback should hand the message off to a worker pool, a queue, or — for true durable handoff — a [Redis Streams](https://redis.io/docs/latest/develop/data-types/streams) consumer group." }, { "id": "tune-the-subscriber-buffer-for-your-traffic-shape", "title": "Tune the subscriber buffer for your traffic shape", "role": "content", "text": "The demo caps each subscriber's in-memory message buffer at 50. That's right for showing the recent activity in a UI, but a real subscriber typically processes each message and discards it — the buffer is only there for human inspection. If you keep a buffer, make sure it's bounded; an unbounded ring on a chatty pattern subscriber will eventually OOM the process." }, { "id": "sharded-pub-sub-on-a-redis-cluster", "title": "Sharded pub/sub on a Redis Cluster", "role": "content", "text": "On a Redis Cluster, plain `PUBLISH` fans every message out to every node via the cluster bus, which becomes a hotspot at high throughput. Redis 7.0 added [sharded pub/sub](https://redis.io/docs/latest/develop/pubsub#sharded-pubsub): channels are hashed to slots, and `SPUBLISH` / `SSUBSCRIBE` only touch the shard that owns the slot. If you're scaling pub/sub on a cluster, prefer the sharded commands and pick channel names whose hash distribution matches your traffic." }, { "id": "inspect-pub-sub-state-directly-in-redis", "title": "Inspect pub/sub state directly in Redis", "role": "content", "text": "Because pub/sub has no keyspace, `KEYS`/`SCAN` won't show you anything. Use the introspection commands instead:\n\n[code example]\n\n`redis-cli` in subscribe mode only exits with `Ctrl-C` — it can't issue any other commands while subscribed." }, { "id": "learn-more", "title": "Learn more", "role": "related", "text": "This example uses the following Redis commands:\n\n* [`PUBLISH`](https://redis.io/docs/latest/commands/publish) to fan a message out to every subscriber of a channel.\n* [`SUBSCRIBE`](https://redis.io/docs/latest/commands/subscribe) and [`UNSUBSCRIBE`](https://redis.io/docs/latest/commands/unsubscribe) for exact-match topic subscriptions.\n* [`PSUBSCRIBE`](https://redis.io/docs/latest/commands/psubscribe) and [`PUNSUBSCRIBE`](https://redis.io/docs/latest/commands/punsubscribe) for glob-style pattern subscriptions.\n* [`PUBSUB CHANNELS`](https://redis.io/docs/latest/commands/pubsub-channels) to list channels with at least one active exact-match subscriber.\n* [`PUBSUB NUMSUB`](https://redis.io/docs/latest/commands/pubsub-numsub) to count subscribers per named channel.\n* [`PUBSUB NUMPAT`](https://redis.io/docs/latest/commands/pubsub-numpat) to count active pattern subscriptions server-wide.\n\nSee the [`redis-py` documentation](https://redis.io/docs/latest/develop/clients/redis-py) for full client reference, including the [`PubSub` class](https://redis.readthedocs.io/en/stable/advanced_features.html#publish-subscribe) and the `run_in_thread()` dispatch helper." } ], "examples": [ { "id": "the-pub-sub-hub-helper-ex0", "language": "python", "code": "import redis\nfrom pubsub_hub import RedisPubSubHub\n\nr = redis.Redis(host=\"localhost\", port=6379, decode_responses=True)\nhub = RedisPubSubHub(redis_client=r)\n\n# Exact-match subscriber\nhub.subscribe(name=\"orders-listener\", channels=[\"orders:new\"])\n\n# Pattern subscriber covering an entire topic hierarchy\nhub.psubscribe(name=\"all-notifications\", patterns=[\"notifications:*\"])\n\n# Publish — returns Redis' delivered count for this PUBLISH\ndelivered = hub.publish(\"orders:new\", {\"order_id\": 42, \"total\": 199.0})\nprint(f\"Redis delivered to {delivered} subscriber(s)\")\n\n# Look at what each subscriber received\nfor sub in hub.subscriptions():\n print(sub.name, sub.received_total(), \"messages\")\n for message in sub.messages(limit=5):\n print(\" \", message.channel, message.payload)\n\nhub.unsubscribe(\"orders-listener\")\nhub.shutdown() # closes every remaining subscription", "section_id": "the-pub-sub-hub-helper" }, { "id": "data-model-ex0", "language": "text", "code": "RedisPubSubHub (in-process)\n subscriptions dict[str, Subscription]\n published_total int\n delivered_total int\n channel_published dict[channel -> count]\n\nSubscription (in-process, one per subscriber)\n name str\n targets list[channel | pattern]\n is_pattern bool\n buffer deque[ReceivedMessage] (capped, default 50)\n received_total int\n pubsub redis.client.PubSub (owns one connection)\n thread PubSubWorkerThread (run_in_thread)", "section_id": "data-model" }, { "id": "publishing-messages-ex0", "language": "python", "code": "def publish(self, channel: str, message: object) -> int:\n payload = json.dumps(message, default=str)\n delivered = int(self.redis.publish(channel, payload))\n with self._stats_lock:\n self._published_total += 1\n self._delivered_total += delivered\n self._channel_published[channel] += 1\n return delivered", "section_id": "publishing-messages" }, { "id": "subscribing-to-channels-ex0", "language": "python", "code": "def subscribe(self, name: str, channels: list[str], on_message=None) -> Subscription:\n return self._register(name, channels, is_pattern=False, on_message=on_message)", "section_id": "subscribing-to-channels" }, { "id": "subscribing-to-channels-ex1", "language": "python", "code": "self._pubsub = hub.redis.pubsub(ignore_subscribe_messages=True)\nbindings = {target: self._dispatch for target in self.targets}\nif is_pattern:\n self._pubsub.psubscribe(**bindings)\nelse:\n self._pubsub.subscribe(**bindings)\nself._thread = self._pubsub.run_in_thread(sleep_time=0.01, daemon=True)", "section_id": "subscribing-to-channels" }, { "id": "pattern-subscriptions-with-psubscribe-ex0", "language": "python", "code": "hub.psubscribe(name=\"all-notifications\", patterns=[\"notifications:*\"])\nhub.psubscribe(name=\"cache-invalidator\", patterns=[\"cache:invalidate:*\"])", "section_id": "pattern-subscriptions-with-psubscribe" }, { "id": "pattern-subscriptions-with-psubscribe-ex1", "language": "python", "code": "def _dispatch(self, raw: dict) -> None:\n channel = raw.get(\"channel\") # the actual channel the message was sent to\n pattern = raw.get(\"pattern\") # the pattern that matched (None for exact subs)\n data = raw.get(\"data\")\n ...", "section_id": "pattern-subscriptions-with-psubscribe" }, { "id": "inspecting-active-subscribers-ex0", "language": "python", "code": "hub.active_channels() # PUBSUB CHANNELS *\nhub.channel_subscriber_counts(chs) # PUBSUB NUMSUB ch1 ch2 ...\nhub.pattern_subscriber_count() # PUBSUB NUMPAT", "section_id": "inspecting-active-subscribers" }, { "id": "stats-and-history-ex0", "language": "python", "code": "def stats(self) -> dict:\n return {\n \"published_total\": self._published_total,\n \"delivered_total\": self._delivered_total, # sum of PUBLISH return values\n \"received_total\": sum(s.received_total() for s in self.subscriptions()),\n \"active_subscriptions\": len(self.subscriptions()),\n \"channel_published\": dict(self._channel_published),\n \"pattern_subscriptions\": self.pattern_subscriber_count(),\n }", "section_id": "stats-and-history" }, { "id": "prerequisites-ex0", "language": "bash", "code": "pip install \"redis>=5.0\"", "section_id": "prerequisites" }, { "id": "get-the-source-files-ex0", "language": "bash", "code": "mkdir pub-sub-demo && cd pub-sub-demo\nBASE=https://raw.githubusercontent.com/redis/docs/main/content/develop/use-cases/pub-sub/redis-py\ncurl -O $BASE/pubsub_hub.py\ncurl -O $BASE/demo_server.py", "section_id": "get-the-source-files" }, { "id": "start-the-demo-server-ex0", "language": "bash", "code": "python3 demo_server.py", "section_id": "start-the-demo-server" }, { "id": "start-the-demo-server-ex1", "language": "text", "code": "Redis pub/sub demo server listening on http://127.0.0.1:8095\nUsing Redis at localhost:6379\nSeeded 3 default subscription(s)", "section_id": "start-the-demo-server" }, { "id": "inspect-pub-sub-state-directly-in-redis-ex0", "language": "bash", "code": "# Which channels currently have at least one exact-match subscriber?\nredis-cli pubsub channels '*'\n\n# How many subscribers does each channel have?\nredis-cli pubsub numsub orders:new notifications:billing chat:lobby\n\n# How many active pattern subscriptions across the whole server?\nredis-cli pubsub numpat\n\n# Subscribe interactively from the CLI to watch traffic on a pattern\nredis-cli psubscribe 'orders:*'", "section_id": "inspect-pub-sub-state-directly-in-redis" } ] }