{
  "id": "ruby",
  "title": "Redis job queue with redis-rb",
  "url": "https://redis.io/docs/latest/develop/use-cases/job-queue/ruby/",
  "summary": "Implement a Redis job queue in Ruby with redis-rb",
  "tags": [
    "docs",
    "develop",
    "stack",
    "oss",
    "rs",
    "rc"
  ],
  "last_updated": "2026-05-14T08:58:05-05:00",
  "children": [],
  "page_type": "content",
  "content_hash": "72fc884f7a7600ef1682c6564ee686802956fc23184d79e266110e035d9e5bad",
  "sections": [
    {
      "id": "overview",
      "title": "Overview",
      "role": "overview",
      "text": "This guide shows you how to implement a Redis-backed job queue in Ruby with [`redis-rb`](https://redis.io/docs/latest/develop/clients/ruby). It includes a small local web server built with `webrick` from the Ruby standard library so you can enqueue jobs, watch a pool of workers drain them, and see the reclaimer recover jobs from a simulated worker crash."
    },
    {
      "id": "overview",
      "title": "Overview",
      "role": "overview",
      "text": "A job queue lets your application offload background work — sending email, processing payments, image transcoding, ML inference, webhooks — from the request path. Producers enqueue jobs in milliseconds and return to the user; workers pull from the queue and process them on their own schedule.\n\nThat gives you:\n\n* Low-latency user-facing requests, even when downstream work is slow or bursty\n* Horizontal scale across many worker processes that share one Redis instance\n* At-least-once delivery so a worker crash doesn't lose work\n* Visibility-timeout reclaim that returns stuck jobs to the queue automatically\n* Job metadata, retry counts, and completion results in Redis hashes with TTL\n\nIn this example, each job is identified by a random hex ID and its payload, status, and result live in a Redis hash under `queue:jobs:job:{id}`. Pending IDs sit in a list, claimed IDs move atomically to a *processing* list, and completed or failed IDs land in short history lists."
    },
    {
      "id": "how-it-works",
      "title": "How it works",
      "role": "content",
      "text": "The flow looks like this:\n\n1. The application calls `queue.enqueue(payload)`.\n2. The helper writes the job metadata hash and `LPUSH`es the job ID onto the pending list.\n3. A worker calls `queue.claim(timeout_ms: ...)`.\n4. The helper runs `BRPOPLPUSH` to atomically move the next pending ID into the processing list and writes a per-claim `claim_token` plus `claimed_at_ms` on the hash.\n5. The worker runs the job and calls `queue.complete(job, result)` or `queue.fail(job, error)`.\n6. `complete` removes the job from the processing list, writes the result, and `LPUSH`es the ID onto the completed history (with `LTRIM` and an `EXPIRE` on the hash for cleanup).\n7. `fail` either retries the job (back to pending) or moves it to the failed list once retries are exhausted.\n\nIf a worker dies before completing a job, the job sits in the processing list with a `claimed_at_ms` older than the visibility timeout. A periodic call to `queue.reclaim_stuck` finds those jobs and moves them back to pending so another worker can pick them up.\n\nEvery state change holds the token: a worker that has been reclaimed cannot later complete or fail a job another worker has already claimed."
    },
    {
      "id": "the-job-queue-helper",
      "title": "The job queue helper",
      "role": "content",
      "text": "The `RedisJobQueue` class wraps the queue operations\n([source](https://github.com/redis/docs/blob/main/content/develop/use-cases/job-queue/ruby/job_queue.rb)):\n\n[code example]"
    },
    {
      "id": "data-model",
      "title": "Data model",
      "role": "content",
      "text": "Each job's state lives in a Redis hash plus a position in one of four lists:\n\n[code example]\n\nA job's hash carries:\n\n[code example]\n\nThe implementation uses:\n\n* [`LPUSH`](https://redis.io/docs/latest/commands/lpush) to add new job IDs to the pending list.\n* [`BRPOPLPUSH`](https://redis.io/docs/latest/commands/brpoplpush) to atomically claim a job into the processing list.\n* [`LREM`](https://redis.io/docs/latest/commands/lrem) to remove a claimed job from the processing list on complete or fail.\n* [`LTRIM`](https://redis.io/docs/latest/commands/ltrim) to cap the completed and failed history lists.\n* [`HSET`](https://redis.io/docs/latest/commands/hset) / [`HGETALL`](https://redis.io/docs/latest/commands/hgetall) for job metadata.\n* [`EXPIRE`](https://redis.io/docs/latest/commands/expire) on completed and failed hashes for automatic cleanup.\n* [`HINCRBY`](https://redis.io/docs/latest/commands/hincrby) for the attempt counter and the shared totals hash so the demo's multiple worker threads (each with its own Redis connection) report a single consistent count.\n* [`PUBLISH`](https://redis.io/docs/latest/commands/publish) on `queue:jobs:events` for completion signalling.\n* [Lua scripting](https://redis.io/docs/latest/develop/programmability/eval-intro) ([`EVALSHA`](https://redis.io/docs/latest/commands/evalsha)) for the complete, fail, and reclaim flows so each runs atomically against the processing list and metadata hash."
    },
    {
      "id": "enqueueing-jobs",
      "title": "Enqueueing jobs",
      "role": "content",
      "text": "`enqueue` writes the metadata hash and pushes the job ID onto the pending list in one pipeline:\n\n[code example]\n\nThe payload is stored as JSON so the queue can carry arbitrary nested structures without forcing every field into a hash."
    },
    {
      "id": "claiming-jobs-with-brpoplpush",
      "title": "Claiming jobs with BRPOPLPUSH",
      "role": "content",
      "text": "A worker blocks until a job is available, then atomically pops it from the pending list and pushes it onto the processing list. `BRPOPLPUSH` does both in a single Redis call:\n\n[code example]\n\nThe `claim_token` is the worker's proof of ownership for this attempt. Every subsequent state change (complete, fail) checks it before touching the processing list, so a worker that hung past the visibility timeout cannot interfere with the new claimant."
    },
    {
      "id": "completing-jobs",
      "title": "Completing jobs",
      "role": "content",
      "text": "`complete` runs a Lua script so the processing-list removal, the metadata write, and the history push happen atomically:\n\n[code example]\n\nThe Lua script checks the token first and returns `0` if the worker no longer owns the job (because the reclaimer moved it back to pending). The metadata hash also gets an `EXPIRE` so completed jobs are cleaned up automatically."
    },
    {
      "id": "failing-and-retrying",
      "title": "Failing and retrying",
      "role": "content",
      "text": "`fail` either retries the job (back to pending) or moves it to the failed list once retries are exhausted:\n\n[code example]\n\nThe attempt counter is incremented on every `claim`, so a job that fails three times is moved to the failed list with `attempts = 3` and the final `last_error` preserved."
    },
    {
      "id": "reclaiming-stuck-jobs",
      "title": "Reclaiming stuck jobs",
      "role": "content",
      "text": "If a worker dies mid-job — the process is killed, the host loses power, the network partitions — the job sits in the processing list with a `claimed_at_ms` that never advances. A periodic call to `reclaim_stuck` walks the processing list and moves any job past the visibility timeout back to pending:\n\n[code example]\n\nThe Lua script also handles a narrower race: a worker that crashed between `BRPOPLPUSH` and writing `claimed_at_ms`. Those jobs are reclaimed after `2 × visibility_ms` using `enqueued_at_ms` as a fallback timer, so they aren't stranded forever."
    },
    {
      "id": "stats-and-history",
      "title": "Stats and history",
      "role": "content",
      "text": "`stats` reports queue depth plus shared counter totals:\n\n[code example]\n\nTotals live in a Redis hash so each worker thread — which holds its own Redis connection — increments them atomically with `HINCRBY` and the orchestrator sees a single consistent view. The completed and failed lists are capped via `LTRIM` so they never grow unbounded; a real deployment would also write completion events to a separate Stream if you need a longer audit history."
    },
    {
      "id": "prerequisites",
      "title": "Prerequisites",
      "role": "content",
      "text": "* Redis 6.2 or later running locally on the default port (6379). Earlier versions still work, since the helper uses commands that have existed since Redis 2.6.\n* Ruby 3.0 or later.\n* The `redis-rb` client (5.x) and the `webrick` gem (a stdlib gem in Ruby 3 — install it explicitly with `gem install webrick` if it isn't already on your load path):\n\n  [code example]"
    },
    {
      "id": "running-the-demo",
      "title": "Running the demo",
      "role": "content",
      "text": ""
    },
    {
      "id": "get-the-source-files",
      "title": "Get the source files",
      "role": "content",
      "text": "The demo consists of three Ruby files. Download them from the [`ruby` source folder](https://github.com/redis/docs/tree/main/content/develop/use-cases/job-queue/ruby) on GitHub, or grab them with `curl`:\n\n[code example]"
    },
    {
      "id": "start-the-demo-server",
      "title": "Start the demo server",
      "role": "content",
      "text": "From that directory:\n\n[code example]\n\nYou should see:\n\n[code example]\n\nOpen [http://127.0.0.1:8797](http://127.0.0.1:8797) in a browser. You can:\n\n* Enqueue jobs of different kinds (email, webhook, thumbnail, invoice) in batches.\n* Start a pool of workers with configurable size, work latency, and *failure* / *hang* rates. A non-zero hang rate simulates worker crashes.\n* Click **Run reclaim sweep** to move any timed-out processing jobs back to pending.\n* Watch pending / processing / completed / failed lists update every 800 ms.\n\nIf your Redis server is running elsewhere, start the demo with `--redis-host` and `--redis-port`. You can also tune the visibility timeout with `--visibility-ms` and pick a different queue name with `--queue-name` if you want to share a Redis instance with other demos."
    },
    {
      "id": "the-mock-worker-pool",
      "title": "The mock worker pool",
      "role": "content",
      "text": "The demo includes a small `Worker` and `WorkerPool` ([source](https://github.com/redis/docs/blob/main/content/develop/use-cases/job-queue/ruby/worker.rb)) that stands in for whatever real background work your application would run. Each worker:\n\n* Blocks on `queue.claim` for new jobs.\n* Sleeps `work_latency_ms` to simulate doing the work.\n* Either completes successfully, fails (calling `queue.fail`), or *hangs* — returning without completing or failing the job so the reclaimer has to recover it.\n\nThe `fail_rate` and `hang_rate` knobs let you watch the at-least-once delivery and reclaim behaviours from the UI without writing test code.\n\nEach `Worker` runs in its own `Thread` and holds its **own** `RedisJobQueue` instance backed by a dedicated Redis connection. A blocking `BRPOPLPUSH` reserves the underlying connection until it returns, so giving each worker its own connection keeps the HTTP server's stats and jobs endpoints responsive even while every worker is parked on a claim call. The `WorkerPool` is constructed with a `queue_factory` lambda so it can mint one fresh helper per worker."
    },
    {
      "id": "production-usage",
      "title": "Production usage",
      "role": "content",
      "text": ""
    },
    {
      "id": "choose-a-visibility-timeout-that-matches-your-worst-case-job-latency",
      "title": "Choose a visibility timeout that matches your worst-case job latency",
      "role": "content",
      "text": "The visibility timeout has to exceed the longest real job time, with margin. If it's too short, a healthy worker that's running a slow job will get its work duplicated when the reclaimer fires. If it's too long, a real crash takes longer to detect. Most production deployments use a per-queue value tuned to the 99th-percentile job latency — for example, 2 minutes for email and 30 minutes for video transcoding."
    },
    {
      "id": "run-the-reclaimer-on-a-schedule",
      "title": "Run the reclaimer on a schedule",
      "role": "content",
      "text": "The demo only reclaims when you click the button. In production, run `reclaim_stuck` from a periodic task (every few seconds for fast queues, every minute for slow ones), or from each worker before it blocks on `claim`. Both patterns work as long as *someone* runs the sweep."
    },
    {
      "id": "use-a-separate-redis-database-or-key-prefix-per-queue",
      "title": "Use a separate Redis database or key prefix per queue",
      "role": "content",
      "text": "The helper takes a `queue_name` argument so you can run multiple independent queues against one Redis instance — for example, one queue per priority level, or one per job kind. Keep queue keys under a clearly-namespaced prefix (here, `queue:jobs:*`) so they're easy to inspect and easy to clear without touching application data."
    },
    {
      "id": "cap-the-completed-and-failed-history",
      "title": "Cap the completed and failed history",
      "role": "content",
      "text": "The demo keeps the last 50 completed and 50 failed job IDs via `LTRIM`. If you need longer history for audit purposes, write completion events to a separate Redis Stream (or to an external store) and keep the in-queue history short. Stream consumer groups give you the same fan-out semantics with a much richer history."
    },
    {
      "id": "tune-max-attempts-per-job-kind",
      "title": "Tune `max_attempts` per job kind",
      "role": "content",
      "text": "A blanket `max_attempts = 3` is a reasonable default for transient failures (network timeouts, rate limits). Jobs that talk to non-idempotent external systems — for example, posting a Stripe charge — need either application-level idempotency keys or a much lower retry count. The helper exposes `max_attempts` so each queue can pick its own policy."
    },
    {
      "id": "give-each-blocking-worker-its-own-connection",
      "title": "Give each blocking worker its own connection",
      "role": "content",
      "text": "`redis-rb` 5.x is thread-safe — every call through a `Redis` instance is serialised on an internal mutex — but a `BRPOPLPUSH` parks the connection until the call returns. If multiple Ruby threads share a single `Redis` instance and one is blocked on a claim, the others wait behind it. In the demo, the `WorkerPool` builds a fresh `Redis` (and a fresh `RedisJobQueue`) per worker. In production, use a connection pool (such as the `connection_pool` gem) and check out a dedicated connection for blocking commands."
    },
    {
      "id": "inspect-queue-state-directly-in-redis",
      "title": "Inspect queue state directly in Redis",
      "role": "content",
      "text": "Because the queue is just lists and hashes, you can inspect it with `redis-cli`:\n\n[code example]"
    },
    {
      "id": "learn-more",
      "title": "Learn more",
      "role": "related",
      "text": "This example uses the following Redis commands:\n\n* [`LPUSH`](https://redis.io/docs/latest/commands/lpush) to enqueue a job ID.\n* [`BRPOPLPUSH`](https://redis.io/docs/latest/commands/brpoplpush) to atomically claim a job into the processing list.\n* [`LREM`](https://redis.io/docs/latest/commands/lrem) to remove a job from the processing list on complete or fail.\n* [`LRANGE`](https://redis.io/docs/latest/commands/lrange) and [`LLEN`](https://redis.io/docs/latest/commands/llen) to read queue depth and list contents.\n* [`LTRIM`](https://redis.io/docs/latest/commands/ltrim) to cap the completed and failed history.\n* [`HSET`](https://redis.io/docs/latest/commands/hset) and [`HGETALL`](https://redis.io/docs/latest/commands/hgetall) for job metadata.\n* [`HINCRBY`](https://redis.io/docs/latest/commands/hincrby) for the attempt counter and the shared totals hash.\n* [`EXPIRE`](https://redis.io/docs/latest/commands/expire) for automatic cleanup of completed and failed jobs.\n* [`PUBLISH`](https://redis.io/docs/latest/commands/publish) for job-completion notifications.\n* [`EVALSHA`](https://redis.io/docs/latest/commands/evalsha) for atomic complete, fail, and reclaim flows.\n\nSee the [`redis-rb` documentation](https://redis.io/docs/latest/develop/clients/ruby) for full client reference."
    }
  ],
  "examples": [
    {
      "id": "the-job-queue-helper-ex0",
      "language": "ruby",
      "code": "require 'redis'\nrequire_relative 'job_queue'\n\nredis = Redis.new(host: 'localhost', port: 6379)\nqueue = RedisJobQueue.new(redis: redis, visibility_ms: 5000)\n\njob_id = queue.enqueue(kind: 'email', recipient: 'alice@example.com')\n\n# In a worker process:\njob = queue.claim(timeout_ms: 1000)\nif job\n  begin\n    # ... run the job ...\n    queue.complete(job, sent_at: '2026-05-11T15:00:00Z')\n  rescue StandardError => e\n    queue.fail(job, e.message)\n  end\nend\n\n# In a periodic sweeper:\nreclaimed = queue.reclaim_stuck",
      "section_id": "the-job-queue-helper"
    },
    {
      "id": "data-model-ex0",
      "language": "text",
      "code": "queue:jobs:pending          (list)   pending job IDs, oldest at the right\nqueue:jobs:processing       (list)   claimed but not yet completed\nqueue:jobs:completed        (list)   recent successes (LTRIM-capped history)\nqueue:jobs:failed           (list)   terminally failed jobs\nqueue:jobs:job:{id}         (hash)   per-job metadata\nqueue:jobs:events           (pubsub) completion notifications\nqueue:jobs:stats            (hash)   cross-process counter totals",
      "section_id": "data-model"
    },
    {
      "id": "data-model-ex1",
      "language": "text",
      "code": "queue:jobs:job:9a4f...\n  id              = 9a4f...\n  payload         = {\"kind\":\"email\",\"recipient\":\"alice@example.com\"}\n  status          = pending | processing | completed | failed\n  attempts        = 1\n  enqueued_at_ms  = 1715441000000\n  claimed_at_ms   = 1715441000123\n  claim_token     = b3c0d1e2...        (per-claim random token)\n  completed_at_ms = 1715441000456\n  result          = {\"sent_at\":\"...\"}\n  last_error      = \"smtp timeout\"",
      "section_id": "data-model"
    },
    {
      "id": "enqueueing-jobs-ex0",
      "language": "ruby",
      "code": "def enqueue(payload)\n  job_id = SecureRandom.hex(8)\n  now_ms = self.class.now_ms\n  meta = {\n    'id' => job_id,\n    'payload' => JSON.generate(payload),\n    'status' => 'pending',\n    'attempts' => 0,\n    'enqueued_at_ms' => now_ms,\n    'claim_token' => '',\n  }\n  @redis.pipelined do |pipe|\n    pipe.hset(meta_key(job_id), meta)\n    pipe.lpush(@pending_key, job_id)\n    pipe.hincrby(@stats_key, 'enqueued_total', 1)\n  end\n  job_id\nend",
      "section_id": "enqueueing-jobs"
    },
    {
      "id": "claiming-jobs-with-brpoplpush-ex0",
      "language": "ruby",
      "code": "def claim(timeout_ms: 1000)\n  timeout_s = [timeout_ms / 1000.0, 0.1].max\n  job_id = @redis.brpoplpush(@pending_key, @processing_key, timeout: timeout_s)\n  return nil if job_id.nil?\n\n  token = SecureRandom.hex(8)\n  now_ms = self.class.now_ms\n  mkey = meta_key(job_id)\n  results = @redis.pipelined do |pipe|\n    pipe.hset(mkey,\n              'status', 'processing',\n              'claimed_at_ms', now_ms,\n              'claim_token', token)\n    pipe.hincrby(mkey, 'attempts', 1)\n    pipe.hgetall(mkey)\n  end\n  meta = results.last || {}\n  payload = JSON.parse(meta['payload'] || '{}') rescue {}\n  ClaimedJob.new(job_id, payload, (meta['attempts'] || '1').to_i, token)\nend",
      "section_id": "claiming-jobs-with-brpoplpush"
    },
    {
      "id": "completing-jobs-ex0",
      "language": "ruby",
      "code": "def complete(job, result)\n  ok = @redis.evalsha(\n    @complete_sha,\n    keys: [@meta_prefix, @processing_key, @completed_key],\n    argv: [\n      job.id,\n      job.claim_token,\n      'completed',\n      self.class.now_ms,\n      JSON.generate(result),\n      @completed_ttl,\n      @completed_history,\n    ],\n  )\n  return false if ok.nil? || ok.to_i.zero?\n\n  @redis.publish(@events_channel, JSON.generate(id: job.id, status: 'completed'))\n  @redis.hincrby(@stats_key, 'completed_total', 1)\n  true\nend",
      "section_id": "completing-jobs"
    },
    {
      "id": "failing-and-retrying-ex0",
      "language": "ruby",
      "code": "def fail(job, error)\n  retry_flag = job.attempts < @max_attempts\n  result = @redis.evalsha(\n    @fail_sha,\n    keys: [@meta_prefix, @processing_key, @pending_key, @failed_key],\n    argv: [\n      job.id,\n      job.claim_token,\n      error,\n      self.class.now_ms,\n      @completed_ttl,\n      @completed_history,\n      retry_flag ? '1' : '0',\n    ],\n  )\n  return false if result.nil? || result.to_i.zero?\n  @redis.publish(@events_channel,\n                 JSON.generate(id: job.id, status: retry_flag ? 'retry' : 'failed'))\n  @redis.hincrby(@stats_key, 'failed_total', 1) unless retry_flag\n  true\nend",
      "section_id": "failing-and-retrying"
    },
    {
      "id": "reclaiming-stuck-jobs-ex0",
      "language": "ruby",
      "code": "def reclaim_stuck\n  reclaimed = @redis.evalsha(\n    @reclaim_sha,\n    keys: [@pending_key, @processing_key, @meta_prefix],\n    argv: [self.class.now_ms, @visibility_ms],\n  ) || []\n  @redis.hincrby(@stats_key, 'reclaimed_total', reclaimed.length) if reclaimed.any?\n  reclaimed\nend",
      "section_id": "reclaiming-stuck-jobs"
    },
    {
      "id": "stats-and-history-ex0",
      "language": "ruby",
      "code": "def stats\n  pending, processing, completed, failed, counters = @redis.pipelined do |pipe|\n    pipe.llen(@pending_key)\n    pipe.llen(@processing_key)\n    pipe.llen(@completed_key)\n    pipe.llen(@failed_key)\n    pipe.hgetall(@stats_key)\n  end\n  {\n    'enqueued_total'  => (counters['enqueued_total']  || 0).to_i,\n    'completed_total' => (counters['completed_total'] || 0).to_i,\n    'failed_total'    => (counters['failed_total']    || 0).to_i,\n    'reclaimed_total' => (counters['reclaimed_total'] || 0).to_i,\n    'pending_depth'    => pending,\n    'processing_depth' => processing,\n    'completed_depth'  => completed,\n    'failed_depth'     => failed,\n    'visibility_ms'    => @visibility_ms,\n  }\nend",
      "section_id": "stats-and-history"
    },
    {
      "id": "prerequisites-ex0",
      "language": "bash",
      "code": "gem install redis webrick",
      "section_id": "prerequisites"
    },
    {
      "id": "get-the-source-files-ex0",
      "language": "bash",
      "code": "mkdir job-queue-demo && cd job-queue-demo\nBASE=https://raw.githubusercontent.com/redis/docs/main/content/develop/use-cases/job-queue/ruby\ncurl -O $BASE/job_queue.rb\ncurl -O $BASE/worker.rb\ncurl -O $BASE/demo_server.rb",
      "section_id": "get-the-source-files"
    },
    {
      "id": "start-the-demo-server-ex0",
      "language": "bash",
      "code": "ruby demo_server.rb",
      "section_id": "start-the-demo-server"
    },
    {
      "id": "start-the-demo-server-ex1",
      "language": "text",
      "code": "Redis job-queue demo server listening on http://127.0.0.1:8797\nUsing Redis at localhost:6379\nVisibility timeout: 5000 ms",
      "section_id": "start-the-demo-server"
    },
    {
      "id": "inspect-queue-state-directly-in-redis-ex0",
      "language": "bash",
      "code": "# How many pending jobs?\nredis-cli LLEN queue:jobs:pending\n\n# Look at the next 5 jobs to be picked up.\nredis-cli LRANGE queue:jobs:pending -5 -1\n\n# Read a job's metadata.\nredis-cli HGETALL queue:jobs:job:9a4f0d1c\n\n# How many jobs are currently being processed?\nredis-cli LLEN queue:jobs:processing\n\n# Read the shared counter totals.\nredis-cli HGETALL queue:jobs:stats\n\n# Clear everything for this queue (be careful — this deletes work).\nredis-cli --scan --pattern 'queue:jobs:*' | xargs redis-cli DEL",
      "section_id": "inspect-queue-state-directly-in-redis"
    }
  ]
}