{ "schema_version": 2, "id": "develop/ai/context-engine/agent-memory/ingestion-webhooks", "title": "Review content with ingestion webhooks", "url": "https://redis.io/docs/latest/develop/ai/context-engine/agent-memory/ingestion-webhooks/", "summary": "Configure blocking ingestion webhooks, verify signed callbacks, and recover from failures.", "tags": [ "docs", "develop", "ai" ], "last_updated": "2026-11-03T10:02:56-08:00", "page_type": "content", "content_hash": "3498cd0b5567297d25402bdb2ca88737af9603a88b4efddd6ceeb7f34688da8a", "sections": [ { "id": "code-examples", "title": "Code examples", "role": "example", "text": "These examples connect to Iris services using SDKs or HTTP requests with curl. Follow one language throughout each walkthrough.\n\n\nIngestion webhooks let your application accept, reject, or transform content before Redis Agent Memory saves it. A callback returns a decision; successful HTTP delivery alone does not approve a write.\n\nIngestion webhooks require a self-managed deployment with the Redis background-job backend. Hooks are off by default." }, { "id": "choose-what-to-review", "title": "Choose what to review", "role": "content", "text": "Select each path your policy needs. Selecting creation does not also select updates or deletion." }, { "id": "before-input-is-saved-before_input", "title": "Before input is saved (`before_input`)", "role": "content", "text": "| Path | What triggers it | Decisions |\n|:-----|:-----------------|:----------|\n| `session.append` | An application adds a session event. | Accept, reject, or transform. |\n| `memory.create` | An application creates a memory batch. | Accept, reject, transform, or review each operation. |\n| `document.submit` | An application submits a document for extraction. | Accept, reject, or transform. |\n| `memory.update` | An authorized caller updates a memory with either PATCH endpoint, or a bulk move changes a memory's namespace. Each changed record in a bulk move is reviewed separately. | Accept or reject. |\n| `memory.delete` | An authorized caller deletes a memory. | Accept or reject. |\n| `session.summary`, `session.property` | An application or worker writes a session summary or property. | Accept or reject. |\n| `session.delete`, `session.events.delete` | An application explicitly deletes a session or event. | Accept or reject. |" }, { "id": "before-generated-changes-are-applied-before_apply", "title": "Before generated changes are applied (`before_apply`)", "role": "content", "text": "| Path | What triggers it | Decisions |\n|:-----|:-----------------|:----------|\n| `session.builtin` | Built-in session extraction produces memory changes. | Whole-batch or per-operation decisions. |\n| `session.custom` | Custom session extraction produces memory changes. | Whole-batch or per-operation decisions. |\n| `document.chunk` | Document ingestion reconciles the candidates from every chunk into one final batch. | Whole-batch or per-operation decisions. |\n| `summary_view.refresh` | A Summary View generates a value for publication. | Accept or reject. |\n\nFor `memory.update`, `input.namespaceId` is the proposed namespace and `input.previousNamespaceId` is the stored namespace. Both are namespace IDs, omitted when no namespace is assigned.\n\n> [!NOTE]\n> The `memory.move` path was removed. Before upgrading, replace `memory.move` with `memory.update` in each store policy that selects it.\n\nConfiguration, reads, searches, automatic expiration, and store teardown are outside content hooks. Empty extraction batches and empty Summary View results have no generated content to review. Authorization runs before content is sent to a receiver; an acceptance cannot grant access or bypass validation." }, { "id": "prepare-the-deployment", "title": "Prepare the deployment", "role": "content", "text": "Ask your operator to [enable ingestion webhooks](https://redis.io/docs/latest/operate/iris/agent-memory/self-managed/configuration#enable-ingestion-webhooks) and configure the signing-secret backend." }, { "id": "configure-a-store-policy", "title": "Configure a store policy", "role": "configuration", "text": "Your operator must [configure a store policy](https://redis.io/docs/latest/operate/iris/agent-memory/self-managed/api-examples#configure-a-webhook-policy), then provide the receiver with its signing keys and key IDs through a secret store. The receiver example reads them from `HOOK_KEYS_JSON`. Enable delivery after the receiver is ready." }, { "id": "verify-callbacks-and-return-decisions", "title": "Verify callbacks and return decisions", "role": "content", "text": "The payload identifies `version`, `eventId`, `ingestionOperationId`, `phase`, `path`, store/owner context, configuration revision, correlation ID, and timestamp. Input appears under `input`; generated candidates appear under `operations`. A candidate's `operationId` identifies the proposed operation, separately from its target memory ID.\n\nVerify the exact raw request bytes before interpreting the JSON. The signature is:\n\n[code example]\n\nRead the timestamp, event ID, key ID, and signature from `X-Redis-Webhook-Timestamp`, `X-Redis-Webhook-Event-Id`, `X-Redis-Webhook-Key-Id`, and `X-Redis-Webhook-Signature`. Check freshness and use a constant-time signature comparison. Retries keep the event ID and raw body but receive a fresh signed timestamp.\n\nThis runnable receiver uses FastAPI and the Python standard library. It:\n\n1. Checks the four signature headers and the timestamp.\n1. Verifies the hash-based message authentication code (HMAC) over the raw body.\n1. Returns the saved decision if the event and body match a previous request.\n1. Rejects content containing `[reject]` and accepts other signed content.\n\nReplace the demonstration marker policy with your application's policy. Save as `receiver.py`:\n\n[code example]\n\nInstall and run it with a certificate for the configured receiver hostname:\n\n[code example]\n\nThe example retains up to 1,000 decisions in one process for seven days. Restarting loses that cache. Use a durable shared store for production decisions and store the body fingerprint with each decision. Do not perform external side effects before saving an idempotent decision.\n\nReturn HTTP `200` and `Content-Type: application/json` with a versioned decision. An empty body, bare success status, unknown fields, duplicate keys, or an invalid decision fails closed. Whole-request decisions are:\n\n[code example]\n\n[code example]\n\nA document-input transform replaces its decoded text:\n\n[code example]\n\n| Path | What a transform can change |\n|:-----|:----------------------------|\n| `document.submit` | The decoded document text in `replacement.content`. |\n| `session.append` | `actorId`, `role`, `content`, `createdAt`, and `metadata`. Session identity, owner, and placement stay fixed. |\n| `memory.create` | Editable text, type, session, owner, namespace placement, topics, and attributes. Record count, order, and IDs stay fixed. |\n\nUse `namespaceId` for namespace placement. Transformed records must still pass authorization and validation.\n\nFor `memory.create`, `session.builtin`, `session.custom`, and `document.chunk`, `decision: review` can decide each operation separately:\n\n[code example]\n\nReturn exactly one decision for every proposed operation; missing, duplicate, or unknown IDs invalidate the batch. Manual-create operation IDs are the supplied memory IDs. Generated candidates have separate operation IDs. Generated updates can transform editable fields; deletion can only be accepted or rejected. A receiver cannot add operations, change targets or kinds, or rewrite provenance. Native authorization, exclusions, namespace admission, and concurrency checks still apply." }, { "id": "make-input-retries-idempotent", "title": "Make input retries idempotent", "role": "content", "text": "When selected, session append and manual memory creation require `idempotencyKey` containing 1–128 printable ASCII characters. In Python this is `idempotency_key`; REST uses `idempotencyKey`. Document ingestion already requires this field. Without a selected hook, the key is optional for manual memory creation. Reusing it with the original request replays the saved result; reusing it with a different request returns `409`.\n\nSave the complete original request, including timestamps, before sending it. After a lost response, reuse that request and key. Changing content under the same key conflicts. Saved acceptance and write receipts let a retry resume without repeating confirmed writes. A manual batch can still have per-record errors.\n\nTo try the receiver's rejection marker, create a memory whose text contains `[reject]`. The examples continue from the [quickstart](https://redis.io/docs/latest/develop/ai/context-engine/agent-memory/quickstart) and use its connection values and `quickstart-user`. Each run uses a new memory ID and key, sends the request, and then repeats the identical request to test a retry. The complete program deletes the test memory if it was created.\n\n**Available in:** Python (Iris), cURL (Iris)\n\n##### Python (Iris)\n\n[code example]\n\n##### cURL (Iris)\n\n[code example]\n\n\n\nWith `memory.create` selected, inspect the per-record result and verify that the rejected record was not created. Without `memory.create` selected, the memory is created and the retry replays the same result. Use a new key for changed input; changing the request under the same key conflicts.\n\nThe additional mutation paths and Summary View publication use fresh events rather than this input-replay contract. Retrying them reviews current state again; a lost response after a successful mutation can lead to another review. A callback's acceptance does not promise that its target will remain unchanged until persistence." }, { "id": "handle-failures-and-inspect-status", "title": "Handle failures and inspect status", "role": "content", "text": "Callbacks have a five-second limit. Input callbacks have no automatic receiver retry. Interrupted undecided input can be resumed with the same original request and key; receiver failures remain terminal. Abandoned undecided input is cancelled after 60 seconds without a live lease.\n\nSession and document extraction output have at most three total callback attempts, with delays of 30 and 120 seconds within the job deadline. Document ingestion sends `document.chunk` one final reconciled batch per ingestion, not one batch per chunk. Its `source.chunkId` is the internal `reconcile-v1` work ID. Saved candidates are resumed rather than regenerated. Explicit rejection filters candidates and permits progress; invalid responses or transport failures stop the current batch. Earlier batches may already be saved.\n\nFor `summary_view.refresh`, a recorded rejection, invalid decision, or receiver failure stops that refresh's normal retries. Request a new refresh to generate a new proposal. Interrupted attempts can still be delivered again. Native generation or signing-key lookup failures have their own retry policy.\n\nSession summary and property hooks also apply to worker writes. A rejected summary is not saved and does not advance the worker's summary boundary; property hooks can also stop bookmark writes.\n\n| Response | Action |\n|:---------|:-------|\n| `503` with `/errors/ingestion-hook-retry` | Inspect `retryAt` and resume the same operation after that time. |\n| `422` with `/errors/ingestion-hook-stopped` | Inspect the terminal state. Repeating the same stopped operation is not recovery. |\n| `429` | Hook capacity is exhausted. Apply backoff and inspect pending work. |\n| Bulk success with item errors | Check every record's result; HTTP success does not mean all records were applied. |\n\nDecision `state` and `applyState` are separate. Inspect `applied`, `filtered`, `conflicts`, attempts, safe errors, and retry times. A later failure or an incomplete completion record does not prove earlier writes were rolled back. You cannot manually retry a stopped operation through a retry endpoint.\n\nList hook events and inspect one in detail. The examples print each event's path and states, then read the first event by its ID. For delegated access on a self-managed service, use the authenticated client setup in [delegation](https://redis.io/docs/latest/operate/iris/agent-memory/self-managed/authentication#delegate-a-user-with-an-agent-key).\n\n**Available in:** Python (Iris), cURL (Iris)\n\n##### Python (Iris)\n\n[code example]\n\n##### cURL (Iris)\n\n[code example]\n\n\n\nLists return `items` and `nextCursor`. Pass `nextCursor` as `cursor`, even when the page is empty; ownership filtering can produce that result. Keep the same store, credential, and list parameters. The maximum page size is 100.\n\nPrivate pending content expires within 24 hours or the earlier operation deadline. Compact status and write receipts remain for seven days after completion or expiry. Status omits private input, generated content, prompts, endpoint configuration, and keys.\n\nEach store permits 10,000 hook records and 64 MiB of reserved pending content. Encoded callback requests and responses are limited to 8 MiB; decoded reviewed content is limited to 1 MiB. Selecting `memory.delete` limits a bulk deletion to four memory IDs, and selecting `memory.update` limits a bulk move to four memory IDs; otherwise the native limit is 100. Split larger selected mutations into smaller requests and inspect every result." } ], "examples": [ { "id": "verify-callbacks-and-return-decisions-ex0", "language": "text", "code": "v1=", "section_id": "verify-callbacks-and-return-decisions" }, { "id": "verify-callbacks-and-return-decisions-ex1", "language": "python", "code": "import base64\nimport hashlib\nimport hmac\nimport json\nimport os\nimport time\n\nfrom fastapi import FastAPI, HTTPException, Request\n\napp = FastAPI()\nkeys = {}\nfor key_id, encoded in json.loads(os.environ[\"HOOK_KEYS_JSON\"]).items():\n key = base64.b64decode(encoded, validate=True)\n if not key_id or len(key) != 32 or base64.b64encode(key).decode(\"ascii\") != encoded:\n raise ValueError(\"Keys must be canonical base64 encodings of 32 bytes\")\n keys[key_id] = key\nif not keys:\n raise ValueError(\"Configure at least one signing key\")\ndecisions = {}\n\n\ndef unique_object(pairs):\n result = {}\n for key, value in pairs:\n if key in result:\n raise ValueError(\"Duplicate JSON key\")\n result[key] = value\n return result\n\n\n@app.post(\"/hook\")\nasync def hook(request: Request):\n names = [\"X-Redis-Webhook-Key-Id\", \"X-Redis-Webhook-Event-Id\",\n \"X-Redis-Webhook-Timestamp\", \"X-Redis-Webhook-Signature\"]\n if any(len(request.headers.getlist(name)) != 1 for name in names):\n raise HTTPException(401, \"Invalid signature headers\")\n key_id, event_id, stamp, signature = [request.headers[name] for name in names]\n key = keys.get(key_id)\n try:\n fresh = abs(time.time() - int(stamp)) <= 300\n except ValueError:\n fresh = False\n if not key or not event_id or not fresh:\n raise HTTPException(401, \"Invalid signing identity or timestamp\")\n body = bytearray()\n async for chunk in request.stream():\n body.extend(chunk)\n if len(body) > 8 * 1024 * 1024:\n raise HTTPException(413, \"Body too large\")\n expected = \"v1=\" + hmac.new(\n key, f\"{stamp}.{event_id}.\".encode() + body, hashlib.sha256,\n ).hexdigest()\n if not signature.isascii() or not hmac.compare_digest(signature, expected):\n raise HTTPException(401, \"Invalid signature\")\n try:\n payload = json.loads(body, object_pairs_hook=unique_object)\n except (ValueError, UnicodeDecodeError):\n raise HTTPException(400, \"Invalid JSON\")\n if not isinstance(payload, dict) or payload.get(\"version\") != \"1\" or payload.get(\"eventId\") != event_id:\n raise HTTPException(400, \"Invalid envelope\")\n if payload.get(\"phase\") not in {\"before_input\", \"before_apply\"}:\n raise HTTPException(400, \"Invalid phase\")\n\n # Demo only: one process and at most 1,000 saved decisions.\n # Use a durable shared store before adding replicas or external effects.\n now = time.time()\n for identity, saved in list(decisions.items()):\n if saved[2] <= now:\n del decisions[identity]\n identity = (key_id, event_id)\n digest = hashlib.sha256(body).digest()\n if identity in decisions:\n saved_digest, decision, _ = decisions[identity]\n if saved_digest != digest:\n raise HTTPException(409, \"Event content changed\")\n return decision\n if len(decisions) >= 1000:\n raise HTTPException(503, \"Decision capacity reached\")\n decision = {\"version\": \"1\", \"decision\": \"accept\"}\n if \"[reject]\" in json.dumps(payload, ensure_ascii=False):\n decision.update(decision=\"reject\", reasonCode=\"policy_denied\")\n decisions[identity] = (digest, decision, now + 7 * 24 * 60 * 60)\n return decision", "section_id": "verify-callbacks-and-return-decisions" }, { "id": "verify-callbacks-and-return-decisions-ex2", "language": "sh", "code": "uv venv\nsource .venv/bin/activate\nuv pip install fastapi uvicorn\nuvicorn receiver:app --host 0.0.0.0 --port 8443 \\\n --ssl-certfile --ssl-keyfile ", "section_id": "verify-callbacks-and-return-decisions" }, { "id": "verify-callbacks-and-return-decisions-ex3", "language": "json", "code": "{\"version\":\"1\",\"decision\":\"accept\"}", "section_id": "verify-callbacks-and-return-decisions" }, { "id": "verify-callbacks-and-return-decisions-ex4", "language": "json", "code": "{\"version\":\"1\",\"decision\":\"reject\",\"reasonCode\":\"policy_denied\"}", "section_id": "verify-callbacks-and-return-decisions" }, { "id": "verify-callbacks-and-return-decisions-ex5", "language": "json", "code": "{\"version\":\"1\",\"decision\":\"transform\",\"replacement\":{\"content\":\"Approved document text.\"}}", "section_id": "verify-callbacks-and-return-decisions" }, { "id": "verify-callbacks-and-return-decisions-ex6", "language": "json", "code": "{\n \"version\":\"1\",\n \"decision\":\"review\",\n \"operations\":[\n {\"operationId\":\"\",\"decision\":\"accept\"},\n {\"operationId\":\"\",\"decision\":\"transform\",\"replacement\":{\"text\":\"Approved fact.\"}},\n {\"operationId\":\"\",\"decision\":\"reject\",\"reasonCode\":\"policy_denied\"}\n ]\n}", "section_id": "verify-callbacks-and-return-decisions" }, { "id": "make-input-retries-idempotent-ex0", "language": "python", "code": "suffix = uuid.uuid4().hex[:12]\nrequest = {\n \"idempotency_key\": f\"example-hook-request-{suffix}\",\n \"memories\": [\n {\n \"memory_id\": f\"example-hook-test-{suffix}\",\n \"owner_id\": OWNER_ID,\n \"text\": \"[reject] synthetic webhook test\",\n }\n ],\n}\nfirst = memory.client.bulk_create_long_term_memories(**request)\nprint(\"Created:\", first.created)\nfor error in first.errors or []:\n print(\"Not created:\", error.memory_id, error.status, error.detail)\n\nretry = memory.client.bulk_create_long_term_memories(**request)\nprint(\"Identical retry returns the same result:\", retry.created == first.created)", "section_id": "make-input-retries-idempotent" }, { "id": "make-input-retries-idempotent-ex1", "language": "bash", "code": "SUFFIX=$(openssl rand -hex 6)\njq -n --arg id \"example-hook-test-$SUFFIX\" --arg key \"example-hook-request-$SUFFIX\" --arg owner \"$OWNER_ID\" '{\n idempotencyKey: $key,\n memories: [{memoryId: $id, ownerId: $owner, text: \"[reject] synthetic webhook test\"}]\n}' > hook-test.json\nfor attempt in first retry; do # the retry repeats the identical request\n echo \"$attempt:\"\n curl --fail-with-body --silent --show-error \\\n -X POST \"$MEMORY_API/memories\" \\\n -H \"$AUTH\" -H 'Content-Type: application/json' --data-binary @hook-test.json | jq .\ndone", "section_id": "make-input-retries-idempotent" }, { "id": "handle-failures-and-inspect-status-ex0", "language": "python", "code": "cursor = None\nfirst_event_id = None\nwhile True:\n page = memory.client.list_ingestion_hook_statuses(limit=20, cursor=cursor)\n for event in page.items:\n print(event.event_id, event.path, event.state, event.apply_state)\n first_event_id = first_event_id or event.event_id\n cursor = page.next_cursor\n if not cursor:\n break\n\nif first_event_id:\n detail = memory.client.get_ingestion_hook_status(event_id=first_event_id)\n print(detail.status.model_dump_json(by_alias=True, exclude_none=True))", "section_id": "handle-failures-and-inspect-status" }, { "id": "handle-failures-and-inspect-status-ex1", "language": "bash", "code": "CURSOR=''\nFIRST_EVENT_ID=''\nwhile :; do\n PAGE=$(curl --fail-with-body --silent --show-error --get \\\n \"$MEMORY_API/ingestion-hook-events\" -H \"$AUTH\" \\\n --data-urlencode 'limit=20' ${CURSOR:+--data-urlencode \"cursor=$CURSOR\"})\n jq -c '.items[] | {eventId, path, state, applyState}' <<< \"$PAGE\"\n [ -n \"$FIRST_EVENT_ID\" ] || FIRST_EVENT_ID=$(jq -r '.items[0].eventId // empty' <<< \"$PAGE\")\n CURSOR=$(jq -r '.nextCursor // empty' <<< \"$PAGE\")\n [ -n \"$CURSOR\" ] || break\ndone\n\nif [ -n \"$FIRST_EVENT_ID\" ]; then\n curl --fail-with-body --silent --show-error \\\n \"$MEMORY_API/ingestion-hook-events/$FIRST_EVENT_ID\" -H \"$AUTH\" | jq .\nfi", "section_id": "handle-failures-and-inspect-status" } ] }