{
  "schema_version": 2,
  "id": "integrate/redis-data-integration/data-pipelines/transform-examples/redis-lookup-example",
  "title": "Reading Redis data with redis.lookup",
  "url": "https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/data-pipelines/transform-examples/redis-lookup-example/",
  "summary": "",
  "content": "\nYou can use the\n[`redis.lookup`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/data-transformation/lookup)\ntransformation to read existing data from Redis during the `transform` stage of a\njob. This lets you enrich an incoming record with values that are already present\nin the target database.\n\nFor example, a pipeline for the Chinook database might read an `artist` record\nthat is already stored in Redis and use `redis.lookup` in an `album` table job to\nadd selected artist details to each album record before writing it to the target\ndatabase.\n\n\u003e [!WARNING]\n\u003e Do not rely on `redis.lookup` to *denormalize* data that RDI writes from another\n\u003e table in the **same pipeline**. RDI can't guarantee that the looked-up data will be\n\u003e present or up to date when the lookup runs, for the following reasons:\n\u003e\n\u003e - **Snapshot order isn't guaranteed.** During the initial snapshot, RDI can't\n\u003e   guarantee that the table you look up is ingested before the table that depends\n\u003e   on it. If a dependent job runs before the referenced key has been written, the\n\u003e   lookup misses.\n\u003e - **Change (CDC) order isn't guaranteed.** If a parent and child record are\n\u003e   inserted or updated at around the same time, RDI has no way to order these\n\u003e   events, so the lookup can still miss.\n\u003e - **Parent updates don't refresh existing keys.** Even if the lookup succeeds,\n\u003e   updating the source record later does *not* update the keys that already copied\n\u003e   its values. The denormalized data becomes stale.\n\u003e\n\u003e The only case where `redis.lookup` is safe for enrichment is when you can guarantee\n\u003e that the looked-up data is present in the target database *independently* of the\n\u003e RDI pipeline (for example, a reference table that is loaded and maintained\n\u003e separately).\n\u003e\n\u003e To denormalize data that RDI ingests, use a supported technique instead. See\n\u003e [Data denormalization](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/data-pipelines/data-denormalization)\n\u003e for one-to-one joins (using `merge`) and one-to-many joins (using nesting).\n\n## Reading a hash field\n\nThe `redis.lookup` transformation works by executing a Redis command and adding the\nresult to the record. You specify the command and its arguments in the\n`transform` configuration with the `cmd` and `args` properties. For example, the\nfollowing transformation job uses the\n[`HGET`](https://redis.io/docs/latest/commands/hget) command to read the `name` field from an\nartist [hash](https://redis.io/docs/latest/develop/data-types/hashes) and adds it to the\nalbum record under the `artist` field. A particularly important thing to note\nhere is that the `args` elements are all interpreted as [JMESPath](https://jmespath.org/)\nexpressions, but YAML syntax allows for each element to be a quoted string. This means that\nyou must *double quote* any string arguments that you want to be treated as\nliteral strings (as with `name` below), otherwise JMESPath will try to interpret\nthem as field names, which will generally give the wrong result. Specifically, use\na different quote character for the outer quotes and the inner quotes.\n\n```yaml\nsource:\n  table: album\ntransform:\n  - uses: redis.lookup\n    with:\n      connection: target\n      cmd: HGET\n      args:\n        - concat(['artist:artistid:', artistid])\n        - '`name`'\n      language: jmespath\n      field: artist\noutput:\n  - uses: redis.write\n    with:\n      connection: target\n      data_type: hash\n      key:\n        expression: concat(['album:albumid:', albumid])\n        language: jmespath\n```\n\nBefore the lookup runs, the album hash object contains only the `artistid` field to\nreference the artist:\n\n```bash\n\u003e hgetall album:albumid:1\n1) \"albumid\"\n2) \"1\"\n3) \"title\"\n4) \"For Those About To Rock We Salute You\"\n5) \"artistid\"\n6) \"1\"\n```\n\nAfter running the job specified above, querying one of the album hash objects shows the\nextra `artist` field obtained by looking up the artist with the `artistid`:\n\n```bash\n\u003e hgetall album:albumid:1\n1) \"albumid\"\n2) \"1\"\n3) \"title\"\n4) \"For Those About To Rock We Salute You\"\n5) \"artistid\"\n6) \"1\"\n7) \"artist\"\n8) \"AC/DC\"\n```\n\n## Embedding a JSON document\n\nIf you are using [JSON](https://redis.io/docs/latest/develop/data-types/json) objects,\nyou can read the whole of one object and embed it\nas a field of another. The following example shows how to do this using a temporary field\nto hold the result of the `redis.lookup` command. It then uses\n[`add_field`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/data-transformation/add_field)\nto insert the new field and\n[`remove_field`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/data-transformation/remove_field)\nto remove the temporary field and the now-redundant `artistid` field before writing the album object.\n\n```yaml\nsource:\n  table: album\ntransform:\n  - uses: redis.lookup\n    with:\n      connection: target\n      cmd: JSON.GET\n      args:\n        - concat(['artist:artistid:', artistid])\n      language: jmespath\n      field: artiststring\n  - uses: add_field\n    with:\n      field: artist\n      language: jmespath\n      expression: json_parse(artiststring)\n  - uses: remove_field\n    with:\n      fields:\n        - field: artistid\n        - field: artiststring\noutput:\n  - uses: redis.write\n    with:\n      connection: target\n      data_type: json\n      key:\n        expression: concat(['album:albumid:', albumid])\n        language: jmespath\n```\n\nAfter running this job, the album JSON object includes the artist object\nin a new `artist` field:\n\n```json\n{\n  \"albumid\": 239,\n  \"title\": \"War\",\n  \"artist\": {\n    \"artistid\": 150,\n    \"name\": \"U2\"\n  }\n}\n```\n",
  "tags": ["docs","integrate","rs","rdi"],
  "last_updated": "2026-09-19T17:55:58-07:00"
}