{ "id": "redis-add-field-example", "title": "Add new fields to a key", "url": "https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/transform-examples/redis-add-field-example/", "summary": "", "tags": [ "docs", "integrate", "rs", "rdi" ], "last_updated": "2026-04-01T08:10:08-05:00", "page_type": "content", "content_hash": "9e5087b52b16e65cad57f59124b8766cfc58ad5f5fe6e1fe913eb835849cd3a2", "sections": [ { "id": "overview", "title": "Overview", "role": "overview", "text": "By default, RDI adds fields to\n[hash]() or\n[JSON]() objects in the target\ndatabase that match the columns of the source table.\nThe examples below show how to add extra fields to the target data with the\n[`add_field`]() transformation." }, { "id": "add-a-single-field", "title": "Add a single field", "role": "content", "text": "The first example adds a single field to the data.\nThe `source` section selects the `customer` table of the\n[`chinook`](https://github.com/Redislabs-Solution-Architects/rdi-quickstart-postgres)\ndatabase (the optional `db` value here corresponds to the\n`sources..connection.database` value defined in\n[`config.yaml`]()).\n\nIn the `transform` section, the `add_field` transformation adds an extra field called `localphone`\nto the object, which is created by removing the country and area code from the `phone`\nfield with the\n[JMESPath]() function `regex_replace()`.\nYou can also specify `sql` as the `language` if you prefer to create the new\nfield with an [SQL](https://en.wikipedia.org/wiki/SQL) expression.\n\nThe `output` section specifies `hash` as the `data_type` to write to the target, which\noverrides the default setting of `target_data_type` defined in `config.yaml`. Also, the\n`output.with.key` section specifies a custom key format of the form `cust:` where\nthe `id` part is generated by the `uuid()` function.\n\nThe full example is shown below:\n\n[code example]\n\nIf you queried the generated target data from the default transformation\nusing [`redis-cli`](), you would\nsee something like the following:\n\n[code example]\n\nUsing the job file above, the data also includes the new `localphone` field:\n\n[code example]" }, { "id": "add-multiple-fields", "title": "Add multiple fields", "role": "content", "text": "The `add_field` transformation can also add multiple fields at the same time\nif you specify them under a `fields` subsection. The example below adds two\nfields to the `track` objects. The first new field, `seconds`, is created using a SQL\nexpression to calculate the duration of the track in seconds from the\n`milliseconds` field.\nThe second new field, `composerlist`, adds a JSON array using the `split()` function\nto split the `composer` string field wherever it contains a comma.\n\n[code example]\n\nYou can query the target database to see the new fields in\nthe JSON object:\n\n[code example]" }, { "id": "using-add-field-with-remove-field", "title": "Using `add_field` with `remove_field`", "role": "content", "text": "You can use the `add_field` and\n[`remove_field`]()\ntransformations together to completely replace fields from the source. For example,\nif you add a new `fullname` field, you might not need the separate `firstname` and\n`lastname` fields. You can remove them with a job file like the following:\n\n[code example]" } ], "examples": [ { "id": "add-a-single-field-ex0", "language": "yaml", "code": "name: Add local phone field to customer\nsource:\n db: chinook\n table: customer\ntransform:\n - uses: add_field\n with:\n expression: regex_replace(phone, '\\+[0-9]+ (\\([0-9]+\\) )?', '')\n field: localphone\n language: jmespath\noutput:\n - uses: redis.write\n with:\n connection: target\n data_type: hash\n key:\n expression: concat(['cust:', uuid()])\n language: jmespath", "section_id": "add-a-single-field" }, { "id": "add-a-single-field-ex1", "language": "plaintext", "code": "1) \"customerid\"\n 2) \"27\"\n 3) \"firstname\"\n 4) \"Patrick\"\n 5) \"lastname\"\n 6) \"Gray\"\n.\n.\n17) \"phone\"\n18) \"+1 (520) 622-4200\"\n.\n.", "section_id": "add-a-single-field" }, { "id": "add-a-single-field-ex2", "language": "plaintext", "code": "1) \"customerid\"\n 2) \"27\"\n 3) \"firstname\"\n 4) \"Patrick\"\n 5) \"lastname\"\n 6) \"Gray\"\n .\n .\n23) \"localphone\"\n24) \"622-4200\"", "section_id": "add-a-single-field" }, { "id": "add-multiple-fields-ex0", "language": "yaml", "code": "name: Add multiple fields to track\nsource:\n db: chinook\n table: track\ntransform:\n - uses: add_field\n with:\n fields:\n - expression: floor(milliseconds / 1000)\n field: seconds\n language: sql\n - expression: split(composer)\n field: composerlist\n language: jmespath\noutput:\n - uses: redis.write\n with:\n connection: target\n data_type: json\n key:\n expression: concat(['track:', trackid])\n language: jmespath", "section_id": "add-multiple-fields" }, { "id": "add-multiple-fields-ex1", "language": "bash", "code": "> JSON.GET track:1 $\n\n\"[{\\\"trackid\\\":1,\\\"name\\\":\\\"For Those About To Rock (We Salute You)\\\",\\\"albumid\\\":1,\\\"mediatypeid\\\":1,\\\"genreid\\\":1,\\\"composer\\\":\\\"Angus Young, Malcolm Young, Brian Johnson\\\",\\\"milliseconds\\\":343719,\\\"bytes\\\":11170334,\\\"unitprice\\\":\\\"0.99\\\",\\\"seconds\\\":343,\\\"composerlist\\\":[\\\"Angus Young\\\",\\\" Malcolm Young\\\",\\\" Brian Johnson\\\"]}]\"", "section_id": "add-multiple-fields" }, { "id": "using-add-field-with-remove-field-ex0", "language": "yaml", "code": "name: Add fullname and remove separate name fields\nsource:\n db: chinook\n table: customer\ntransform:\n - uses: add_field\n with:\n expression: concat(firstname, ' ', lastname)\n field: fullname\n language: sql\n - uses: remove_field\n with:\n fields:\n - field: firstname\n - field: lastname\noutput:\n - uses: redis.write\n with:\n connection: target\n data_type: hash\n key:\n expression: concat(['cust:', customerid])\n language: jmespath", "section_id": "using-add-field-with-remove-field" } ] }