{
  "schema_version": 2,
  "id": "integrate/redis-data-integration/data-pipelines/transform-examples/redis-add-field-example",
  "title": "Add new fields to a key",
  "url": "https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/data-pipelines/transform-examples/redis-add-field-example/",
  "summary": "",
  "content": "\nBy default, RDI adds fields to\n[hash](https://redis.io/docs/latest/develop/data-types/hashes) or\n[JSON](https://redis.io/docs/latest/develop/data-types/json) objects in the target\ndatabase that match the columns of the source table.\nThe examples below show how to add extra fields to the target data with the\n[`add_field`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/data-transformation/add_field) transformation.\n\n## Add a single field\n\nThe first example adds a single field to the data.\nThe `source` section selects the `customer` table of the\n[`chinook`](https://github.com/Redislabs-Solution-Architects/rdi-quickstart-postgres)\ndatabase (the optional `db` value here corresponds to the\n`sources.\u003csource-name\u003e.connection.database` value defined in\n[`config.yaml`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config)).\n\nIn the `transform` section, the `add_field` transformation adds an extra field called `localphone`\nto the object, which is created by removing the country and area code from the `phone`\nfield with the\n[JMESPath](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/jmespath-custom-functions) function `regex_replace()`.\nYou can also specify `sql` as the `language` if you prefer to create the new\nfield with an [SQL](https://en.wikipedia.org/wiki/SQL) expression.\n\nThe `output` section specifies `hash` as the `data_type` to write to the target, which\noverrides the default setting of `target_data_type` defined in `config.yaml`. Also, the\n`output.with.key` section specifies a custom key format of the form `cust:\u003cid\u003e` where\nthe `id` part is generated by the `uuid()` function.\n\nThe full example is shown below:\n\n```yaml\nname: Add local phone field to customer\nsource:\n  db: chinook\n  table: customer\ntransform:\n  - uses: add_field\n    with:\n      expression: regex_replace(phone, '\\+[0-9]+ (\\([0-9]+\\) )?', '')\n      field: localphone\n      language: jmespath\noutput:\n  - uses: redis.write\n    with:\n      connection: target\n      data_type: hash\n      key:\n        expression: concat(['cust:', uuid()])\n        language: jmespath\n```\n\nIf you queried the generated target data from the default transformation\nusing [`redis-cli`](https://redis.io/docs/latest/develop/tools/cli), you would\nsee something like the following:\n\n```\n 1) \"customerid\"\n 2) \"27\"\n 3) \"firstname\"\n 4) \"Patrick\"\n 5) \"lastname\"\n 6) \"Gray\"\n.\n.\n17) \"phone\"\n18) \"+1 (520) 622-4200\"\n.\n.\n```\n\nUsing the job file above, the data also includes the new `localphone` field:\n\n```\n 1) \"customerid\"\n 2) \"27\"\n 3) \"firstname\"\n 4) \"Patrick\"\n 5) \"lastname\"\n 6) \"Gray\"\n .\n .\n23) \"localphone\"\n24) \"622-4200\"\n```\n\n## Add multiple fields\n\nThe `add_field` transformation can also add multiple fields at the same time\nif you specify them under a `fields` subsection. The example below adds two\nfields to the `track` objects. The first new field, `seconds`, is created using a SQL\nexpression to calculate the duration of the track in seconds from the\n`milliseconds` field.\nThe second new field, `composerlist`, adds a JSON array using the `split()` function\nto split the `composer` string field wherever it contains a comma.\n\n```yaml\nname: Add multiple fields to track\nsource:\n  db: chinook\n  table: track\ntransform:\n  - uses: add_field\n    with:\n      fields:\n        - expression: floor(milliseconds / 1000)\n          field: seconds\n          language: sql\n        - expression: split(composer)\n          field: composerlist\n          language: jmespath\noutput:\n  - uses: redis.write\n    with:\n      connection: target\n      data_type: json\n      key:\n        expression: concat(['track:', trackid])\n        language: jmespath\n```\n\nYou can query the target database to see the new fields in\nthe JSON object:\n\n```bash\n\u003e JSON.GET track:1 $\n\n\"[{\\\"trackid\\\":1,\\\"name\\\":\\\"For Those About To Rock (We Salute You)\\\",\\\"albumid\\\":1,\\\"mediatypeid\\\":1,\\\"genreid\\\":1,\\\"composer\\\":\\\"Angus Young, Malcolm Young, Brian Johnson\\\",\\\"milliseconds\\\":343719,\\\"bytes\\\":11170334,\\\"unitprice\\\":\\\"0.99\\\",\\\"seconds\\\":343,\\\"composerlist\\\":[\\\"Angus Young\\\",\\\" Malcolm Young\\\",\\\" Brian Johnson\\\"]}]\"\n```\n\n## Using `add_field` with `remove_field`\n\nYou can use the `add_field` and\n[`remove_field`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/data-pipelines/transform-examples/redis-remove-field-example)\ntransformations together to completely replace fields from the source. For example,\nif you add a new `fullname` field, you might not need the separate `firstname` and\n`lastname` fields. You can remove them with a job file like the following:\n\n```yaml\nname: Add fullname and remove separate name fields\nsource:\n  db: chinook\n  table: customer\ntransform:\n  - uses: add_field\n    with:\n      expression: concat(firstname, ' ', lastname)\n      field: fullname\n      language: sql\n  - uses: remove_field\n    with:\n      fields:\n        - field: firstname\n        - field: lastname\noutput:\n  - uses: redis.write\n    with:\n      connection: target\n      data_type: hash\n      key:\n        expression: concat(['cust:', customerid])\n        language: jmespath\n```\n",
  "tags": ["docs","integrate","rs","rdi"],
  "last_updated": "2026-09-19T17:55:58-07:00"
}