Data modeling for search
Learn how to store records in Redis so they can be searched, and when to choose hashes versus JSON documents.
This is step 1 of the Redis Search tutorial.
Before you can search anything, you need to decide how to store it. Redis gives you two natural ways to represent a structured record like a product: as a hash or as a JSON document. Both can be indexed and searched. This page explains the difference, helps you choose, and then loads the tutorial dataset.
A record as a hash
A hash stores a flat set of field-value pairs under a single key. It is the simplest way to represent a record and maps neatly onto a row of fields:
Foundational: Store a record as a hash with HSET when your data is a flat set of fields
r.hset(
"product:1",
mapping={
"name": "Aurora AcousticPro Headphones",
"brand": "Aurora",
"category": "Audio",
"price": 199.99,
"rating": 4.6,
},
)
# >>> 5
"""
Code samples for the search and query tutorial:
https://redis.io/docs/latest/develop/get-started/search-tutorial/
"""
import json
import redis
import redis.commands.search.aggregation as aggregations
import redis.commands.search.reducers as reducers
from redis.commands.json.path import Path
from redis.commands.search.field import (
NumericField,
TagField,
TextField,
VectorField,
)
from redis.commands.search.index_definition import IndexDefinition, IndexType
from redis.commands.search.query import Query
r = redis.Redis(host="localhost", port=6379, db=0, decode_responses=True)
r.hset(
"product:1",
mapping={
"name": "Aurora AcousticPro Headphones",
"brand": "Aurora",
"category": "Audio",
"price": 199.99,
"rating": 4.6,
},
)
# >>> 5
r.json().set(
"product:1",
Path.root_path(),
{
"name": "Aurora AcousticPro Headphones",
"brand": "Aurora",
"category": "Audio",
"price": 199.99,
"rating": 4.6,
"features": ["wireless", "noise-cancelling", "bluetooth"],
"specs": {"color": "midnight black", "weight_grams": 268},
},
)
# >>> True
catalog = [
{
"name": "Aurora AcousticPro Headphones",
"brand": "Aurora",
"category": "Audio",
"description": (
"Over-ear wireless headphones with active noise cancelling and a "
"40-hour battery. Plush memory-foam earcups and a lightweight frame "
"make them comfortable for all-day listening, whether you are "
"commuting, working, or relaxing at home."
),
"price": 199.99,
"rating": 4.6,
"review_count": 1284,
"stock": 42,
"release_year": 2024,
"features": ["wireless", "noise-cancelling", "bluetooth", "over-ear"],
"specs": {"color": "midnight black", "weight_grams": 268, "warranty_years": 2},
},
{
"name": "Aurora BudsMini Earbuds",
"brand": "Aurora",
"category": "Audio",
"description": (
"Tiny true-wireless earbuds with a secure in-ear fit and sweat "
"resistance for workouts. The compact charging case slips into a "
"pocket and delivers three full recharges on the go."
),
"price": 89.99,
"rating": 4.3,
"review_count": 942,
"stock": 130,
"release_year": 2023,
"features": ["wireless", "bluetooth", "in-ear", "water-resistant"],
"specs": {"color": "pearl white", "weight_grams": 5, "warranty_years": 1},
},
{
"name": "Sonus Boom Portable Speaker",
"brand": "Sonus",
"category": "Audio",
"description": (
"A rugged portable Bluetooth speaker with deep bass and a waterproof "
"shell. Toss it in a bag for the beach or a campsite and enjoy "
"room-filling sound for up to 20 hours per charge."
),
"price": 129.5,
"rating": 4.5,
"review_count": 512,
"stock": 64,
"release_year": 2024,
"features": ["wireless", "bluetooth", "portable", "waterproof"],
"specs": {"color": "slate gray", "weight_grams": 540, "warranty_years": 1},
},
{
"name": "Pixma Vortex 15 Laptop",
"brand": "Pixma",
"category": "Computers",
"description": (
"A thin-and-light 15-inch laptop with a fast multi-core processor, "
"16 GB of memory, and a speedy solid-state drive. The backlit keyboard "
"and bright display make it a capable companion for work and study."
),
"price": 1399.0,
"rating": 4.7,
"review_count": 318,
"stock": 18,
"release_year": 2024,
"features": ["laptop", "ssd", "backlit-keyboard", "lightweight"],
"specs": {"color": "space silver", "weight_grams": 1600, "warranty_years": 2},
},
{
"name": "Pixma UltraView 27 Monitor",
"brand": "Pixma",
"category": "Computers",
"description": (
"A 27-inch 4K monitor with an IPS panel for accurate colors and wide "
"viewing angles. A single USB-C cable carries video and power, keeping "
"your desk tidy."
),
"price": 329.99,
"rating": 4.4,
"review_count": 221,
"stock": 27,
"release_year": 2023,
"features": ["monitor", "4k", "ips", "usb-c"],
"specs": {"color": "black", "weight_grams": 5200, "warranty_years": 3},
},
{
"name": "Clackr Mechanical Keyboard",
"brand": "Clackr",
"category": "Accessories",
"description": (
"A compact mechanical keyboard with tactile switches, per-key RGB "
"lighting, and wireless connectivity. Hot-swappable switches let you "
"tune the typing feel without soldering."
),
"price": 119.0,
"rating": 4.8,
"review_count": 1502,
"stock": 88,
"release_year": 2024,
"features": ["keyboard", "mechanical", "rgb", "wireless"],
"specs": {"color": "graphite", "weight_grams": 720, "warranty_years": 2},
},
{
"name": "Glide Pro Wireless Mouse",
"brand": "Glide",
"category": "Accessories",
"description": (
"An ergonomic wireless mouse with a high-precision sensor and a "
"contoured shape that reduces wrist strain. A single charge lasts for "
"weeks of everyday use."
),
"price": 59.99,
"rating": 4.2,
"review_count": 869,
"stock": 150,
"release_year": 2022,
"features": ["mouse", "wireless", "ergonomic"],
"specs": {"color": "charcoal", "weight_grams": 98, "warranty_years": 1},
},
{
"name": "Pulse Series 6 Smartwatch",
"brand": "Pulse",
"category": "Wearables",
"description": (
"A sleek smartwatch with built-in GPS, continuous heart-rate "
"monitoring, and water resistance for swimming. Track workouts, sleep, "
"and notifications from your wrist."
),
"price": 249.0,
"rating": 4.5,
"review_count": 1733,
"stock": 51,
"release_year": 2024,
"features": ["smartwatch", "gps", "heart-rate", "water-resistant"],
"specs": {"color": "rose gold", "weight_grams": 38, "warranty_years": 1},
},
{
"name": "Pulse Band Fitness Tracker",
"brand": "Pulse",
"category": "Wearables",
"description": (
"A lightweight fitness band that tracks steps, heart rate, and sleep "
"stages. The slim screen shows daily progress and the battery lasts a "
"full week between charges."
),
"price": 79.99,
"rating": 4.1,
"review_count": 2210,
"stock": 200,
"release_year": 2023,
"features": ["fitness-tracker", "heart-rate", "sleep-tracking"],
"specs": {"color": "ocean blue", "weight_grams": 24, "warranty_years": 1},
},
{
"name": "Lumi Glow Smart Bulb",
"brand": "Lumi",
"category": "Home",
"description": (
"A color-changing smart bulb that connects over Wi-Fi and works with "
"voice assistants. Dim it for movie night or set a warm white for "
"reading, all from your phone."
),
"price": 24.99,
"rating": 4.0,
"review_count": 640,
"stock": 320,
"release_year": 2022,
"features": ["smart-home", "wifi", "dimmable", "color"],
"specs": {"color": "white", "weight_grams": 70, "warranty_years": 2},
},
{
"name": "Lumi Climate Smart Thermostat",
"brand": "Lumi",
"category": "Home",
"description": (
"A learning smart thermostat that adjusts heating and cooling to your "
"routine and helps lower energy bills. The crisp display and Wi-Fi app "
"make scheduling effortless."
),
"price": 149.0,
"rating": 4.6,
"review_count": 388,
"stock": 75,
"release_year": 2024,
"features": ["smart-home", "wifi", "energy-saving"],
"specs": {"color": "white", "weight_grams": 210, "warranty_years": 3},
},
{
"name": "Vista Action Cam 4K",
"brand": "Vista",
"category": "Cameras",
"description": (
"A pocket-sized action camera that shoots stabilized 4K video and is "
"waterproof without a case. Mount it on a helmet or bike and capture "
"your adventures in sharp detail."
),
"price": 299.0,
"rating": 4.3,
"review_count": 455,
"stock": 33,
"release_year": 2023,
"features": ["camera", "4k", "waterproof", "wifi"],
"specs": {"color": "black", "weight_grams": 128, "warranty_years": 1},
},
]
for product_id, product in enumerate(catalog, start=1):
r.json().set(f"product:{product_id}", Path.root_path(), product)
res = r.json().get("product:1", "$.name")
print(res) # >>> ['Aurora AcousticPro Headphones']
schema = (
TextField("$.name", as_name="name"),
TagField("$.brand", as_name="brand", sortable=True),
TagField("$.category", as_name="category"),
TextField("$.description", as_name="description"),
NumericField("$.price", as_name="price", sortable=True),
NumericField("$.rating", as_name="rating", sortable=True),
NumericField("$.review_count", as_name="review_count"),
NumericField("$.stock", as_name="stock"),
NumericField("$.release_year", as_name="release_year", sortable=True),
TagField("$.features[*]", as_name="features"),
)
index = r.ft("idx:catalog")
index.create_index(
schema,
definition=IndexDefinition(prefix=["product:"], index_type=IndexType.JSON),
)
info = r.ft("idx:catalog").info()
print("Documents indexed:", info["num_docs"]) # >>> Documents indexed: 12
res = index.search(Query("*").paging(0, 0))
print("Total products:", res.total) # >>> Total products: 12
res = index.search(Query("@name:headphones").return_field("name"))
print(res.docs)
# >>> [Document {'id': 'product:1', ... 'name': 'Aurora AcousticPro Headphones'}]
res = index.search(Query('@description:"noise cancelling"').return_field("name"))
print(res.total, [d.name for d in res.docs])
# >>> 1 ['Aurora AcousticPro Headphones']
res = index.search(Query("@category:{Audio}").return_fields("name", "price"))
print(res.total, [d.name for d in res.docs])
# >>> 3 ['Aurora BudsMini Earbuds', 'Sonus Boom Portable Speaker', ...]
res = index.search(Query("@features:{waterproof}").return_field("name"))
print(res.total, [d.name for d in res.docs])
# >>> 2 ['Sonus Boom Portable Speaker', 'Vista Action Cam 4K']
res = index.search(
Query("@price:[0 100]").sort_by("price", asc=True).return_fields("name", "price")
)
print([(d.name, d.price) for d in res.docs])
# >>> [('Lumi Glow Smart Bulb', '24.99'), ('Glide Pro Wireless Mouse', '59.99'), ...]
res = index.search(
Query("@category:{Audio} @price:[0 100]").return_fields("name", "price")
)
print(res.total, [d.name for d in res.docs])
# >>> 1 ['Aurora BudsMini Earbuds']
res = index.search(
Query("*").sort_by("price", asc=False).return_fields("name", "price").paging(0, 3)
)
print([(d.name, d.price) for d in res.docs])
# >>> [('Pixma Vortex 15 Laptop', '1399'), ('Pixma UltraView 27 Monitor', '329.99'), ...]
req = aggregations.AggregateRequest("*").group_by(
"@category", reducers.count().alias("count")
)
res = index.aggregate(req).rows
print(res)
# >>> [['category', 'Audio', 'count', '3'], ['category', 'Computers', 'count', '2'], ...]
req = (
aggregations.AggregateRequest("*")
.group_by("@category", reducers.avg("@price").alias("avg_price"))
.sort_by(aggregations.Desc("@avg_price"))
)
res = index.aggregate(req).rows
print(res)
# >>> [['category', 'Computers', 'avg_price', '864.495'], ...]
req = (
aggregations.AggregateRequest("@category:{Audio}")
.load("name", "price")
.apply(sale_price="@price - (@price * 0.1)")
)
res = index.aggregate(req).rows
print(res)
# >>> [['name', 'Aurora AcousticPro Headphones', 'price', '199.99', 'sale_price', '179.991'], ...]
req = (
aggregations.AggregateRequest("*")
.group_by("@brand", reducers.avg("@rating").alias("avg_rating"))
.sort_by(aggregations.Desc("@avg_rating"))
)
res = index.aggregate(req).rows
print(res)
# >>> [['brand', 'Clackr', 'avg_rating', '4.8'], ['brand', 'Pixma', 'avg_rating', '4.55'], ...]
from sentence_transformers import SentenceTransformer
embedder = SentenceTransformer("msmarco-distilbert-base-v4") # 768-dimensional vectors
for key in r.scan_iter(match="product:*"):
description = r.json().get(key, "$.description")[0]
embedding = embedder.encode(description).astype("float32").tolist()
r.json().set(key, "$.embedding", embedding)
r.ft("idx:catalog").dropindex()
schema = (
TextField("$.name", as_name="name"),
TagField("$.brand", as_name="brand", sortable=True),
TagField("$.category", as_name="category"),
TextField("$.description", as_name="description"),
NumericField("$.price", as_name="price", sortable=True),
NumericField("$.rating", as_name="rating", sortable=True),
TagField("$.features[*]", as_name="features"),
VectorField(
"$.embedding",
"FLAT",
{"TYPE": "FLOAT32", "DIM": 768, "DISTANCE_METRIC": "COSINE"},
as_name="embedding",
),
)
index = r.ft("idx:catalog")
index.create_index(
schema,
definition=IndexDefinition(prefix=["product:"], index_type=IndexType.JSON),
)
query_vector = (
embedder.encode("portable music for the outdoors").astype("float32").tobytes()
)
res = index.search(
Query("(*)=>[KNN 3 @embedding $query_vector AS score]")
.sort_by("score", asc=True)
.return_fields("score", "name")
.dialect(2),
query_params={"query_vector": query_vector},
)
print([d.name for d in res.docs])
# >>> ['Sonus Boom Portable Speaker', 'Aurora BudsMini Earbuds', 'Aurora AcousticPro Headphones']
res = index.search(
Query("(@category:{Audio})=>[KNN 3 @embedding $query_vector AS score]")
.sort_by("score", asc=True)
.return_field("name")
.dialect(2),
query_params={"query_vector": query_vector},
)
print([d.name for d in res.docs])
# >>> ['Sonus Boom Portable Speaker', 'Aurora BudsMini Earbuds', 'Aurora AcousticPro Headphones']
hybrid_vector = (
embedder.encode("wireless headphones for listening to music")
.astype("float32")
.tobytes()
)
res = r.execute_command(
"FT.HYBRID",
"idx:catalog",
"SEARCH",
"wireless",
"VSIM",
"@embedding",
"$query_vector",
"KNN",
"2",
"K",
"5",
"LOAD",
"1",
"@name",
"PARAMS",
"2",
"query_vector",
hybrid_vector,
)
print(res)
# >>> {'total_results': 7, 'results': [{'name': 'Aurora AcousticPro Headphones'}, ...]}
Hashes are compact and fast, but they are flat: every value is a string or number. There is no natural place to put a nested object (like a specs sub-record) or a list of values (like multiple features) without flattening or encoding it yourself.
A record as a JSON document
The JSON data type stores a full JSON document under a key. It can represent nested objects and arrays directly, which matches how application data usually looks:
Foundational: Store a record as a JSON document with JSON.SET when your data has nested objects or arrays
r.json().set(
"product:1",
Path.root_path(),
{
"name": "Aurora AcousticPro Headphones",
"brand": "Aurora",
"category": "Audio",
"price": 199.99,
"rating": 4.6,
"features": ["wireless", "noise-cancelling", "bluetooth"],
"specs": {"color": "midnight black", "weight_grams": 268},
},
)
# >>> True
Notice that features is a real array and specs is a real nested object. You did not have to flatten them.
Which should you use?
Both hashes and JSON documents can be indexed and searched by Redis Search, so you can search either one. Use this as a guide:
| Choose hashes when... | Choose JSON when... |
|---|---|
| Your records are flat (no nesting). | Your records have nested objects or arrays. |
| You want the smallest possible memory footprint. | You want your stored shape to match your application objects. |
| You frequently update individual fields. | You want to read, update, or index nested paths directly. |
For this tutorial, the catalog records have arrays (features) and a nested object (specs), so we will use JSON documents for the rest of the tutorial. If you are coming from a background where every record is a flat row, JSON is also a gentle way to keep your existing object shapes.
Note:
This is a modeling choice, not a limitation. The indexing and query commands you will learn (FT.CREATE, FT.SEARCH, FT.AGGREGATE) work with both hashes and JSON. The main practical difference shows up when indexing arrays, which you will see on the next page.Load the dataset
Now load the full catalog of 12 products. Each product is stored as a JSON document under a key with the prefix product:. The key prefix matters: in the next step you will tell Redis to index every key that starts with product:.
Foundational: Load the tutorial dataset as JSON documents under a shared key prefix using JSON.SET
catalog = [
{
"name": "Aurora AcousticPro Headphones",
"brand": "Aurora",
"category": "Audio",
"description": (
"Over-ear wireless headphones with active noise cancelling and a "
"40-hour battery. Plush memory-foam earcups and a lightweight frame "
"make them comfortable for all-day listening, whether you are "
"commuting, working, or relaxing at home."
),
"price": 199.99,
"rating": 4.6,
"review_count": 1284,
"stock": 42,
"release_year": 2024,
"features": ["wireless", "noise-cancelling", "bluetooth", "over-ear"],
"specs": {"color": "midnight black", "weight_grams": 268, "warranty_years": 2},
},
{
"name": "Aurora BudsMini Earbuds",
"brand": "Aurora",
"category": "Audio",
"description": (
"Tiny true-wireless earbuds with a secure in-ear fit and sweat "
"resistance for workouts. The compact charging case slips into a "
"pocket and delivers three full recharges on the go."
),
"price": 89.99,
"rating": 4.3,
"review_count": 942,
"stock": 130,
"release_year": 2023,
"features": ["wireless", "bluetooth", "in-ear", "water-resistant"],
"specs": {"color": "pearl white", "weight_grams": 5, "warranty_years": 1},
},
{
"name": "Sonus Boom Portable Speaker",
"brand": "Sonus",
"category": "Audio",
"description": (
"A rugged portable Bluetooth speaker with deep bass and a waterproof "
"shell. Toss it in a bag for the beach or a campsite and enjoy "
"room-filling sound for up to 20 hours per charge."
),
"price": 129.5,
"rating": 4.5,
"review_count": 512,
"stock": 64,
"release_year": 2024,
"features": ["wireless", "bluetooth", "portable", "waterproof"],
"specs": {"color": "slate gray", "weight_grams": 540, "warranty_years": 1},
},
{
"name": "Pixma Vortex 15 Laptop",
"brand": "Pixma",
"category": "Computers",
"description": (
"A thin-and-light 15-inch laptop with a fast multi-core processor, "
"16 GB of memory, and a speedy solid-state drive. The backlit keyboard "
"and bright display make it a capable companion for work and study."
),
"price": 1399.0,
"rating": 4.7,
"review_count": 318,
"stock": 18,
"release_year": 2024,
"features": ["laptop", "ssd", "backlit-keyboard", "lightweight"],
"specs": {"color": "space silver", "weight_grams": 1600, "warranty_years": 2},
},
{
"name": "Pixma UltraView 27 Monitor",
"brand": "Pixma",
"category": "Computers",
"description": (
"A 27-inch 4K monitor with an IPS panel for accurate colors and wide "
"viewing angles. A single USB-C cable carries video and power, keeping "
"your desk tidy."
),
"price": 329.99,
"rating": 4.4,
"review_count": 221,
"stock": 27,
"release_year": 2023,
"features": ["monitor", "4k", "ips", "usb-c"],
"specs": {"color": "black", "weight_grams": 5200, "warranty_years": 3},
},
{
"name": "Clackr Mechanical Keyboard",
"brand": "Clackr",
"category": "Accessories",
"description": (
"A compact mechanical keyboard with tactile switches, per-key RGB "
"lighting, and wireless connectivity. Hot-swappable switches let you "
"tune the typing feel without soldering."
),
"price": 119.0,
"rating": 4.8,
"review_count": 1502,
"stock": 88,
"release_year": 2024,
"features": ["keyboard", "mechanical", "rgb", "wireless"],
"specs": {"color": "graphite", "weight_grams": 720, "warranty_years": 2},
},
{
"name": "Glide Pro Wireless Mouse",
"brand": "Glide",
"category": "Accessories",
"description": (
"An ergonomic wireless mouse with a high-precision sensor and a "
"contoured shape that reduces wrist strain. A single charge lasts for "
"weeks of everyday use."
),
"price": 59.99,
"rating": 4.2,
"review_count": 869,
"stock": 150,
"release_year": 2022,
"features": ["mouse", "wireless", "ergonomic"],
"specs": {"color": "charcoal", "weight_grams": 98, "warranty_years": 1},
},
{
"name": "Pulse Series 6 Smartwatch",
"brand": "Pulse",
"category": "Wearables",
"description": (
"A sleek smartwatch with built-in GPS, continuous heart-rate "
"monitoring, and water resistance for swimming. Track workouts, sleep, "
"and notifications from your wrist."
),
"price": 249.0,
"rating": 4.5,
"review_count": 1733,
"stock": 51,
"release_year": 2024,
"features": ["smartwatch", "gps", "heart-rate", "water-resistant"],
"specs": {"color": "rose gold", "weight_grams": 38, "warranty_years": 1},
},
{
"name": "Pulse Band Fitness Tracker",
"brand": "Pulse",
"category": "Wearables",
"description": (
"A lightweight fitness band that tracks steps, heart rate, and sleep "
"stages. The slim screen shows daily progress and the battery lasts a "
"full week between charges."
),
"price": 79.99,
"rating": 4.1,
"review_count": 2210,
"stock": 200,
"release_year": 2023,
"features": ["fitness-tracker", "heart-rate", "sleep-tracking"],
"specs": {"color": "ocean blue", "weight_grams": 24, "warranty_years": 1},
},
{
"name": "Lumi Glow Smart Bulb",
"brand": "Lumi",
"category": "Home",
"description": (
"A color-changing smart bulb that connects over Wi-Fi and works with "
"voice assistants. Dim it for movie night or set a warm white for "
"reading, all from your phone."
),
"price": 24.99,
"rating": 4.0,
"review_count": 640,
"stock": 320,
"release_year": 2022,
"features": ["smart-home", "wifi", "dimmable", "color"],
"specs": {"color": "white", "weight_grams": 70, "warranty_years": 2},
},
{
"name": "Lumi Climate Smart Thermostat",
"brand": "Lumi",
"category": "Home",
"description": (
"A learning smart thermostat that adjusts heating and cooling to your "
"routine and helps lower energy bills. The crisp display and Wi-Fi app "
"make scheduling effortless."
),
"price": 149.0,
"rating": 4.6,
"review_count": 388,
"stock": 75,
"release_year": 2024,
"features": ["smart-home", "wifi", "energy-saving"],
"specs": {"color": "white", "weight_grams": 210, "warranty_years": 3},
},
{
"name": "Vista Action Cam 4K",
"brand": "Vista",
"category": "Cameras",
"description": (
"A pocket-sized action camera that shoots stabilized 4K video and is "
"waterproof without a case. Mount it on a helmet or bike and capture "
"your adventures in sharp detail."
),
"price": 299.0,
"rating": 4.3,
"review_count": 455,
"stock": 33,
"release_year": 2023,
"features": ["camera", "4k", "waterproof", "wifi"],
"specs": {"color": "black", "weight_grams": 128, "warranty_years": 1},
},
]
for product_id, product in enumerate(catalog, start=1):
r.json().set(f"product:{product_id}", Path.root_path(), product)
You can read any single document back by its key with JSON.GET:
Foundational: Read one JSON document back by its key with JSON.GET
res = r.json().get("product:1", "$.name")
print(res) # >>> ['Aurora AcousticPro Headphones']
At this point the data is in Redis, but you can only fetch it one key at a time. To search across all products, you need an index.
Next steps
Continue to creating an index to make this data searchable.