Aggregate your data
Use FT.AGGREGATE to group, summarize, and transform your data with GROUPBY, REDUCE, and APPLY.
This is step 4 of the Redis Search tutorial. It builds on the index and the queries from the previous step.
FT.SEARCH answers "which records match?". Often you want to answer a different kind of question:
- How many products are in each category?
- What is the average price per category?
- Which brand has the highest average rating?
These are aggregation questions. They summarize across many documents instead of returning them one by one. The FT.AGGREGATE command handles them by running your results through a pipeline of steps. The three you will use most are:
GROUPBY— collect documents into groups that share a field value.REDUCE— compute something for each group, such as a count or an average.APPLY— calculate a new value from existing fields.
Count documents per group
The most common aggregation is a grouped count. This groups every product by category and counts how many fall into each. REDUCE COUNT 0 counts the documents in each group, and AS count names the result:
req = aggregations.AggregateRequest("*").group_by(
"@category", reducers.count().alias("count")
)
res = index.aggregate(req).rows
print(res)
# >>> [['category', 'Audio', 'count', '3'], ['category', 'Computers', 'count', '2'], ...]
"""
Code samples for the search and query tutorial:
https://redis.io/docs/latest/develop/get-started/search-tutorial/
"""
import json
import redis
import redis.commands.search.aggregation as aggregations
import redis.commands.search.reducers as reducers
from redis.commands.json.path import Path
from redis.commands.search.field import (
NumericField,
TagField,
TextField,
VectorField,
)
from redis.commands.search.index_definition import IndexDefinition, IndexType
from redis.commands.search.query import Query
r = redis.Redis(host="localhost", port=6379, db=0, decode_responses=True)
r.hset(
"product:1",
mapping={
"name": "Aurora AcousticPro Headphones",
"brand": "Aurora",
"category": "Audio",
"price": 199.99,
"rating": 4.6,
},
)
# >>> 5
r.json().set(
"product:1",
Path.root_path(),
{
"name": "Aurora AcousticPro Headphones",
"brand": "Aurora",
"category": "Audio",
"price": 199.99,
"rating": 4.6,
"features": ["wireless", "noise-cancelling", "bluetooth"],
"specs": {"color": "midnight black", "weight_grams": 268},
},
)
# >>> True
catalog = [
{
"name": "Aurora AcousticPro Headphones",
"brand": "Aurora",
"category": "Audio",
"description": (
"Over-ear wireless headphones with active noise cancelling and a "
"40-hour battery. Plush memory-foam earcups and a lightweight frame "
"make them comfortable for all-day listening, whether you are "
"commuting, working, or relaxing at home."
),
"price": 199.99,
"rating": 4.6,
"review_count": 1284,
"stock": 42,
"release_year": 2024,
"features": ["wireless", "noise-cancelling", "bluetooth", "over-ear"],
"specs": {"color": "midnight black", "weight_grams": 268, "warranty_years": 2},
},
{
"name": "Aurora BudsMini Earbuds",
"brand": "Aurora",
"category": "Audio",
"description": (
"Tiny true-wireless earbuds with a secure in-ear fit and sweat "
"resistance for workouts. The compact charging case slips into a "
"pocket and delivers three full recharges on the go."
),
"price": 89.99,
"rating": 4.3,
"review_count": 942,
"stock": 130,
"release_year": 2023,
"features": ["wireless", "bluetooth", "in-ear", "water-resistant"],
"specs": {"color": "pearl white", "weight_grams": 5, "warranty_years": 1},
},
{
"name": "Sonus Boom Portable Speaker",
"brand": "Sonus",
"category": "Audio",
"description": (
"A rugged portable Bluetooth speaker with deep bass and a waterproof "
"shell. Toss it in a bag for the beach or a campsite and enjoy "
"room-filling sound for up to 20 hours per charge."
),
"price": 129.5,
"rating": 4.5,
"review_count": 512,
"stock": 64,
"release_year": 2024,
"features": ["wireless", "bluetooth", "portable", "waterproof"],
"specs": {"color": "slate gray", "weight_grams": 540, "warranty_years": 1},
},
{
"name": "Pixma Vortex 15 Laptop",
"brand": "Pixma",
"category": "Computers",
"description": (
"A thin-and-light 15-inch laptop with a fast multi-core processor, "
"16 GB of memory, and a speedy solid-state drive. The backlit keyboard "
"and bright display make it a capable companion for work and study."
),
"price": 1399.0,
"rating": 4.7,
"review_count": 318,
"stock": 18,
"release_year": 2024,
"features": ["laptop", "ssd", "backlit-keyboard", "lightweight"],
"specs": {"color": "space silver", "weight_grams": 1600, "warranty_years": 2},
},
{
"name": "Pixma UltraView 27 Monitor",
"brand": "Pixma",
"category": "Computers",
"description": (
"A 27-inch 4K monitor with an IPS panel for accurate colors and wide "
"viewing angles. A single USB-C cable carries video and power, keeping "
"your desk tidy."
),
"price": 329.99,
"rating": 4.4,
"review_count": 221,
"stock": 27,
"release_year": 2023,
"features": ["monitor", "4k", "ips", "usb-c"],
"specs": {"color": "black", "weight_grams": 5200, "warranty_years": 3},
},
{
"name": "Clackr Mechanical Keyboard",
"brand": "Clackr",
"category": "Accessories",
"description": (
"A compact mechanical keyboard with tactile switches, per-key RGB "
"lighting, and wireless connectivity. Hot-swappable switches let you "
"tune the typing feel without soldering."
),
"price": 119.0,
"rating": 4.8,
"review_count": 1502,
"stock": 88,
"release_year": 2024,
"features": ["keyboard", "mechanical", "rgb", "wireless"],
"specs": {"color": "graphite", "weight_grams": 720, "warranty_years": 2},
},
{
"name": "Glide Pro Wireless Mouse",
"brand": "Glide",
"category": "Accessories",
"description": (
"An ergonomic wireless mouse with a high-precision sensor and a "
"contoured shape that reduces wrist strain. A single charge lasts for "
"weeks of everyday use."
),
"price": 59.99,
"rating": 4.2,
"review_count": 869,
"stock": 150,
"release_year": 2022,
"features": ["mouse", "wireless", "ergonomic"],
"specs": {"color": "charcoal", "weight_grams": 98, "warranty_years": 1},
},
{
"name": "Pulse Series 6 Smartwatch",
"brand": "Pulse",
"category": "Wearables",
"description": (
"A sleek smartwatch with built-in GPS, continuous heart-rate "
"monitoring, and water resistance for swimming. Track workouts, sleep, "
"and notifications from your wrist."
),
"price": 249.0,
"rating": 4.5,
"review_count": 1733,
"stock": 51,
"release_year": 2024,
"features": ["smartwatch", "gps", "heart-rate", "water-resistant"],
"specs": {"color": "rose gold", "weight_grams": 38, "warranty_years": 1},
},
{
"name": "Pulse Band Fitness Tracker",
"brand": "Pulse",
"category": "Wearables",
"description": (
"A lightweight fitness band that tracks steps, heart rate, and sleep "
"stages. The slim screen shows daily progress and the battery lasts a "
"full week between charges."
),
"price": 79.99,
"rating": 4.1,
"review_count": 2210,
"stock": 200,
"release_year": 2023,
"features": ["fitness-tracker", "heart-rate", "sleep-tracking"],
"specs": {"color": "ocean blue", "weight_grams": 24, "warranty_years": 1},
},
{
"name": "Lumi Glow Smart Bulb",
"brand": "Lumi",
"category": "Home",
"description": (
"A color-changing smart bulb that connects over Wi-Fi and works with "
"voice assistants. Dim it for movie night or set a warm white for "
"reading, all from your phone."
),
"price": 24.99,
"rating": 4.0,
"review_count": 640,
"stock": 320,
"release_year": 2022,
"features": ["smart-home", "wifi", "dimmable", "color"],
"specs": {"color": "white", "weight_grams": 70, "warranty_years": 2},
},
{
"name": "Lumi Climate Smart Thermostat",
"brand": "Lumi",
"category": "Home",
"description": (
"A learning smart thermostat that adjusts heating and cooling to your "
"routine and helps lower energy bills. The crisp display and Wi-Fi app "
"make scheduling effortless."
),
"price": 149.0,
"rating": 4.6,
"review_count": 388,
"stock": 75,
"release_year": 2024,
"features": ["smart-home", "wifi", "energy-saving"],
"specs": {"color": "white", "weight_grams": 210, "warranty_years": 3},
},
{
"name": "Vista Action Cam 4K",
"brand": "Vista",
"category": "Cameras",
"description": (
"A pocket-sized action camera that shoots stabilized 4K video and is "
"waterproof without a case. Mount it on a helmet or bike and capture "
"your adventures in sharp detail."
),
"price": 299.0,
"rating": 4.3,
"review_count": 455,
"stock": 33,
"release_year": 2023,
"features": ["camera", "4k", "waterproof", "wifi"],
"specs": {"color": "black", "weight_grams": 128, "warranty_years": 1},
},
]
for product_id, product in enumerate(catalog, start=1):
r.json().set(f"product:{product_id}", Path.root_path(), product)
res = r.json().get("product:1", "$.name")
print(res) # >>> ['Aurora AcousticPro Headphones']
schema = (
TextField("$.name", as_name="name"),
TagField("$.brand", as_name="brand", sortable=True),
TagField("$.category", as_name="category"),
TextField("$.description", as_name="description"),
NumericField("$.price", as_name="price", sortable=True),
NumericField("$.rating", as_name="rating", sortable=True),
NumericField("$.review_count", as_name="review_count"),
NumericField("$.stock", as_name="stock"),
NumericField("$.release_year", as_name="release_year", sortable=True),
TagField("$.features[*]", as_name="features"),
)
index = r.ft("idx:catalog")
index.create_index(
schema,
definition=IndexDefinition(prefix=["product:"], index_type=IndexType.JSON),
)
info = r.ft("idx:catalog").info()
print("Documents indexed:", info["num_docs"]) # >>> Documents indexed: 12
res = index.search(Query("*").paging(0, 0))
print("Total products:", res.total) # >>> Total products: 12
res = index.search(Query("@name:headphones").return_field("name"))
print(res.docs)
# >>> [Document {'id': 'product:1', ... 'name': 'Aurora AcousticPro Headphones'}]
res = index.search(Query('@description:"noise cancelling"').return_field("name"))
print(res.total, [d.name for d in res.docs])
# >>> 1 ['Aurora AcousticPro Headphones']
res = index.search(Query("@category:{Audio}").return_fields("name", "price"))
print(res.total, [d.name for d in res.docs])
# >>> 3 ['Aurora BudsMini Earbuds', 'Sonus Boom Portable Speaker', ...]
res = index.search(Query("@features:{waterproof}").return_field("name"))
print(res.total, [d.name for d in res.docs])
# >>> 2 ['Sonus Boom Portable Speaker', 'Vista Action Cam 4K']
res = index.search(
Query("@price:[0 100]").sort_by("price", asc=True).return_fields("name", "price")
)
print([(d.name, d.price) for d in res.docs])
# >>> [('Lumi Glow Smart Bulb', '24.99'), ('Glide Pro Wireless Mouse', '59.99'), ...]
res = index.search(
Query("@category:{Audio} @price:[0 100]").return_fields("name", "price")
)
print(res.total, [d.name for d in res.docs])
# >>> 1 ['Aurora BudsMini Earbuds']
res = index.search(
Query("*").sort_by("price", asc=False).return_fields("name", "price").paging(0, 3)
)
print([(d.name, d.price) for d in res.docs])
# >>> [('Pixma Vortex 15 Laptop', '1399'), ('Pixma UltraView 27 Monitor', '329.99'), ...]
req = aggregations.AggregateRequest("*").group_by(
"@category", reducers.count().alias("count")
)
res = index.aggregate(req).rows
print(res)
# >>> [['category', 'Audio', 'count', '3'], ['category', 'Computers', 'count', '2'], ...]
req = (
aggregations.AggregateRequest("*")
.group_by("@category", reducers.avg("@price").alias("avg_price"))
.sort_by(aggregations.Desc("@avg_price"))
)
res = index.aggregate(req).rows
print(res)
# >>> [['category', 'Computers', 'avg_price', '864.495'], ...]
req = (
aggregations.AggregateRequest("@category:{Audio}")
.load("name", "price")
.apply(sale_price="@price - (@price * 0.1)")
)
res = index.aggregate(req).rows
print(res)
# >>> [['name', 'Aurora AcousticPro Headphones', 'price', '199.99', 'sale_price', '179.991'], ...]
req = (
aggregations.AggregateRequest("*")
.group_by("@brand", reducers.avg("@rating").alias("avg_rating"))
.sort_by(aggregations.Desc("@avg_rating"))
)
res = index.aggregate(req).rows
print(res)
# >>> [['brand', 'Clackr', 'avg_rating', '4.8'], ['brand', 'Pixma', 'avg_rating', '4.55'], ...]
from sentence_transformers import SentenceTransformer
embedder = SentenceTransformer("msmarco-distilbert-base-v4") # 768-dimensional vectors
for key in r.scan_iter(match="product:*"):
description = r.json().get(key, "$.description")[0]
embedding = embedder.encode(description).astype("float32").tolist()
r.json().set(key, "$.embedding", embedding)
r.ft("idx:catalog").dropindex()
schema = (
TextField("$.name", as_name="name"),
TagField("$.brand", as_name="brand", sortable=True),
TagField("$.category", as_name="category"),
TextField("$.description", as_name="description"),
NumericField("$.price", as_name="price", sortable=True),
NumericField("$.rating", as_name="rating", sortable=True),
TagField("$.features[*]", as_name="features"),
VectorField(
"$.embedding",
"FLAT",
{"TYPE": "FLOAT32", "DIM": 768, "DISTANCE_METRIC": "COSINE"},
as_name="embedding",
),
)
index = r.ft("idx:catalog")
index.create_index(
schema,
definition=IndexDefinition(prefix=["product:"], index_type=IndexType.JSON),
)
query_vector = (
embedder.encode("portable music for the outdoors").astype("float32").tobytes()
)
res = index.search(
Query("(*)=>[KNN 3 @embedding $query_vector AS score]")
.sort_by("score", asc=True)
.return_fields("score", "name")
.dialect(2),
query_params={"query_vector": query_vector},
)
print([d.name for d in res.docs])
# >>> ['Sonus Boom Portable Speaker', 'Aurora BudsMini Earbuds', 'Aurora AcousticPro Headphones']
res = index.search(
Query("(@category:{Audio})=>[KNN 3 @embedding $query_vector AS score]")
.sort_by("score", asc=True)
.return_field("name")
.dialect(2),
query_params={"query_vector": query_vector},
)
print([d.name for d in res.docs])
# >>> ['Sonus Boom Portable Speaker', 'Aurora BudsMini Earbuds', 'Aurora AcousticPro Headphones']
hybrid_vector = (
embedder.encode("wireless headphones for listening to music")
.astype("float32")
.tobytes()
)
res = r.execute_command(
"FT.HYBRID",
"idx:catalog",
"SEARCH",
"wireless",
"VSIM",
"@embedding",
"$query_vector",
"KNN",
"2",
"K",
"5",
"LOAD",
"1",
"@name",
"PARAMS",
"2",
"query_vector",
hybrid_vector,
)
print(res)
# >>> {'total_results': 7, 'results': [{'name': 'Aurora AcousticPro Headphones'}, ...]}
The "*" after the index name is a query expression, exactly like in FT.SEARCH. Here it means "aggregate over all documents", but you could narrow the input first, for example @price:[0 100] to aggregate only the cheaper products. GROUPBY 1 @category reads as "group by one field: category".
Compute an average per group
Swap COUNT for a different reducer to compute other summaries. This calculates the average price in each category and sorts the groups from most to least expensive with SORTBY:
req = (
aggregations.AggregateRequest("*")
.group_by("@category", reducers.avg("@price").alias("avg_price"))
.sort_by(aggregations.Desc("@avg_price"))
)
res = index.aggregate(req).rows
print(res)
# >>> [['category', 'Computers', 'avg_price', '864.495'], ...]
REDUCE AVG 1 @price reads as "apply the AVG reducer to one field: price". The SORTBY 2 @avg_price DESC clause sorts by the computed avg_price value; the 2 is the number of arguments that follow (@avg_price and DESC). Other reducers include SUM, MIN, MAX, and COUNT_DISTINCT; see the aggregation reference for the full list.
Calculate new values with APPLY
APPLY evaluates an expression against each record and adds the result as a new field. This takes the Audio products, loads their name and price, and computes a 10%-off sale_price:
req = (
aggregations.AggregateRequest("@category:{Audio}")
.load("name", "price")
.apply(sale_price="@price - (@price * 0.1)")
)
res = index.aggregate(req).rows
print(res)
# >>> [['name', 'Aurora AcousticPro Headphones', 'price', '199.99', 'sale_price', '179.991'], ...]
The LOAD 2 name price clause pulls those two fields into the pipeline so the expression can use them and so they appear in the output. APPLY does not group anything; it transforms each record in place.
Build a pipeline
The real power of FT.AGGREGATE is chaining these steps. This finds the average rating per brand and returns the highest-rated brands first — a simple "best brands" leaderboard:
req = (
aggregations.AggregateRequest("*")
.group_by("@brand", reducers.avg("@rating").alias("avg_rating"))
.sort_by(aggregations.Desc("@avg_rating"))
)
res = index.aggregate(req).rows
print(res)
# >>> [['brand', 'Clackr', 'avg_rating', '4.8'], ['brand', 'Pixma', 'avg_rating', '4.55'], ...]
(The output is truncated; eight brands are returned in all.) You can keep extending the pipeline — apply multiple reducers under one GROUPBY, chain a second GROUPBY, add FILTER and LIMIT steps, and more. See the aggregation queries guide for deeper examples.
FT.AGGREGATE command from this page into the query editor to see each group as a row.Next steps
You can now find, filter, and summarize structured data. The final step goes beyond keywords and exact values to search by meaning. Continue to vector and hybrid search.