Key and field expiration behavior

Summary: How Redis Search handles expiring keys and hash fields

Overview

The Redis Search behavior with expiring keys and hash fields has been enhanced starting with Redis 8 to provide more consistent and predictable results.

Key expiration

Expiration times

**Before Redis 8**: Expiration times were not taken into account when computing the result set.

**Redis 8 and later**: Redis Search returns only documents that are valid (not expired) at the time when the query or cursor read started.

Active expiration

Active expiration can affect the number of results returned. For example, if a user requests 100 documents but 10 are actively expired during query execution, only 90 documents will be returned.

**Note**: This behavior did not change in Redis 8 - active expiration has always affected result counts.

Passive expiration

**Before Redis 8**: A query could return `nil` as a document name in the result set for a key that was passively expired.

**Redis 8 and later**: Only valid document names will be returned. Passively expired keys are filtered out from the result set.

Field expiration

Field expiration was introduced in Redis 7.4 and provides fine-grained control over hash field lifecycles.

Expiration times

**Before Redis 8**: Field expiration times were not taken into account when computing the result set.

**Redis 8 and later**: Redis Search returns only documents that are valid (fields not expired) at the time when the query or cursor read started.

Active expiration

Similar to key expiration, active field expiration can affect the number of results returned. If fields that match query criteria are actively expired during execution, fewer results will be returned than requested.

**Note**: This behavior did not change in Redis 8 - active expiration has always affected result counts.

Passive expiration

**Before Redis 8**: Redis Search could return documents without fields that were passively expired, even if the expired field caused the document to match the query. This behavior depended to some degree on the `SORTABLE` keyword usage.

**Redis 8 and later**: Documents will return with all fields that existed when the query or cursor read started. Passively expired fields are handled consistently.

Best practices

* **Consistent timing**: Be aware that query results are based on the state at query start time, not when individual documents are processed.

* **Result count expectations**: Be aware that when you use expiring keys or fields, the actual number of results may be less than you expected due to active expiration during the execution of the query.

* **Field-level expiration**: Use hash field expiration (available since Redis 7.4) for fine-grained control over document field lifecycles without affecting the entire document.

* **Query planning**: Consider expiration patterns when designing queries, especially for time-sensitive applications where expired data should not appear in results.

Related commands

The following commands are directly related to key and field expiration:

- [`EXPIRE`]() - Set expiration time for keys
- [`HEXPIRE`]() - Set expiration time for hash fields
- [`FT.SEARCH`]() - Search queries affected by expiration behavior
- [`FT.AGGREGATE`]() - Aggregation queries affected by expiration behavior


Geospatial

Summary: Learn how to use geospatial fields and perform geospatial queries in Redis

Overview

Redis Search supports geospatial data. This feature
lets you store geographical locations and geometric shapes
in the fields of JSON objects.

Take care not to confuse the geospatial indexing
features in Redis Search with the
[Geospatial data type]()
that Redis also supports. Although there are some similarities between
these two features, the data type is intended for simpler use
cases and doesn't have the range of format options and queries
available in Redis Search.

You can index these fields and use queries to find the objects
by their location or the relationship of their shape to other shapes.
For example, if you add the locations of a set of shops, you can
find all the shops within 5km of a user's position or determine
which ones are within the boundary of a particular town.

Redis uses coordinate points to represent geospatial locations.
You can store individual points but you can also
use a set of points to define a polygon shape (the shape of a
town, for example). You can query several types of interactions
between points and shapes, such as whether a point lies within
a shape or whether two shapes overlap.

Redis can interpret coordinates either as geographical longitude
and latitude or as Cartesian coordinates on a flat plane.
Geographical coordinates are ideal for large real-world locations
and areas (such as towns and countries). Cartesian coordinates
are more suitable for smaller areas (such as rooms in a building)
or for games, simulations, and other artificial scenarios.

Storing geospatial data

Redis supports two different
[schema types]()
for geospatial data:

-   [`GEO`](#geo): This uses a simple format where individual geospatial
    points are specified as numeric longitude-latitude pairs.

-   [`GEOSHAPE`](#geoshape): [Redis Open Source]() also
    supports `GEOSHAPE` indexing in v7.2 and later.
    This uses a subset of the
    [Well-Known Text (WKT)](https://en.wikipedia.org/wiki/Well-known_text_representation_of_geometry)
    format to specify both points and polygons using either geographical
    coordinates or Cartesian coordinates. A
    `GEOSHAPE` field supports more advanced queries than `GEO`,
    such as checking if one shape overlaps or contains another.

The sections below describe these schema types in more detail.

`GEO`

A `GEO` index lets you represent geospatial data either as
a string containing a longitude-latitude pair (for example,
"-104.991531, 39.742043") or as a JSON array of these
strings. Note that the longitude value comes first in the
string.

For example, you could index the `location` fields of the
the [JSON]() objects
shown below as `GEO`:

[code example]

`GEO` fields allow only basic point and radius queries.
For example, the query below finds products within a 100 mile radius of Colorado Springs
(Longitude=-104.800644, Latitude=38.846127).

[code example]

Note that `GEO` fields are stored in C `double` type variables and they are limited to approximately 15 to 17 digits of precision.

See [Geospatial queries]()
for more information about the available query options and see
[Geospatial indexing]()
for examples of indexing `GEO` fields.

`GEOSHAPE`

Fields indexed as `GEOSHAPE` support the `POINT` and `POLYGON` primitives from the
[Well-Known Text](https://en.wikipedia.org/wiki/Well-known_text_representation_of_geometry)
representation of geometry. The `POINT` primitive defines a single point
in a similar way to a `GEO` field.
The `geom` field of the example JSON object shown below specifies a point
(in Cartesian coordinates, using the standard x,y order):

[code example]

The `POLYGON` primitive can approximate the outline of any shape using a
sequence of points. Specify the coordinates of the corners in the order they
occur around the shape (either clockwise or counter-clockwise) and ensure the
shape is "closed" by making the final coordinate exactly the same as the first.

Note that `POLYGON` requires double parentheses around the coordinate list.
This is because you can specify additional shapes as a comma-separated list
that define "holes" within the enclosing polygon. The holes must have the opposite
winding order to the outer polygon (so, if the outer polygon uses a clockwise winding
order, the holes must use counter-clockwise).
The `geom` field of the example JSON object shown below specifies a
square using Cartesian coordinates in a clockwise winding order:

[code example]

The following examples define one `POINT` and three `POLYGON` primitives,
which are shown in the image below:

[code example]

You can run various types of queries against a geospatial index. For
example, the query below returns one primitive that lies within the boundary
of the green square (from the example above) but omits the square itself:

[code example]

There are four query operations that you can use with `GEOSHAPE` fields:

-   `WITHIN`: Find points or shapes that lie entirely within an
    enclosing shape that you specify in the query.
-   `CONTAINS`: Find shapes that completely contain the specified point
    or shape.
-   `INTERSECTS`: Find shapes whose boundary overlaps another specified
    shape.
-   `DISJOINT`: Find shapes whose boundary does not overlap another specified
    shape.

See
[Geospatial queries]()
for more information about these query types and see
[Geospatial indexing]()
for examples of indexing `GEOSHAPE` fields.

Limitations of geographical coordinates

Planet Earth is actually shaped more like an
[ellipsoid](https://en.wikipedia.org/wiki/Earth_ellipsoid) than a perfect sphere.
The spherical coordinate system used by Redis Search is a close
approximation to the shape of the Earth but not exact. For most practical
uses of geospatial queries, the approximation works very well, but you
shouldn't rely on it if you need very precise location data (for example, to track
the GPS locations of boats in an emergency response system).

> FT.SEARCH geomidx "(-@name:(Green Square) @geom:[WITHIN $qshape])" PARAMS 2 qshape "POLYGON ((1 1, 1 3, 3 3, 3 1, 1 1))" RETURN 1 name DIALECT 2

1) (integer) 1
2) "shape:4"
3) 1) "name"
   2) "[\"Purple Point\"]"


Highlighting

Summary: Highlighting full-text results

Overview

Redis Open Source uses advanced algorithms for highlighting and summarizing, which enable only the relevant portions of a document to appear in response to a search query. This feature allows users to immediately understand the relevance of a document to their search criteria, typically highlighting the matching terms in bold text.

Command syntax

[code example]

There are two sub-commands used for highlighting. The first is `HIGHLIGHT`, which surrounds matching text with an open and/or close tag. The second is `SUMMARIZE`, which splits a field into contextual fragments surrounding the found terms. It is possible to summarize a field, highlight a field, or perform both actions in the same query.

Summarization

[code example]

Summarization will fragment the text into smaller sized snippets, each of which containing the found term(s) and some additional surrounding context.

Redis can perform summarization using the `SUMMARIZE` keyword. If no additional arguments are passed, all returned fields are summarized using built-in defaults.

The `SUMMARIZE` keyword accepts the following arguments:

* **`FIELDS`**: If present, it must be the first argument. This should be followed
    by the number of fields to summarize, which itself is followed by a list of
    fields. Each field is summarized. If no `FIELDS` directive is passed,
    then all returned fields are summarized.

* **`FRAGS`**: The number of fragments to be returned. If not specified, a default is 3.

* **`LEN`**: The number of context words each fragment should contain. Context
    words surround the found term. A higher value will return a larger block of
    text. If not specified, the default value is 20.

* **`SEPARATOR`**: The string used to divide individual summary snippets.
    The default is `... ` which is common among search engines, but you may
    override this with any other string if you desire to programmatically divide the snippets
    later on. You may also use a newline sequence, as newlines are stripped from the
    result body during processing.

Highlighting

[code example]

Highlighting will surround the found term (and its variants) with a user-defined pair of tags. This may be used to display the matched text in a different typeface using a markup language, or to otherwise make the text appear differently.

Redis performs highlighting using the `HIGHLIGHT` keyword. If no additional arguments are passed, all returned fields are highlighted using built-in defaults.

The `HIGHLIGHT` keyword accepts the following arguments:

* **`FIELDS`**: If present, it must be the first argument. This should be followed
    by the number of fields to highlight, which itself is followed by a list of
    fields. Each field present is highlighted. If no `FIELDS` directive is passed,
    then all returned fields are highlighted.

* **`TAGS`**: If present, it must be followed by two strings. The first string is prepended
    to each matched term. The second string is appended to each matched term. If no `TAGS` are
    specified, a built-in tag pair is prepended and appended to each matched term.

#### Field selection

If no specific fields are passed to the `RETURN`, `SUMMARIZE`, or `HIGHLIGHT` keywords, then all of a document's fields are returned. However, if any of these keywords contain a `FIELD` directive, then the `SEARCH` command will only return the sum total of all fields enumerated in any of those directives.

The `RETURN` keyword is treated specially, as it overrides any fields specified in `SUMMARIZE` or `HIGHLIGHT`.

In the command `RETURN 1 foo SUMMARIZE FIELDS 1 bar HIGHLIGHT FIELDS 1 baz`, the fields `foo` is returned as-is, while `bar` and `baz` are not returned, because `RETURN` was specified, but did not include those fields.

In the command `SUMMARIZE FIELDS 1 bar HIGHLIGHT FIELDS 1 baz`, `bar` is returned summarized and `baz` is returned highlighted.

FT.SEARCH ...
    SUMMARIZE [FIELDS {num} {field}] [FRAGS {numFrags}] [LEN {fragLen}] [SEPARATOR {sepstr}]
    HIGHLIGHT [FIELDS {num} {field}] [TAGS {openTag} {closeTag}]

FT.SEARCH ...
    SUMMARIZE [FIELDS {num} {field}] [FRAGS {numFrags}] [LEN {fragLen}] [SEPARATOR {sepStr}]

FT.SEARCH ... HIGHLIGHT [FIELDS {num} {field}] [TAGS {openTag} {closeTag}]


Search concepts

Summary: Details about query syntax, aggregation, scoring, and other search and query options

Summary: Groupings, projections, and aggregation functions

Aggregations

Summary: Order of operations for the FT.AGGREGATE command

FT.AGGREGATE order of operations

Summary: Learn how to use the autocomplete feature of Redis for efficient prefix-based suggestion retrieval.

Autocomplete with Redis

Summary: Controlling text tokenization and escaping

Tokenization

Summary: Stop words support

Stop words

Summary: Learn how to use query dialects

Query dialects

Summary: Learn how to use query syntax

Query syntax

Summary: Learn how to use query attributes

Query attributes

Summary: Support for sorting query results

Sorting by indexed fields

Summary: How to use tag fields for exact match searches and high-performance filtering

Tag fields

Summary: Highlighting full-text results

Highlighting

Key and field expiration behavior

Summary: Full-text scoring functions

Scoring documents

Summary: Stemming support

Stemming

Summary: Details on synonym support with Redis Open Source

Synonym support

Summary: Query spelling correction support

Spellchecking

Geospatial

Summary: Details about phonetic matching capabilities

Phonetic matching

Summary: Chinese support for searching and querying in Redis Open Source

Chinese support

Overview

Redis Open Source supports the following Redis Search features. This article provides you an overview.

Indexing features

* Secondary indexing
* Vector indexing
* Index on [JSON]() documents
* Full-text indexing of multiple fields in a document
* Incremental indexing without performance loss
* Document deletion and updating with index garbage collection

Query features

* Multi-field queries
* Query on [JSON]() documents
* [Aggregation]()
* Boolean queries with AND, OR, and NOT operators between subqueries
* Optional query clauses
* Retrieval of full document contents or only their IDs
* Exact phrase search and slop-based search
* Numeric filters and ranges
* Geo-filtering using Redis [geo commands](?group=geo)
* [Vector search]()
* [Key and field expiration behavior]()

Full-text search features

* [Prefix-based searches]()
* Field weights
* [Auto-complete]() and fuzzy prefix suggestions
* [Stemming]()-based query expansion for [many languages]() using [Snowball](http://snowballstem.org/)
* Support for custom functions for query expansion and scoring (see [Extensions]())
* Unicode support (UTF-8 input required)
* Document ranking

Cluster support

The Redis Search features of Redis Open Source are also available for distributed databases that can scale to billions of documents and hundreds of servers.

Supported platforms

Redis Open Source is developed and tested on Linux and macOS on x86_64 CPUs.

Atom CPUs are not supported.

<br/>


Phonetic matching

Overview

Phonetic matching, for example "Jon" vs. "John", allows searching for terms based on their pronunciation. This capability can be a useful tool when searching for names of people.

Phonetic matching is based on the use of a phonetic algorithm. A phonetic algorithm transforms the input term to an approximate representation of its pronunciation. This allows terms to be indexed and searched by their pronunciation.

As of v1.4, Redis Search, which is included in Redis Open Source, provides phonetic matching of text fields specified with the `PHONETIC` attribute. This causes the terms in such fields to be indexed both by their textual value as well as their phonetic approximation.

Performing a search on `PHONETIC` fields will, by default, also return results for phonetically similar terms. This behavior can be controlled with the [`$phonetic` query attribute]().

Phonetic algorithms support

Redis currently supports a single phonetic algorithm, the [Double Metaphone](https://en.wikipedia.org/wiki/Metaphone#Double_Metaphone) (DM). It uses the implementation at the [slacy/double-metaphone GitHub site](https://github.com/slacy/double-metaphone), which provides general support for Latin languages.


Query attributes

Summary: Learn how to use query attributes

Query attributes

As of v1.2.0, you can apply specific query modifying attributes to specific clauses of the query.

The syntax is `(foo bar) => { $attribute: value; $attribute:value; ...}`:

[code example]

The supported attributes are:

1. **$weight**: determines the weight of the sub-query or token in the overall ranking on the result (default: 1.0).
2. **$slop**: determines the maximum allowed slop (space between terms) in the query clause (default: 0).
3. **$inorder**: whether or not the terms in a query clause must appear in the same order as in the query. This is usually set alongside with `$slop` (default: false).
4. **$phonetic**: whether or not to perform phonetic matching (default: true). Note: setting this attribute to true for fields which were not created as `PHONETIC` will produce an error.

As of v2.6.1, the query attributes syntax supports these additional attributes:

* **$yield_distance_as**: specifies the distance field name for clauses that yield some distance metric. This is used for later sorting and/or returning. It is currently supported for vector queries only (both KNN and range).
* **$shard_k_ratio**: controls how many results each shard retrieves relative to the requested top_k in cluster setups. Value range: 0.1 - 1.0 (default: 1.0). Only applicable to vector KNN queries in Redis cluster environments. See [Cluster-specific query parameters]() for detailed information.
* **vector query params**: pass optional parameters for [vector queries]() in key-value format.

(foo bar) => { $weight: 2.0; $slop: 1; $inorder: true; }
~(bar baz) => { $weight: 0.5; }


Query syntax

Summary: Learn how to use query syntax

Overview

The query syntax that RediSearch uses has improved over time,
adding new features and making queries simpler to write. However,
changing the syntax like this could potentially break existing queries that rely on
an older version of the syntax. To avoid this problem, RediSearch supports
different query syntax *dialects* to ensure backward compatibility.
Any breaking changes to the syntax are introduced in a new dialect, while
RediSearch continues to support older dialects. This means you can always choose
the correct dialect to support the query you are using.
See
[Query dialects]()
for full details of the dialects and the RediSearch versions that introduced them.

Basic syntax

You can use simple syntax for complex queries using these rules:

* Exact phrases are wrapped in quotes, for example, `"hello world"`.
* Multiword phrases are lists of tokens, for example, `foo bar baz`, and imply intersection (AND) of the terms.
* `OR` unions are expressed with a pipe (`|`) character, for example, `hello|hallo|shalom|hola`.

   **Notes**:

   Consider the differences in parser behavior in example `hello world | "goodbye" moon`:
   * In DIALECT 1, this query is interpreted as searching for `(hello world | "goodbye") moon`.
   * In DIALECT 2 or greater, this query is interpreted as searching for either `hello world` **OR** `"goodbye" moon`.

* `NOT` negation of expressions or subqueries is expressed with a subtraction symbol (`-`), for example, `hello -world`. Purely negative queries such as `-foo` and `-@title:(foo|bar)` are also supported.

   **Notes**:

   Consider a simple query with negation `-hello world`:
   * In DIALECT 1, this query is interpreted as "find values in any field that does not contain `hello` **AND** does not contain `world`". The equivalent is `-(hello world)` or `-hello -world`.
   * In DIALECT 2 or greater, this query is interpreted `as -hello` **AND** `world` (only `hello` is negated).
   * In DIALECT 2 or greater, to achieve the default behavior of DIALECT 1, update your query to `-(hello world)`.

* Prefix/infix/suffix matches (all terms starting/containing/ending with a term) are expressed with an asterisk `*`. For performance reasons, a minimum term length is enforced. The default is 2, but it's configurable.
* In DIALECT 2 or greater, wildcard pattern matches are expressed as `"w'foo*bar?'"`. Note the use of double quotes to contain the _w_ pattern.
* A special wildcard query that returns all results in the index is just the asterisk `*`. This cannot be combined with other options.
* As of v2.6.1, `DIALECT 3` returns JSON rather than scalars from multivalue attributes.
* Selection of specific fields using the syntax `hello @field:world`.
* Numeric range matches on numeric fields with the syntax `@field:[{min} {max}]`.
* Georadius matches on geo fields with the syntax `@field:[{lon} {lat} {radius} {m|km|mi|ft}]`.
* As of 2.6, range queries on vector fields with the syntax `@field:[VECTOR_RANGE {radius} $query_vec]`, where `query_vec` is given as a query parameter.
* As of v2.4, k-nearest neighbors (KNN) queries on vector fields with or without pre-filtering with the syntax `{filter_query}=>[KNN {num} @field $query_vec]`.
* Tag field filters with the syntax `@field:{tag | tag | ...}`. See the full documentation on [tags]().
* Optional terms or clauses: `foo ~bar` means bar is optional but documents containing `bar` will rank higher.
* Fuzzy matching on terms: `%hello%` means all terms with Levenshtein distance of 1 from it. Use multiple pairs of '%' brackets, up to three deep, to increase the Levenshtein distance.
* An expression in a query can be wrapped in parentheses to disambiguate, for example, `(hello|hella) (world|werld)`.
* Query attributes can be applied to individual clauses, for example, `(foo bar) => { $weight: 2.0; $slop: 1; $inorder: false; }`.
* Combinations of the above can be used together, for example, `hello (world|foo) "bar baz" bbbb`.

Pure negative queries

As of v0.19.3, it is possible to have a query consisting of just a negative expression. For example `-hello` or `-(@title:(foo|bar))`. The results are all the documents not containing the query terms.

Any complex expression can be negated this way, however, caution should be taken here: if a negative expression has little or no results, this is equivalent to traversing and ranking all the documents in the index, which can be slow and cause high CPU consumption.

Field modifiers

You can specify field modifiers in a query, and not just by using the `INFIELDS` global keyword.

To specify which fields the query matches, prepend the expression with the `@` symbol, the field name, and a `:` (colon) symbol, for each expression or subexpression.

If a field modifier precedes multiple words or expressions, it applies only to the adjacent expression with DIALECT 1. With DIALECT 2 or greater, you extend the query to other fields.

Consider this simple query: `@name:James Brown`. Here, the field modifier `@name` is followed by two words: `James` and `Brown`.

* In DIALECT 1, this query would be interpreted as "find `James Brown` in the `@name` field".
* In DIALECT 2 or greater, this query will be interpreted as "find `James` in the `@name` field **AND** `Brown` in **ANY** text field. In other words, it would be interpreted as `(@name:James) Brown`.
* In DIALECT 2 or greater, to achieve the default behavior of DIALECT 1, update your query to `@name:(James Brown)`.

If a field modifier precedes an expression in parentheses, it applies only to the expression inside the parentheses. The expression should be valid for the specified field, otherwise it is skipped.

To create complex filtering on several fields, you can combine multiple modifiers. For example, if you have an index of car models, with a vehicle class, country of origin, and engine type, you can search for SUVs made in Korea with hybrid or diesel engines using the following query:

[code example]

You can apply multiple modifiers to the same term or grouped terms:

[code example]

Now, you search for documents that have `"hello"` and `"world"` either in the body or the title and the term `mydomain` in their `url` or `image` fields.

Numeric filters in query

If a field in the schema is defined as NUMERIC, it is possible to use the FILTER argument in the Redis request or filter with it by specifying filtering rules in the query. The syntax is `@field:[{min} {max}]`, for example, `@price:[100 200]`.

A few notes on numeric predicates

1. It is possible to specify a numeric predicate as the entire query, whereas it is impossible to do it with the `FILTER` argument.

2. It is possible to intersect or union multiple numeric filters in the same query, be it for the same field or different ones.

3. `-inf`, `inf` and `+inf` are acceptable numbers in a range. Therefore, _greater than 100_ is expressed as `[(100 inf]`.

4. Numeric filters are inclusive. Exclusive min or max are expressed with `(` prepended to the number, for example, `[(100 (200]`.

5. It is possible to negate a numeric filter by prepending a `-` sign to the filter. For example, returning a result where price differs from 100 is expressed as: `@title:foo -@price:[100 100]`.

Tag filters

As of v0.91, you can use a special field type called a
[_tag field_](), with simpler
[tokenization]()
and encoding in the index. You can't access the values in these fields using a general fieldless search. Instead, you use special syntax:

[code example]

Example:

[code example]

Tags can have multiple words or include other punctuation marks other than the field's separator (`,` by default). The following characters in tags should be escaped with a backslash (`\`): `$`, `{`, `}`, `\`, and `|`.

Before RediSearch 2.4, it was also recommended to escape spaces. The reason was that, if a multiword tag included stopwords, a syntax error was returned. So tags, like "to be or not to be" needed be escaped as "to\ be\ or\ not\ to\ be". For good measure, you also could escape all spaces within tags. Starting with RediSearch 2.4, using [`DIALECT 2`]() or greater you can use spaces in a `tag` query, even with stopwords.

Notice that multiple tags in the same clause create a union of documents containing either tags. To create an intersection of documents containing all tags, you should repeat the tag filter several times. For example:

[code example]

Tag clauses can be combined into any subclause, used as negative expressions, optional expressions, and so on.

Geo filters

As of v0.21, it is possible to add geo radius queries directly into the query language with the syntax `@field:[{lon} {lat} {radius} {m|km|mi|ft}]`. This filters the result to a given radius from a lon,lat point, defined in meters, kilometers, miles or feet. See Redis's own [`GEORADIUS`]() command for more details.

Radius filters can be added into the query just like numeric filters. For example, in a database of businesses, looking for Chinese restaurants near San Francisco (within a 5km radius) would be expressed as: `chinese restaurant @location:[-122.41 37.77 5 km]`.

Polygon search

Geospatial databases are essential for managing and analyzing location-based data in a variety of industries. They help organizations make data-driven decisions, optimize operations, and achieve their strategic goals more efficiently. Polygon search extends Redis's geospatial search capabilities to be able to query against a value in a `GEOSHAPE` attribute. This value must follow a ["well-known text"](https://en.wikipedia.org/wiki/Well-known_text_representation_of_geometry) (WKT) representation of geometry. Two such geometries are supported:

- `POINT`, for example `POINT(2 4)`.
- `POLYGON`, for example `POLYGON((2 2, 2 8, 6 11, 10 8, 10 2, 2 2))`.

There is a new schema field type called `GEOSHAPE`, which can be specified as either:

- `FLAT` for Cartesian X Y coordinates
- `SPHERICAL` for geographic longitude and latitude coordinates. This is the default coordinate system.

Finally, there's new [`FT.SEARCH`]() syntax that allows you to query for polygons that either contain or are within a given geoshape.

`@field:[{WITHIN|CONTAINS} $geometry] PARAMS 2 geometry {geometry}`

Here's an example using two stacked polygons that represent a box contained within a house.

First, create an index using a `FLAT` `GEOSHAPE`, representing a 2D X Y coordinate system.

`FT.CREATE polygon_idx PREFIX 1 shape: SCHEMA g GEOSHAPE FLAT t TEXT`

Next, create the data structures that represent the geometries in the picture.

[code example]
Finally, use [`FT.SEARCH`]() to query the geometries. Note the use of `DIALECT 3`, which is required. Here are a few examples.

Search for a polygon that contains a specified point:

[code example]

Search for geometries contained in a specified polygon:

[code example]

Search for a polygon that is not contained in the indexed geometries:

[code example]

Search for a polygon that is known to be contained within the geometries (the box):

[code example]

Note that both the house and box shapes were returned.

GEOSHAPE does not support JSON multi-value or SORTABLE options.

For more examples, see the [`FT.CREATE`]() and [`FT.SEARCH`]() command pages.

Vector search

You can add vector similarity queries directly into the query language by:

1. Using a **range** query with the syntax of `@vector:[VECTOR_RANGE {radius} $query_vec]`, which filters the results to a given radius from a given query vector. The distance metric derives from the definition of a @vector field in the index schema, for example, Cosine or L2 (as of v2.6.1).

2. Running a k nearest neighbors (KNN) query on a @vector field. The basic syntax is `"*=>[ KNN {num|$num} @vector $query_vec ]"`.
It is also possible to run a hybrid query on filtered results. A hybrid query allows the user to specify a filter criteria that all results in a KNN query must satisfy. The filter criteria can include any type of field (i.e., indexes created on both vectors and other values, such as TEXT, PHONETIC, NUMERIC, GEO, etc.).
The general syntax for hybrid query is `{some filter query}=>[ KNN {num|$num} @vector $query_vec]`, where `=>` separates the filter query from the vector KNN query.

**Examples:**

* Return 10 nearest neighbors entities in which `query_vec` is closest to the vector stored in `@vector_field`:

  `*=>[KNN 10 @vector_field $query_vec]`

* Among entities published between 2020 and 2022, return 10 nearest neighbors entities in which `query_vec` is closest to the vector stored in `@vector_field`:

  `@published_year:[2020 2022]=>[KNN 10 @vector_field $query_vec]`

* Return every entity for which the distance between the vector stored under its @vector_field and `query_vec` is at most 0.5, in terms of the @vector_field distance metric:

  `@vector_field:[VECTOR_RANGE 0.5 $query_vec]`

As of v2.4, the KNN vector search can be used at most once in a query, while, as of v2.6, the vector range filter can be used multiple times in a query. For more information on vector similarity syntax, see [Querying vector fields](), and [Vector search examples]() sections.

Prefix matching

When indexes are updated, Redis maintains a dictionary of all terms in the index. This can be used to match all terms starting with a given prefix. Selecting prefix matches is done by appending `*` to a prefix token. For example:

[code example]

Will be expanded to cover `(hello|help|helm|...) world`.

A few notes on prefix searches

1. As prefixes can be expanded into many terms, use them with caution. The expansion will create a Union operation of all suffixes.

2. As a protective measure to avoid selecting too many terms, blocking Redis, which is single threaded, there are two limitations on prefix matching:

  * Prefixes are limited to 2 letters or more. You can change this number by using the `MINPREFIX` setting on the module command line.

  * The minimum word length to stem is 4 letters or more. You can change this number by using the `MINSTEMLEN` setting on the module command line.

  * Expansion is limited to 200 terms or less. You can change this number by using the `MAXEXPANSIONS` setting on the module command line.

3. Prefix matching fully supports Unicode and is case insensitive.

4. Currently, there is no sorting or bias based on suffix popularity.

Infix/suffix matching

As of v2.6.0, the dictionary can be used for infix (contains) or suffix queries by appending `*` to the token. For example:

[code example]

These queries are CPU intensive because they require iteration over the whole dictionary.

All notes about prefix searches also apply to infix/suffix queries.

Using a suffix trie

A suffix trie maintains a list of terms that match the suffix. If you add a suffix trie to a field using the `WITHSUFFIXTRIE` keyword, you can create more efficient infix and suffix queries because it eliminates the need to iterate over the whole dictionary. However, the iteration on the union does not change.

Suffix queries create a union of the list of terms from the suffix term node. Infix queries use the suffix terms as prefixes to the trie and create a union of all terms from all matching nodes.

Wildcard matching

As of v2.6.0, you can use the dictionary for wildcard matching queries with these parameters.

* `?` - for any single character
* `*` - for any character repeating zero or more times
* `\` - for escaping; other special characters are ignored

An example of the syntax is `"w'foo*bar?'"`.

Using a suffix trie

A suffix trie maintains a list of terms which match the suffix. If you add a suffix trie to a field using the `WITHSUFFIXTRIE` keyword, you can create more efficient wildcard matching queries because it eliminates the need to iterate over the whole dictionary. However, the iteration on the union does not change.

With a suffix trie, the wildcard pattern is broken into tokens at every `*` character. A heuristic is used to choose the token with the least terms, and each term is matched with the wildcard pattern.

Fuzzy matching

As of v1.2.0, the dictionary of all terms in the index can also be used to perform [fuzzy matching](https://en.wikipedia.org/wiki/Approximate_string_matching).
Fuzzy matches are performed based on [Levenshtein distance](https://en.wikipedia.org/wiki/Levenshtein_distance) (LD).
Fuzzy matching on a term is performed by surrounding the term with '%', for example:

[code example]

This performs fuzzy matching on `hello` for all terms where LD is 1.

As of v1.4.0, the LD of the fuzzy match can be set by the number of '%' characters surrounding it, so that `%%hello%%` will perform fuzzy matching on 'hello' for all terms where LD is 2.

The maximum LD for fuzzy matching is 3.

Wildcard queries

As of v1.1.0, you can use a special query to retrieve all the documents in an index. This is meant mostly for the aggregation engine. You can call it by specifying only a single star sign as the query string, in other words, `FT.SEARCH myIndex *`.

You can't combine this with any other filters, field modifiers, or anything inside the query. It is technically possible to use the deprecated `FILTER` and `GEOFILTER` request parameters outside the query string in conjunction with a wildcard, but this makes the wildcard meaningless and only hurts performance.

A few query examples

* Simple phrase query - `hello` _AND_ `world`:

        hello world

* Exact phrase query - `hello` _FOLLOWED BY_ `world`:

        "hello world"

* Union - documents containing either `hello` _OR_ `world`:

        hello|world

* Not - documents containing `hello` _BUT NOT_ `world`:

        hello -world

* Intersection of unions:

        (hello|halo) (world|werld)

* Negation of union:

        hello -(world|werld)

* Union inside phrase:

        (barack|barrack) obama

* Optional terms with higher priority to ones containing more matches:

        obama ~barack ~michelle

* Exact phrase in one field, one word in another field:

        @title:"barack obama" @job:president

* Combined _AND_, _OR_ with field specifiers:

        @title:"hello world" @body:(foo bar) @category:(articles|biographies)

* Prefix/infix/suffix queries:

        hello worl*

        hel* *worl

        hello -*worl*

* Wildcard matching queries:

        "w'foo??bar??baz'"

        "w'???????'"

        "w'hello*world'"

* Numeric filtering - products named `tv` with a price range of 200 to 500:

        @name:tv @price:[200 500]

* Numeric filtering - users with age greater than 18:

        @age:[(18 +inf]

* Vector search with cluster optimization - retrieve 100 nearest neighbors with each shard providing 50% of results:

        *=>[KNN 100 @doc_embedding $BLOB]=>{$SHARD_K_RATIO: 0.5; $YIELD_DISTANCE_AS: vector_distance}

Mapping common SQL predicates to Redis Search

| SQL Condition | Redis Search Equivalent | Comments |
|---------------|-----------------------|----------|
| WHERE x='foo' AND y='bar' | @x:foo @y:bar | for less ambiguity use (@x:foo) (@y:bar) |
| WHERE x='foo' AND y!='bar' | @x:foo -@y:bar |
| WHERE x='foo' OR y='bar' | (@x:foo)\|(@y:bar) |
| WHERE x IN ('foo', 'bar','hello world') | @x:(foo\|bar\|"hello world") | quotes mean exact phrase |
| WHERE y='foo' AND x NOT IN ('foo','bar') | @y:foo (-@x:foo) (-@x:bar) |
| WHERE x NOT IN ('foo','bar') | -@x:(foo\|bar) |
| WHERE num BETWEEN 10 AND 20 | @num:[10 20] |
| WHERE num >= 10 | @num:[10 +inf] |
| WHERE num > 10 | @num:[(10 +inf] |
| WHERE num < 10 | @num:[-inf (10] |
| WHERE num <= 10 | @num:[-inf 10] |
| WHERE num < 10 OR num > 20 | @num:[-inf (10] \| @num:[(20 +inf] |
| WHERE name LIKE 'john%' | @name:john* |

Technical notes

The query parser is built using the Lemon Parser Generator and a Ragel based lexer. You can see the `DIALECT 2` grammar definition [at this git repo](https://github.com/RediSearch/RediSearch/blob/master/src/query_parser/v2/parser.y).

You can also see the [search-default-dialect]() configuration parameter.

FT.SEARCH cars "@country:korea @engine:(diesel|hybrid) @class:suv"

FT.SEARCH idx "@title|body:(hello world) @url|image:mydomain"

# Return all documents containing all three cities as tags
@cities:{ New York } @cities:{Los Angeles} @cities:{ Barcelona }

# Now, return all documents containing either city
@cities:{ New York | Los Angeles | Barcelona }

HSET shape:1 t "this is my house" g "POLYGON((2 2, 2 8, 6 11, 10 8, 10 2, 2 2))"
HSET shape:2 t "this is a square in my house" g "POLYGON((4 4, 4 6, 6 6, 6 4, 4 4))"

FT.SEARCH polygon_idx "@g:[CONTAINS $point]" PARAMS 2 point 'POINT(8 8)' DIALECT 3
1) (integer) 1
2) "shape:1"
3) 1) "t"
   2) "this is my house"
   3) "g"
   4) "POLYGON((2 2, 2 8, 6 11, 10 8, 10 2, 2 2))"

FT.SEARCH polygon_idx "@g:[CONTAINS $poly]" PARAMS 2 poly 'POLYGON((14 4, 14 6, 16 6, 16 4, 14 4))' DIALECT 3
1) (integer) 0


Scoring documents

Summary: Full-text scoring functions

Overview

When searching, documents are scored based on their relevance to the query. The score is a floating point number between 0.0 and 1.0, where 1.0 is the highest score. The score is returned as part of the search results and can be used to sort the results.

Redis Open Source comes with a few scoring functions to evaluate document relevance. They are all based on document scores and term frequency. This is regardless of the ability to use [sortable fields](). Scoring functions are specified by adding the `SCORER {scorer_name}` argument to a search query.

If you prefer a custom scoring function, it is possible to add more functions using the [extension API]().

The following is a list of the pre-bundled scoring functions available in Redis and a short explanation about how they work. Each function is mentioned by registered name, which can be passed as a `SCORER` argument in [`FT.SEARCH`]().

TFIDF

Basic [TF-IDF scoring](https://en.wikipedia.org/wiki/Tf%E2%80%93idf) with a few extra features:

1. For each term in each result, the TF-IDF score of that term is calculated to that document. Frequencies are weighted based on field weights that are pre-determined, and each term's frequency is normalized by the highest term frequency in each document.

2. The total TF-IDF for the query term is multiplied by the presumptive document score given on `FT.CREATE` via `SCORE_FIELD`.

3. A penalty is assigned to each result based on "slop" or cumulative distance between the search terms. Exact matches will get no penalty, but matches where the search terms are distant will have their score reduced significantly. For each bigram of consecutive terms, the minimal distance between them is determined. The penalty is the square root of the sum of the distances squared; e.g., `1/sqrt(d(t2-t1)^2 + d(t3-t2)^2 + ...)`.

Given N terms in document D, `T1...Tn`, the resulting score could be described with this Python function:

[code example]

TFIDF.DOCNORM

Identical to the default `TFIDF` scorer, with one important distinction:

Term frequencies are normalized by the length of the document, expressed as the total number of terms. The length is weighted, so that if a document contains two terms, one in a field that has a weight 1 and one in a field with a weight of 5, the total frequency is 6, not 2.

[code example]

BM25STD (default)

A variation on the basic `TFIDF` scorer, see [this Wikipedia article for more info](https://en.wikipedia.org/wiki/Okapi_BM25).

The relevance score for each document is multiplied by the presumptive document score and a penalty is applied based on slop as in `TFIDF`.

The `BM25` scorer was renamed `BM25STD` in Redis Open Source 8.4. `BM25` is deprecated.

[code example]

BM25STD.NORM

A variation of `BM25STD`, where the scores are normalized by the minimum and maximum scores.

`BM25STD.NORM` uses min–max normalization across the collection, making it more accurate in distinguishing documents when term frequency distributions vary significantly. Because it depends on global statistics, results adapt better to collection-specific characteristics, but this comes at a performance cost: min and max values must be computed and updated whenever the collection changes. This method is recommended when ranking precision is critical and the dataset is relatively stable.

BM25STD.TANH

A variation of `BM25STD.NORM`, where the scores are normalised by linear function `tanh(x)`. `BMSTDSTD.TANH` can take an optional argument, `BM25STD_TANH_FACTOR Y`, which is used to smooth the function and the score values. The default value for `Y` is 4.

`BM25STD.TANH` applies a smooth transformation using the `tanh(x/factor)` function, which avoids collection-dependent statistics and yields faster, more efficient scoring. While this makes it more scalable and consistent across different datasets, the trade-off is reduced accuracy in cases where min–max normalization provides sharper separation. This method is recommended when performance and throughput are prioritized over fine-grained ranking sensitivity.

Following is an example of how to use `BM25STD_TANH_FACTOR Y` in a query.

[code example]

DISMAX

A simple scorer that sums up the frequencies of matched terms. In the case of union clauses, it will give the maximum value of those matches. No other penalties or factors are applied.

It is not a one-to-one implementation of [Solr's DISMAX algorithm](https://wiki.apache.org/solr/DisMax), but it follows it in broad terms.

[code example]

DOCSCORE

A scoring function that just returns the presumptive score of the document without applying any calculations to it. Since document scores can be updated, this can be useful if you'd like to use an external score and nothing further.

[code example]

HAMMING

Scoring by the inverse Hamming distance between the document's payload and the query payload is performed. Since the nearest neighbors are of interest, the inverse Hamming distance (`1/(1+d)`) is used so that a distance of 0 gives a perfect score of 1 and is the highest rank.

This only works if:

1. The document has a payload.
2. The query has a payload.
3. Both are exactly the same length.

Payloads are binary-safe, and having payloads with a length that is a multiple of 64 bits yields slightly faster results.

Example:

[code example]

def get_score(terms, doc):
    # the sum of tf-idf
    score = 0

    # the distance penalty for all terms
    dist_penalty = 0

    for i, term in enumerate(terms):
        # tf normalized by maximum frequency
        tf = doc.freq(term) / doc.max_freq

        # idf is global for the index, and not calculated each time in real life
        idf = log2(1 + total_docs / docs_with_term(term))

        score += tf*idf

        # sum up the distance penalty
        if i > 0:
            dist_penalty += min_distance(term, terms[i-1])**2

    # multiply the score by the document score
    score *= doc.score

    # divide the score by the root of the cumulative distance
    if len(terms) > 1:
        score /= sqrt(dist_penalty)

    return score

FT.SEARCH myIndex "foo" SCORER TFIDF.DOCNORM

FT.SEARCH idx "term" SCORER BM25STD.TANH BM25STD_TANH_FACTOR 12 WITHSCORES

> HSET key:1 foo hello payload aaaabbbb
(integer) 2

> HSET key:2 foo bar payload aaaacccc
(integer) 2

> FT.CREATE idx ON HASH PREFIX 1 key: PAYLOAD_FIELD payload SCHEMA foo TEXT
"OK"

> FT.SEARCH idx "*" PAYLOAD "aaaabbbc" SCORER HAMMING WITHSCORES
1) "2"
2) "key:1"
3) "0.5"
4) 1) "foo"
   2) "hello"
5) "key:2"
6) "0.25"
7) 1) "foo"
   2) "bar"


Sorting by indexed fields

Summary: Support for sorting query results

Overview

As of RediSearch 0.15, you can bypass the scoring function mechanism and order search results by the value of different document attributes (fields) directly, even if the sorting field is not used by the query. For example, you can search for first name and sort by last name.

Declaring sortable fields

When creating an index with [`FT.CREATE`](), you can declare `TEXT`, `TAG`, `NUMERIC`, and `GEO` attributes as `SORTABLE`. When an attribute is sortable, you can order the results by its values with relatively low latency. When an attribute is not sortable, it can still be sorted by its values, but with increased latency. For example, in the following schema:

[code example]

The fields `last_name` and `age` are sortable, but `first_name` isn't. This means you can search by either first and/or last name, and sort by last name or age.

Note on sortable fields

In the current implementation, when declaring a sortable field, its content gets copied into a special location in the index that provides for fast access during sorting. This means that making long fields sortable is very expensive and you should be careful with it.

Normalization (UNF option)

By default, text fields get normalized and lowercased in a Unicode-safe way when stored for sorting. For example, `America` and `america` are considered equal in terms of sorting.

Using the `UNF` (un-normalized form) argument, it is possible to disable the normalization and keep the original form of the value. Therefore, `America` will come before `america`.

Specifying SORTBY

If an index includes sortable fields, you can add the `SORTBY` parameter to the search request (outside the query body) to order the results. This overrides the scoring function mechanism, and the two cannot be combined. If `WITHSCORES` is specified together with `SORTBY`, the scores returned are simply the relative position of each result in the result set.

The syntax for `SORTBY` is:

[code example]

* `field_name` must be a sortable field defined in the schema.

* `ASC` means ascending order, `DESC` means descending order.

* The default ordering is `ASC`.

Example

[code example]

FT.CREATE users SCHEMA first_name TEXT last_name TEXT SORTABLE age NUMERIC SORTABLE

> FT.CREATE users ON HASH PREFIX 1 "user" SCHEMA first_name TEXT SORTABLE last_name TEXT age NUMERIC SORTABLE

# Add some users
> HSET user1 first_name "alice" last_name "jones" age 35
> HSET user2 first_name "bob" last_name "jones" age 36

# Searching while sorting

# Searching by last name and sorting by first name
> FT.SEARCH users "@last_name:jones" SORTBY first_name DESC

# Searching by both first and last name, and sorting by age
> FT.SEARCH users "jones" SORTBY age ASC


Spellchecking

Summary: Query spelling correction support

Overview

Query spelling correction provides suggestions for misspelled search terms. For example, the term 'reids' may be a misspelled version of 'redis'.

In such cases, and as of v1.4, RediSearch can be used for generating alternatives to misspelled query terms. A misspelled term is a full text term (i.e., a word) that is:

  1. Not a stop word
  2. Not in the index
  3. At least 3 characters long

The alternatives for a misspelled term are generated from the corpus of already-indexed terms and, optionally, one or more custom dictionaries. Alternatives become spelling suggestions based on their respective Levenshtein distances from the misspelled term. Each spelling suggestion is given a normalized score based on its occurrences in the index.

To obtain the spelling corrections for a query, refer to the documentation of the [`FT.SPELLCHECK`]() command.

Custom dictionaries

A dictionary is a set of terms. Dictionaries can be added with terms, have terms deleted from them, and have their entire contents dumped using the [`FT.DICTADD`](), [`FT.DICTDEL`]() and [`FT.DICTDUMP`]() commands, respectively.

Dictionaries can be used to modify the behavior of spelling corrections by including or excluding their contents from potential spelling correction suggestions.

When used for term inclusion, the terms in a dictionary can be provided as spelling suggestions regardless of their occurrence in the index. Scores of suggestions from inclusion dictionaries are always 0.

Conversely, terms in an exclusion dictionary will never be returned as spelling alternatives.


Stemming

Summary: Stemming support

Overview

RediSearch supports stemming - that is adding the base form of a word to the index. This allows the query for "`hiring`" to also return results for "`hire`" and "`hired`", for example.

The current stemming support is based on the Snowball stemmer library, which supports most European languages, as well as Arabic and other. See the "[Supported languages](#supported-languages)" section below. We hope to include more languages soon (if you need a specific language support, please open an issue).

For further details see the [Snowball Stemmer website](https://snowballstem.org/).

How it works?

Stemming maps different forms of the same word to a common root - "stem" - for example, the English stemmer maps *studied* ,*studies* and *study* to *studi* . So a searching for *studied* would also find documents which only have the other forms.

In order to define which language the Stemmer should apply when building the index, you need to specify the `LANGUAGE` parameter for the entire index or for the specific field. For more details check the [FT.CREATE]() syntax.

**Create a index with language definition**

Create a index for words in German "`wort:`" with a single `TEXT` field "`wort`"

redis> FT.CREATE idx:german ON HASH PREFIX 1 "wort:" LANGUAGE GERMAN SCHEMA wort TEXT

**Adding words**

Adding some words with same stem in German, all variations of the word `stück` ( `piece` in english): `stück stücke stuck stucke` => `stuck`

redis> HSET wort:1 wort stück
(integer) 1
redis> HSET wort:2 wort stücke
(integer) 1
redis> HSET wort:3 wort stuck
(integer) 1
redis> HSET wort:4 wort stucke
(integer) 1

**Searching for a common stem**

Search for "stuck" (german for "piece"). As of v2.10, it's only necessary to specify the `LANGUAGE` argument when it wasn't specified to create the index being used to search.
Note the results for words that contains "`ü`" are encoded in UTF-8.

redis> FT.SEARCH idx:german '@wort:(stuck)' German
1) (integer) 4
2) "wort:3"
3) 1) "wort"
   2) "stuck"
4) "wort:4"
5) 1) "wort"
   2) "stucke"
6) "wort:1"
7) 1) "wort"
   2) "st\xc3\xbcck"
8) "wort:2"
9) 1) "wort"
   2) "st\xc3\xbccke"

Supported languages

The following languages are supported and can be passed to the engine when indexing or querying using lowercase:

* `arabic`
* `armenian`
* `basque`
* `catalan`
* `danish`
* `dutch`
* `english`
* `finnish`
* `french`
* `german`
* `greek`
* `hindi`
* `hungarian`
* `indonesian`
* `irish`
* `italian`
* `lithuanian`
* `nepali`
* `norwegian`
* `portuguese`
* `romanian`
* `russian`
* `serbian`
* `spanish`
* `swedish`
* `tamil`
* `turkish`
* `yiddish`
* `chinese` (see below)

Chinese support

Indexing a Chinese document is different than indexing a document in most other languages because of how tokens are extracted. While most languages can have their tokens distinguished by separation characters and whitespace, this is not common in Chinese.

Chinese tokenization is done by scanning the input text and checking every character or sequence of characters against a dictionary of predefined terms and determining the most likely match based on the surrounding terms and characters.

Redis Open Source makes use of the [Friso](https://github.com/lionsoul2014/friso) chinese tokenization library for this purpose. This is largely transparent to the user and often no additional configuration is required.

Using custom dictionaries

If you wish to use a custom dictionary, you can do so at the module level when loading the module. The `FRISOINI` setting can point to the location of a `friso.ini` file which contains the relevant settings and paths to the dictionary files.

Note that there is no default `friso.ini` file location. RedisSearch comes with its own `friso.ini` and dictionary files that are compiled into the module binary at build-time.


Stop words

Summary: Stop words support

Overview

Redis Open Source has a default list of [stop words](https://en.wikipedia.org/wiki/Stop_words). These are words that are usually so common that they do not add much information to search, but take up a lot of space and CPU time in the index.

When indexing, stop words are discarded and not indexed. When searching, they are also ignored and treated as if they were not sent to the query processor. This is done when parsing the query.

At the moment, the default stop word list applies to all full-text indexes in all languages and can be overridden manually at index creation time.

Default stop word list

The following words are treated as stop words by default:

[code example]

Overriding the default stop word list

Stop words for an index can be defined (or disabled completely) on index creation using the `STOPWORDS` argument with the [[`FT.CREATE`]() command.

The format is `STOPWORDS {number} {stopword} ...` where number is the number of stop words given. The `STOPWORDS` argument must come before the `SCHEMA` argument. For example:

[code example]

Disable the use of stop words

Disabling stop words completely can be done by passing `STOPWORDS 0` to [`FT.CREATE`]().

Avoiding stop word detection in search queries

In rare use cases, where queries are very long and are guaranteed by the client application not to contain stop words, it is possible to avoid checking for them when parsing the query. This saves some CPU time and is only worth it if the query has dozens or more terms in it. Using this without verifying that the query doesn't contain stop words might result in empty queries.

a,    is,    the,   an,   and,  are, as,  at,   be,   but,  by,   for,
 if,   in,    into,  it,   no,   not, of,  on,   or,   such, that, their,
 then, there, these, they, this, to,  was, will, with

FT.CREATE myIndex STOPWORDS 3 foo bar baz SCHEMA title TEXT body TEXT


Synonym support

Overview

Redis Open Source supports synonyms. That is, searching for synonym words defined by the synonym data structure.

The synonym data structure is a set of groups, each of which contains synonym terms. For example, the following synonym data structure contains three groups, and each group contains three synonym terms:

[code example]

When these three groups are located inside the synonym data structure, it is possible to search for "child" and receive documents containing "boy", "girl", "child" and "baby".

The synonym search technique

A simple HashMap is used to map between the terms and the group IDs. During index creation, a check is made to see if the current term appears in the synonym map, and if it does, take all the group IDs that the term belongs to.

For each group ID, another record is added to the inverted index called "\~\<id\>" that contains the same information as the term itself. When performing a search, a check is made to see if the searched term appears in the synonym map, and if it does, take all the group IDs the term belongs to. For each group ID, search for "\~\<id\>" and return the combined results. This technique ensures that all the synonyms of a given term will be returned.

Handling concurrency

Since the indexing is performed in a separate thread, the synonyms map may change during indexing, which in turn may cause data corruption or crashes during indexing or searching. To solve this issue, a read-only copy is created for indexing purposes. The read-only copy is maintained using reference count.

As long as the synonyms map does not change, the original synonym map holds a reference to its read-only copy, so it will not be freed. After the data inside the synonyms map has changed, the synonyms map decreses the reference count of its read only copy. This ensures that when all the indexers are done using the read-only copy, it will automatically be freed. This ensures that the next time an indexer asks for a read-only copy, the synonyms map will create a new copy (containing the new data) and return it.

Example

[code example]

{boy, child, baby}
{girl, child, baby}
{man, person, adult}

# Create an index
> FT.CREATE idx schema t text

# Create a synonym group
> FT.SYNUPDATE idx group1 hello world

# Insert documents
> HSET foo t hello
(integer) 1
> HSET bar t world
(integer) 1

# Search
> FT.SEARCH idx hello
1) (integer) 2
2) "foo"
3) 1) "t"
   2) "hello"
4) "bar"
5) 1) "t"
   2) "world"


Tag fields

Overview

Tag fields provide exact match search capabilities with high performance and memory efficiency. Use tag fields when you need to filter documents by specific values without the complexity of full-text search tokenization.

Tag fields interpret text as a simple list of *tags* delimited by a [separator](#separator-options) character. This approach enables simpler [tokenization]() and encoding, making tag indexes much more efficient than full-text indexes. Note: even though tag and text fields both use text, they are two separate field types and so you don't query them the same way.

- The default separator for hash documents is a comma (`,`).
- There is no default separator for JSON documents. You must explicitly specify one if needed.

Specifying a tag from the text `"foo,bar"` behaves differently:
- For hash documents, two tags are created: `"foo"` and `"bar"`.
- For JSON documents, one tag is created: `"foo,bar"` (unless you add `SEPARATOR ","`).

Tag fields vs text fields

Tag fields excel in scenarios requiring exact matching rather than full-text search. Choose tag fields when you need to index categorical data such as:

- **Product categories**: Electronics, Clothing, Books
- **User roles**: Admin, Editor, Viewer
- **Status values**: Active, Pending, Completed
- **Geographic regions**: US, EU, APAC
- **Content types**: Video, Image, Document

Key differences

| Feature | Tag fields | Text fields |
|---------|------------|-------------|
| **Search type** | Exact match | Full-text search |
| **Tokenization** | Simple delimiter splitting | Complex word tokenization |
| **Stemming** | None | Language-specific stemming |
| **Memory usage** | Very low (1-2 bytes per entry) | Higher (frequencies, positions) |
| **Performance** | Fastest | Slower for exact matches |
| **Multiple values** | Support comma-separated lists | Single text content |
| **Case control** | Optional case-sensitive matching | Typically case-insensitive |
| **Use case** | Categories, filters, IDs | Content search, descriptions |

Technical details

Index structure

- **Compressed storage**: Only document IDs encoded as deltas (1-2 bytes per entry)
- **No frequencies**: Unlike TEXT fields, tag indexes don't store term frequencies
- **No positions**: No offset vectors or field flags stored
- **Limit**: You can create up to 1024 tag fields per index

Tokenization differences

- **Simple splitting**: Text is split only at separator characters
- **No stemming**: Words are indexed exactly as written
- **Case handling**: Optional case-sensitive or case-insensitive matching
- **No stop words**: All tag values are indexed regardless of content

Create a tag field

Add tag fields to your schema using this syntax:

[code example]

Separator options

The separator behavior differs significantly between hash and JSON documents:

**Hash documents**

- The default separator is the comma (`,`).
- Strings are automatically splits at commas. For example,
  the string `"red,blue,green"` becomes three tags: `"red"`, `"blue"`, and `"green"`.
- You can use any printable ASCII character as a custom separator.

**JSON documents**

- There is no default separator; it's effectively `null`.
- Treats the entire string as single tag unless you specify a separator with the `SEPARATOR` option. For example,
  the string `"red,blue,green"` becomes one tag: `"red,blue,green"`
- Add `SEPARATOR ","` to your schema to allow splitting.
- You should use JSON arrays instead of comma-separated strings

**Why the difference?**

JSON has native array support, so the preferred approach is:

[code example]
Rather than:

[code example]

Case sensitivity

- **Default**: Case-insensitive matching (`red` matches `Red`, `RED`)
- **CASESENSITIVE**: Preserves original case for exact matching

Examples

**Hash examples**

1. Basic hash tag field (automatic comma splitting):

    [code example]

1. Hash with custom separator:

    [code example]

1. Case-sensitive hash tags:

    [code example]

**JSON examples**

1. JSON with string and explicit separator (not recommended):

    [code example]

1. JSON with array of strings (recommended approach):

    [code example]

1. JSON without separator (single tag):

    [code example]

Query tag fields

**Important**: Tag fields require special query syntax - you cannot find tag values with general field-less searches.

Basic tag query syntax

Use curly braces to specify tag values (the braces are part of the syntax):

[code example]

Single tag match

Find documents with a specific tag:

[code example]

Multiple tag match (OR)

Find documents with any of the specified tags:

[code example]

Combining with other queries

Tag queries work seamlessly with other field types:

[code example]

Prefix matching

Use the `*` wildcard for prefix matching:

[code example]

Negative matching

Exclude documents with specific tags:

[code example]

Advanced tag queries

OR vs AND logic

**Single clause (OR logic)**: Find documents with ANY of the specified tags
[code example]

**Multiple clauses (AND logic)**: Find documents with ALL of the specified tags
[code example]

Practical example

Consider a travel database:

[code example]

**Find travelers who visited any of these cities:**
[code example]

**Find travelers who visited all of these cities:**
[code example]

Handle special characters

Tag fields can contain any punctuation except the field separator, but you need to escape certain characters in queries.

Defining tags with special characters

You can store tags with punctuation without escaping:

[code example]

Querying tags with special characters

**Escape punctuation in queries** using backslash (`\`):

[code example]

Characters that need escaping

In tag queries, escape these characters:
- Single quotes: `'` → `\\'`
- Hyphens: `-` → `\\-`
- Parentheses: `()` → `\\(\\)`
- Brackets: `[]{}` → `\\[\\]\\{\\}`
- Pipes: `|` → `\\|`

Spaces in tags

**Modern Redis** (v2.4+): Spaces don't need escaping in tag queries
[code example]

**Older versions** or **dialect 1**: Escape spaces
[code example]

Best practices

- **Use simple separators**: Stick to comma (`,`) or semicolon (`;`)
- **Avoid complex punctuation**: Keep tag values simple when possible
- **Test your queries**: Verify escaping works with your specific characters
- **Use consistent casing**: Decide on case sensitivity early in your design

See [Query syntax]() for complete escaping rules.

Performance and architecture considerations

Multiple TAG fields versus a single TAG field

You can structure your data in two ways:

1. Multiple single-value TAG fields

    [code example]

1. Single multi-value TAG field

    [code example]

Performance comparison

Both approaches have similar performance characteristics:

- Memory usage is comparable: TAG indexes are highly compressed regardless of structure.
- Query speed is also comparable, though single-value tags may offer a slight edge: both use the same underlying inverted index structure.
- Index efficiency; TAG fields store only document IDs (1-2 bytes per entry).

Choose TAG fields based on your use case

Use multiple TAG fields when:

- You need field-specific queries (`@color:{blue}` vs `@brand:{ASUS}`).
- Your schema is stable and well-defined.
- You want cleaner, more readable queries.
- You need different configurations per field (for example, case-sensitive versus case-insensitive).

Use single TAG field when:

- You have dynamic or unknown tag categories.
- You want maximum flexibility for adding new tag types.
- Your application manages tag prefixing/namespacing.
- You have many sparse categorical fields.

An e-commerce use case

[code example]

Next steps

- Learn about [tokenization rules]() for tag fields
- Explore [field and type options]() for other field types
- See [query syntax]() for advanced query patterns

FT.CREATE ... SCHEMA ... {field_name} TAG [SEPARATOR {sep}] [CASESENSITIVE]

{"colors": ["red", "blue", "green"]}  // Use with $.colors[*] AS colors TAG

{"colors": "red,blue,green"}  // Requires SEPARATOR ","

HSET product:1 categories "Electronics,Gaming,PC"
    FT.CREATE products ON HASH PREFIX 1 product: SCHEMA categories TAG

    > FT.SEARCH products '@categories:{Gaming}'
    1) "1"
    2) "product:1"

HSET book:1 genres "Fiction;Mystery;Thriller"
    FT.CREATE books ON HASH PREFIX 1 book: SCHEMA genres TAG SEPARATOR ";"

HSET product:1 categories "Electronics,Gaming,PC"
    FT.CREATE products ON HASH PREFIX 1 product: SCHEMA categories TAG CASESENSITIVE

    > FT.SEARCH products '@categories:{PC}'  # Case matters
    1) "1"
    2) "product:1"

JSON.SET key:1 $ '{"colors": "red, orange, yellow"}'
    FT.CREATE idx ON JSON PREFIX 1 key: SCHEMA $.colors AS colors TAG SEPARATOR ","

    > FT.SEARCH idx '@colors:{orange}'
    1) "1"
    2) "key:1"
    3) 1) "$"
       2) "{\"colors\":\"red, orange, yellow\"}"

JSON.SET key:1 $ '{"colors": ["red", "orange", "yellow"]}'
    FT.CREATE idx ON JSON PREFIX 1 key: SCHEMA $.colors[*] AS colors TAG

    > FT.SEARCH idx '@colors:{orange}'
    1) "1"
    2) "key:1"
    3) 1) "$"
       2) "{\"colors\":[\"red\",\"orange\",\"yellow\"]}"

JSON.SET key:1 $ '{"category": "Electronics,Gaming"}'
    FT.CREATE idx ON JSON PREFIX 1 key: SCHEMA $.category AS category TAG
    # No SEPARATOR specified - entire string becomes one tag

    > FT.SEARCH idx '@category:{Electronics,Gaming}'  # Must match exactly
    1) "1"
    2) "key:1"

FT.SEARCH idx "@category:{Electronics}"
FT.SEARCH idx "@status:{Active}"

FT.SEARCH idx "@tags:{ hello world | foo bar }"
FT.SEARCH idx "@category:{ Electronics | Gaming | Software }"

FT.CREATE idx ON HASH PREFIX 1 product: SCHEMA
    title TEXT
    price NUMERIC
    category TAG SEPARATOR ";"

# Combine text search, numeric range, and tag filter
FT.SEARCH idx "@title:laptop @price:[500 1500] @category:{ Electronics | Gaming }"

FT.SEARCH idx "@tags:{ tech* }"        # Matches: technology, technical, tech
FT.SEARCH idx "@tags:{ hello\\ w* }"   # Matches: "hello world", "hello web"

FT.SEARCH idx "-@category:{Discontinued}"
FT.SEARCH idx "@title:phone -@category:{Refurbished}"

@cities:{ New York | Los Angeles | Barcelona }
# Returns: Documents with New York OR Los Angeles OR Barcelona

@cities:{ New York } @cities:{ Los Angeles } @cities:{ Barcelona }
# Returns: Documents with New York AND Los Angeles AND Barcelona

FT.CREATE travelers ON HASH PREFIX 1 traveler: SCHEMA
    name TEXT
    cities TAG

HSET traveler:1 name "John Doe" cities "New York, Barcelona, San Francisco"
HSET traveler:2 name "Jane Smith" cities "New York, Los Angeles, Tokyo"

FT.SEARCH travelers "@cities:{ New York | Los Angeles | Barcelona }"
# Returns: Both John and Jane

FT.SEARCH travelers "@cities:{ New York } @cities:{ Barcelona }"
# Returns: Only John (has both New York and Barcelona)

FT.CREATE products ON HASH PREFIX 1 test: SCHEMA tags TAG

HSET test:1 tags "Andrew's Top 5,Justin's Top 5,5-Star Rating"
HSET test:2 tags "Best Buy,Top-Rated,Editor's Choice"

# Query for "Andrew's Top 5"
FT.SEARCH products "@tags:{ Andrew\\'s Top 5 }"

# Query for "5-Star Rating"
FT.SEARCH products "@tags:{ 5\\-Star Rating }"

# Query for "Editor's Choice"
FT.SEARCH products "@tags:{ Editor\\'s Choice }"

FT.SEARCH products "@tags:{ Top Rated Product }"

FT.SEARCH products "@tags:{ Top\\ Rated\\ Product }"

FT.CREATE products ON JSON PREFIX 1 product: SCHEMA
        $.color AS color TAG
        $.brand AS brand TAG
        $.type AS type TAG

    JSON.SET product:1 $ '{"color": "blue", "brand": "ASUS", "type": "laptop"}'

    # Query specific fields
    FT.SEARCH products '@color:{blue} @brand:{ASUS}'

FT.CREATE products ON JSON PREFIX 1 product: SCHEMA
        $.tags[*] AS tags TAG

    JSON.SET product:1 $ '{"tags": ["color:blue", "brand:ASUS", "type:laptop"]}'

    # Query with prefixed values
    FT.SEARCH products '@tags:{color:blue} @tags:{brand:ASUS}'

# Product categories and attributes
FT.CREATE products ON HASH PREFIX 1 product: SCHEMA
    name TEXT
    category TAG
    brand TAG
    features TAG SEPARATOR ";"

HSET product:1 name "Gaming Laptop" category "Electronics" brand "ASUS" features "RGB;16GB RAM;SSD"

# Find gaming products with specific features
FT.SEARCH products "@category:{Electronics} @features:{RGB} @features:{SSD}"


Move from Development to Production with Redis Search

Overview

Transitioning a Redis Open Source with Redis Search environment from development to production requires thoughtful consideration of configuration, performance tuning, and resource allocation. This guide outlines key practices to ensure your Redis deployment operates optimally under production workloads.

Configuration parameter considerations

Redis Search offers several configurable parameters that influence query results and performance. While a full list of these parameters and their functions can be found [here](), this section highlights the most commonly adjusted parameters for production environments.

1. `TIMEOUT`

- Purpose: limits the duration a query is allowed to execute.
- Default: 500 milliseconds.
- Behavior:
  - Ensures that queries do not monopolize the main Redis thread.
  - If a query exceeds the `TIMEOUT` value, its outcome is determined by the `ON_TIMEOUT` setting:
    - `FAIL`: the query will return an error.
    - `PARTIAL`: this setting will return the top results accumulated by the query until it timed out.
- Recommendations:
  - Caution: be mindful when increasing `TIMEOUT`,<!-- especially in environments without QPF,--> as long-running queries can degrade overall system performance.
 <!-- - With Query Performance Factor (QPF): if QPF (multi-threaded query execution) is enabled, the risks of increasing this value are mitigated, as queries execute against index memory using multiple threads.-->

2. `MINPREFIX`

- Purpose: sets the minimum number of characters required for wildcard searches.
- Default: 2 characters.
- Behavior:
  - Queries like `he*` are valid, while `h*` would not meet the threshold.
- Recommendations:
  - Lowering this value to 1 can significantly increase result sets, which may lead to degraded performance.
  - Keep the default unless there is a strong use case for single-character wildcards.

3. `MAXPREFIXEXPANSIONS`

- Purpose: Defines the maximum number of expansions for a wildcard query term.
- Default: 200 expansions.
- Behavior:
  - Expansions: when a wildcard query term is processed, Redis generates a list of all possible matches from the index that satisfy the wildcard. For example, the query he* might expand to terms like hello, hero, and heat. Each of these matches is an "expansion."
  - This parameter limits how many of these expansions Redis will generate and process. If the number of possible matches exceeds the limit, the query may return incomplete results or fail, depending on the query context.
- Recommendations:
  - Avoid increasing this parameter excessively, as it can lead to performance bottlenecks during query execution.
  - If wildcard searches are common, consider optimizing your index to reduce the reliance on large wildcard expansions.

4. `DEFAULT_DIALECT`

- Purpose: specifies the default query dialect used by [`FT.SEARCH`]() and [`FT.AGGREGATE`]() commands.
- Default: [Dialect 1]().
- Recommendations:
  - Update the default to [**Dialect 4**]() for better performance and access to advanced features.
  - Individual commands can override this parameter if necessary, but setting a higher default ensures consistent performance across queries.

Testing

1. Correctness

- Run a few test queries and check the results are what you expect.
- Use the following tools to validate and debug:
  - Redis CLI: use the [`MONITOR`]() command or [profiling features]() in Redis Insight to analyze commands.
  - [`FT.PROFILE`](): Provides detailed insights into individual query execution paths, helping identify bottlenecks and inefficiencies.

2. Performance

- Test query performance in a controlled test environment that mirrors production as closely as possible.
- Use tools like `memtier_benchmark` or custom test applications to simulate load.
- Network Considerations:
  - Minimize latency during testing by locating test clients in the same network as the Redis instance.
  - For Redis Cloud, ensure test machines are in a **VPC-peered environment** with the target Redis database.

Sizing requirements

Redis Search has resource requirements distinct from general caching use cases. Proper sizing ensures that the system can handle production workloads efficiently.

Key considerations:

1. CPU:
   - Adequate CPU resources are critical<!--, especially when using QPF-->.
   - Ensure CPUs are not over-subscribed with search threads and shard processes.
2. RAM:
   - Plan for sufficient memory to store the dataset and indexes, plus overhead for operations.
3. Network:
   - High throughput and low latency are essential, particularly for applications with demanding query patterns.

Tools:

- Use [Redis Search Sizing Calculator](https://redis.io/redisearch-sizing-calculator/) to estimate resource requirements based on your dataset and workload.

Demand spikes

Production environments must be sized for peak load scenarios to ensure performance remains acceptable under maximum stress.

Recommendations:

1. Plan for Spikes:
   - If query workloads are expected to vary significantly, ensure the infrastructure can handle peak loads.
   - Monitor real-world usage patterns and adjust capacity as needed.
2. Autoscaling:
   - Consider using autoscaling strategies in cloud environments to dynamically adjust resources based on load.

By following these best practices, you can ensure a smooth and efficient transition from development to production with Redis Open Source and Redis Search. Proper configuration, rigorous testing, and careful resource planning are critical to delivering a reliable and high-performance Redis deployment.


Index management best practices for Redis Search

Introduction to managing Redis Search indexes

Redis Search is a powerful tool for executing complex search and query operations on structured, semi-structured, and unstructured data. Indexes are the backbone of this functionality, enabling fast and efficient data retrieval.
Proper management of these indexes is essential for optimal performance, scalability, and resource utilization.

This guide outlines best practices for managing Redis Search indexes throughout their lifecycle. It provides recommendations on:

- Planning and creating indexes to suit your query patterns.
- Using index aliasing to manage schema updates and minimize downtime.
- Monitoring and verifying index population to ensure query readiness.
- Optimizing performance through query profiling and memory management.
- Maintaining and scaling indexes in both standalone and clustered Redis environments.
- Versioning, testing, and automating index management.

Why index management matters

Indexes directly impact query speed and resource consumption.
Poorly managed indexes can lead to increased memory usage, slower query times, and challenges in maintaining data consistency.
By following the strategies outlined in this guide, you can:

- Reduce operational overhead.
- Improve application performance.
- Ensure smooth transitions during schema changes.
- Scale efficiently with your growing datasets.

Plan your indexes strategically

Planning your indexes strategically requires understanding your application’s query patterns and tailoring indexes to match.
Begin by identifying the types of searches your application performs—such as full-text search, range queries, or geospatial lookups—and the fields involved.
Categorize fields based on their purpose: searchable fields (e.g., [`TEXT`]() for full-text searches), filterable fields (e.g., [`TAG`]() for exact match searches), and sortable fields (e.g., [`NUMERIC`]() for range queries or sorting).
Match field types to their intended use and avoid indexing fields that are rarely queried to conserve resources. Here's the list of index types:

- [`TEXT`](): use `TEXT` for free-text searches and set weights if some fields are more important.
- [`TAG`](): use `TAG` for categorical data (e.g., product categories) that benefit from exact matching and filtering.
- [`NUMERIC`](): use `NUMERIC` for numeric ranges (e.g., prices, timestamps).
- [`GEO`](): use `GEO` for geospatial coordinates (e.g., latitude/longitude).
- [`GEOSHAPE`](): use `GEOSHAPE` to represent locations as points, but also to define shapes and query the interactions between points and shapes (e.g., to find all points that are contained within an enclosing shape).
- [`VECTOR`](): use `VECTOR` for high-dimensional similarity searches.

See [these pages]() for discussions and examples on how best to use these index types.

Next, simulate queries on a sample dataset to identify potential bottlenecks.
Use tools like [`FT.PROFILE`]() to analyze query execution and refine your schema if needed.
For example, assign weights to `TEXT` fields for prioritizing results or use the `PREFIX` option of [`FT.CREATE`]() to limit indexing to specific key patterns. Note that you can use multiple `PREFIX` clauses when you create an index (see [below](#index-creation))
After creating the index, validate its performance with real queries and monitor usage with the available tools:

- [`FT.EXPLAIN`]() and [`FT.EXPLAINCLI`]() allow you to see how Redis Search parses a given search query. `FT.EXPLAIN` returns a structured breakdown of the query execution plan, while `FT.EXPLAINCLI` presents a more readable, tree-like format for easier interpretation. These commands are useful for diagnosing query structure and ensuring it aligns with the intended logic.
- [`FT.INFO`]() provides detailed statistics about an index, including the number of indexed documents, memory usage, and configuration settings. It helps in monitoring index growth, assessing memory consumption, and verifying index structure to detect potential inefficiencies.
- [`FT.PROFILE`]() runs a query while capturing execution details, which helps to reveal query performance bottlenecks. It provides insights into processing time, key accesses, and filter application, making it a crucial tool for fine-tuning complex queries and optimizing search efficiency.

Avoid over-indexing. Indexing every field increases memory usage and can slow down updates.
Only index the fields that are essential for your planned queries.

Index creation {#index-creation}

- Use the [`FT.CREATE`]() command to define an index schema.
   - Assign weights to `TEXT` fields to prioritize certain fields in full-text search results.
   - Use the `PREFIX` option to restrict indexing to keys with specific patterns.
      Using multiple PREFIX clauses when creating an index allows you to index multiple key patterns under a single index. This is useful in several scenarios:
      - If your Redis database stores different types of entities under distinct key prefixes (e.g., `user:123`, `order:456`), a single index can cover both by specifying multiple prefixes. For example:

          [code example]

          This approach enables searching across multiple entity types without needing separate indexes.

      - Instead of querying multiple indexes separately, you can search across related data structures using a single query. This is particularly helpful when data structures share common fields, such as searching both customer and vendor records under a unified contacts index.

      - Maintaining multiple indexes for similar data types can be inefficient in terms of memory and query performance. By consolidating data under one index with multiple prefixes, you reduce overhead while still allowing for distinct key organization.

      - If your data model evolves and new key patterns are introduced, using multiple `PREFIX` clauses from the start ensures future compatibility without requiring a full reindexing.
   - Data loading strategy: load data into Redis before creating an index when working with large datasets. Use the `ON HASH` or `ON JSON` options to match the data structure.

Index aliasing

Index aliases act as abstracted names for the underlying indexes, enabling applications to reference the alias instead of the actual index name. This approach simplifies schema updates and index management.

There are several use cases for index aliasing, including:

- Schema updates: when updating an index schema, create a new index and associate the same alias with it. This allows a seamless transition without requiring application-level changes.
- Version control: use aliases to manage different versions of an index. For example, assign the alias products to `products_v1` initially and later to `products_v2` when the schema evolves.
- Testing and rollback: assign an alias to a test index during staged deployments. If issues arise, quickly switch the alias back to the stable index.

Best practices for aliasing:

- Always create an alias for your indexes during initial setup, even if you don’t anticipate immediate schema changes.
- Use clear and descriptive alias names to avoid confusion (e.g., `users_current` or `orders_live`).
- Make sure that an alias points to only one index at a time to maintain predictable query results.
- Use aliases to provide tenant-specific access. For example, assign tenant-specific aliases like `tenant1_products` and `tenant2_products` to different indexes for isolated query performance.

Tools for managing aliases:

- Assign an alias: [`FT.ALIASADD`]() `my_alias my_index`
- Update an alias: [`FT.ALIASUPDATE`]() `my_alias new_index`
- Remove an alias: [`FT.ALIASDEL`]() `my_alias`

Monitoring and troubleshooting aliases:

- Use the `FT.INFO` command to check which aliases are associated with an index.
- Make sure your aliases always points to valid indexes and are correctly updated during schema changes.

Monitor index population

- Use the `FT.INFO` command to monitor the `num_docs` and `indexing` fields, to check that all expected documents are indexed.
    [code example]
- Validate data with sample queries to ensure proper indexing:
    [code example]
- Use `FT.PROFILE` to analyze query plans and validate performance:

    [code example]
 - Implement scripts to periodically verify document counts and query results. For example, in Python:

    [code example]

Monitoring index performance

- Use the `FT.PROFILE` command to analyze query performance and identify bottlenecks.
- Regularly monitor memory usage with the [`INFO`]() `memory` and `FT.INFO` commands to detect growth patterns and optimize resource allocation.

Index maintenance

- If schema changes are required, create a new index with the updated schema and reassign the alias once the index is ready.
- Use [Redis key expiration]() to automatically remove outdated records and keep indexes lean.

FT.ALTER vs. aliasing

Use [`FT.ALTER`]() when you need to add new fields to an existing index without rebuilding it, minimizing downtime and resource usage. However, `FT.ALTER` cannot remove or modify existing fields, limiting its flexibility.

Use index aliasing when making schema changes that require reindexing, such as modifying field types or removing fields. In this case, create a new index with the updated schema, populate it, and then use `FT.ALIASUPDATE` to seamlessly switch queries to the new index without disrupting application functionality.

Scaling and high availability

- In a clustered Redis setup, make sure indexes are designed with key distribution in mind to prevent query inefficiencies.
- Test how indexes behave under replica promotion to ensure consistent query behavior across nodes.

Versioning and testing

- When changing schemas, create a new version of the index alongside the old one and migrate data progressively.
- Test index changes in a staging environment before deploying them to production.

Cleaning up

- Use the [`FT.DROPINDEX`]() command to remove unused indexes and free up memory. Be cautious with the `DD` (Delete Documents) flag to avoid unintended data deletion.
- Make sure no keys remain that were previously associated with dropped indexes if the data is no longer relevant.

Documentation and automation

- Document your index configurations to facilitate future maintenance.
- Use scripts or orchestration tools to automate index creation, monitoring, and cleanup.

FT.CREATE my_index ON HASH PREFIX 2 "user:" "order:" SCHEMA name TEXT age NUMERIC status TAG

FT.PROFILE my_new_index SEARCH QUERY "your_query"

import re
    def check_index_readiness(index_name, expected_docs):
        r = redis.StrictRedis(host='localhost', port=6379, decode_responses=True)
        info = r.execute_command('FT.INFO', index_name)
        num_docs = int(info[info.index('num_docs') + 1])
        return num_docs >= expected_d
    if check_index_readiness('my_new_index', 100000):
        print("Index is fully populated!")
    else:
        print("Index is still populating...")


Best practices

Summary: Redis Search best practices

Best practices for Redis Search performance


Overview

If you're using Redis Software or Redis Cloud, see the [best practices for scalable Redis Search]() page.

Checklist

Below are some basic steps to ensure good performance of Redis Search .

* Create a Redis data model with your query patterns in mind.
* Ensure the Redis architecture has been sized for the expected load using the [sizing calculator](https://redis.io/redisearch-sizing-calculator/).
* Provision Redis nodes with sufficient resources (RAM, CPU, network) to support the expected maximum load.
* Review [`FT.INFO`]() and [`FT.PROFILE`]() outputs for anomalies and/or errors.
* Conduct load testing in a test environment with real-world queries and a load generated by either [memtier_benchmark](https://github.com/redislabs/memtier_benchmark) or a custom load application.

Indexing considerations

General

- Favor [`TAG`]() over [`NUMERIC`]() for use cases that only require matching.
- Favor [`TAG`]() over [`TEXT`]() for use cases that don’t require full-text capabilities (pure match).

Non-threaded search

- Put only those fields used in your queries in the index.
- Only make fields [`SORTABLE`]() if they are used in [`SORTBY`]()
queries.
- Use [`DIALECT 2`]().

Threaded (query performance factor or QPF) search

- Put both query fields and any projected fields (`RETURN` or `LOAD`) in the index.
- Set all fields to `SORTABLE`.
- Set TAG fields to [UNF]().
- Optional: Set `TEXT` fields to `NOSTEM` if the use case will support it.
- Use [`DIALECT 2`]().

Query optimization

- Avoid returning large result sets.  Use `CURSOR` or `LIMIT`.
- Avoid wildcard searches.
- Avoid projecting all fields (e.g., `LOAD *`). Project only those fields that are part of the index schema.
- If queries are long-running, enable threading (query performance factor) to reduce contention for the main Redis thread.

Validate performance (`FT.PROFILE`)

You can analyze [`FT.PROFILE`]() output to gain insights about query execution.
The following informational items are available for analysis:

- Total execution time
- Execution time per shard
- Coordination time (for multi-sharded environments)
- Breakdown of the query into fundamental components, such as `UNION` and `INTERSECT`
- Warnings, such as `TIMEOUT`

Anti-patterns

When designing and querying indexes in Redis Search, certain practices can hinder performance, scalability, and maintainability. Below are some common anti-patterns to avoid:

- **Large documents**: storing excessively large documents in Redis makes data retrieval slower and increases memory usage. Break data into smaller, focused records whenever possible.
- **Deeply-nested fields**: retrieving or indexing deeply-nested JSON fields is computationally expensive. Use a flatter schema for better performance.
- **Large result sets**: fetching unnecessarily large result sets puts a strain on memory and network resources. Limit results to only what is needed.
- **Wildcarding**: using wildcard patterns indiscriminately in queries can lead to large and inefficient scans, especially if the index size is significant.
- **Large projections**: including excessive fields in query results increases memory overhead and slows down query execution. Limit projections to essential fields.

The following examples depict an anti-pattern index schema and query, followed by corrected versions designed for scalability with Redis Search.

Anti-pattern index schema

The following schema introduces challenges for scalability and performance:

[code example]

Issues:

- Minimal schema definition: the schema is sparse and lacks fields like `lastName`, `id`, and `version` that might be frequently queried. This results in additional operations to fetch these fields separately, reducing efficiency.
- Missing `SORTABLE` flag for text fields: sorting operations on unsortable fields require full-text processing, which is slow.
- Wildcard indexing: `$.tags.*` creates a broad index that can lead to excessive memory usage and reduced query performance.

Anti-pattern query

The following query is inefficient and not optimized for vertical scaling:

[code example]
Issues:

- Wildcard projection (`LOAD *`): retrieving all fields in the result set is inefficient and increases memory usage, especially if the documents are large.
- Unnecessary fields: fields that aren't required for the current operation are still fetched, slowing down execution.
- Lack of advanced query syntax: without specifying a query dialect or leveraging features like tagging, the query may perform unnecessary computations.

Improved index schema

Here’s an optimized schema that adheres to best practices for vertical scaling:

[code example]

Improvements:

- `NOSTEM` for text fields: prevents stemming on fields like `firstName` and `lastName` to allow for exact matches (e.g., "Smith" stays "Smith").
- Expanded schema: adds commonly queried fields like `lastName`, `id`, and `version`, making queries more efficient by reducing the need for post-query data retrieval.
- `TAG` fields: `id` and `ver` are defined as `TAG` fields to support fast filtering with exact matches.
- `SORTABLE` for all relevant fields: ensures that sorting operations are efficient without requiring full-text scanning.

You might be wondering why `$.tags.* as t NUMERIC SORTABLE` is acceptable in the improved schema and it wasn't previously.
The inclusion of `$.tags.*` is acceptable when:

- It has a clear purpose: it is actively used in queries, such as filtering on numeric ranges or matching specific values.
- Other fields in the schema complement it: these fields reduce over-reliance on `$.tags.*` for all query operations, distributing the load more evenly.
- Projections and limits are managed carefully: queries that use `$.tags.*` should avoid loading unnecessary fields or returning excessively large result sets.

Improved query

The following query is better suited for vertical scaling:

[code example]

Improvements:

- Targeted projection: the `LOAD` clause specifies only essential fields (`id, t, name, lastname, loc, ver`), reducing memory and network overhead.
- Limited results: the `LIMIT` clause ensures the query retrieves only the first 10 results, avoiding large result sets.
- [`DIALECT 2`](): enables the latest Redis Search syntax and features, ensuring compatibility with modern capabilities.

FT.CREATE jsonidx:profiles ON JSON PREFIX 1 profiles:
          SCHEMA $.tags.* as t NUMERIC SORTABLE
                 $.firstName as name TEXT
                 $.location as loc GEO

FT.AGGREGATE jsonidx:profiles '@t:[1299 1299]' LOAD * LIMIT 0 10

FT.CREATE jsonidx:profiles ON JSON PREFIX 1 profiles:
          SCHEMA $.tags.* as t NUMERIC SORTABLE
                 $.firstName as name TEXT NOSTEM SORTABLE
                 $.lastName as lastname TEXT NOSTEM SORTABLE
                 $.location as loc GEO SORTABLE
                 $.id as id TAG SORTABLE UNF
                 $.ver as ver TAG SORTABLE UNF

FT.AGGREGATE jsonidx:profiles '@t:[1299 1299]'
                LOAD 6 id t name lastname loc ver
                LIMIT 0 10
                DIALECT 2