Vectors

Learn how to use vector fields and perform vector searches in Redis

Redis includes a high-performance vector database that lets you perform semantic searches over vector embeddings. You can augment these searches with filtering over text, numerical, geospatial, and tag metadata.

To quickly get started, check out the Redis vector quickstart guide and the Redis AI Resources Github repo.

Overview

1. Create a vector index: Redis maintains a secondary index over your data with a defined schema (including vector fields and metadata). Redis supports FLAT and HNSW vector index types.
2. Store and update vectors: Redis stores vectors and metadata in hashes or JSON objects.
3. Search with vectors: Redis supports several advanced querying strategies with vector fields including k-nearest neighbor (KNN), vector range queries, and metadata filters.
4. Configure vector queries at runtime.
5. Vector search examples: Explore several vector search examples that cover different use cases and techniques.

Create a vector index

When you define the schema for an index, you can include one or more vector fields as shown below.

Syntax

FT.CREATE <index_name>
  ON <storage_type>
  PREFIX 1 <key_prefix>
  SCHEMA ... <field_name> VECTOR <algorithm> <index_attribute_count> <index_attribute_name> <index_attribute_value>
    [<index_attribute_name> <index_attribute_value> ...]

Refer to the full indexing documentation for additional fields, options, and noted limitations.

Parameters

FLAT index

Choose the FLAT index when you have small datasets (< 1M vectors) or when perfect search accuracy is more important than search latency.

Required attributes

Example

FT.CREATE documents
  ON HASH
  PREFIX 1 docs:
  SCHEMA doc_embedding VECTOR FLAT 6
    TYPE FLOAT32
    DIM 1536
    DISTANCE_METRIC COSINE

In the example above, an index named documents is created over hashes with the key prefix docs: and a FLAT vector field named doc_embedding with three index attributes: TYPE, DIM, and DISTANCE_METRIC.

HNSW index

HNSW, or hierarchical navigable small world, is an approximate nearest neighbors algorithm that uses a multi-layered graph to make vector search more scalable.

• The lowest layer contains all data points, and each higher layer contains a subset, forming a hierarchy.
• At runtime, the search traverses the graph on each layer from top to bottom, finding the local minima before dropping to the subsequent layer.

Choose the HNSW index type when you have larger datasets (> 1M documents) or when search performance and scalability are more important than perfect search accuracy.

Required attributes

Optional attributes

HNSW supports a number of additional parameters to tune the accuracy of the queries, while trading off performance.

Example

FT.CREATE documents
  ON HASH
  PREFIX 1 docs:
  SCHEMA doc_embedding VECTOR HNSW 10
    TYPE FLOAT64
    DIM 1536
    DISTANCE_METRIC COSINE
    M 40
    EF_CONSTRUCTION 250

In the example above, an index named documents is created over hashes with the key prefix docs: and an HNSW vector field named doc_embedding with five index attributes: TYPE, DIM, DISTANCE_METRIC, M, and EF_CONSTRUCTION.

Distance metrics

Redis supports three popular distance metrics to measure the degree of similarity between two vectors $u$, $v$ $\in \mathbb{R}^n$, where $n$ is the length of the vectors:

The above metrics calculate distance between two vectors, where the smaller the value is, the closer the two vectors are in the vector space.

Store and update vectors

On index creation, the <storage_type> dictates how vector and metadata are structured and loaded into Redis.

Hash

Store or update vectors and any metadata in hashes using the HSET command.

Example

HSET docs:01 doc_embedding <vector_bytes> category sports

A common method for converting vectors to bytes uses the redis-py client library and the Python NumPy library.

Example

import numpy as np
from redis import Redis

redis_client = Redis(host='localhost', port=6379)

# Create a FLOAT32 vector
vector = np.array([0.34, 0.63, -0.54, -0.69, 0.98, 0.61], dtype=np.float32)

# Convert vector to bytes
vector_bytes = vector.tobytes()

# Use the Redis client to store the vector bytes and metadata at a specified key
redis_client.hset('docs:01', mapping = {"vector": vector_bytes, "category": "sports"})

JSON

You can store or update vectors and any associated metadata in JSON using the JSON.SET command.

To store vectors in Redis as JSON, you store the vector as a JSON array of floats. Note that this differs from vector storage in Redis hashes, which are instead stored as raw bytes.

Example

JSON.SET docs:01 $ '{"doc_embedding":[0.34,0.63,-0.54,-0.69,0.98,0.61], "category": "sports"}'

One of the benefits of JSON is schema flexibility. As of v2.6.1, JSON supports multi-value indexing. This allows you to index multiple vectors under the same JSONPath.

Here are some examples of multi-value indexing with vectors:

Multi-value indexing example

JSON.SET docs:01 $ '{"doc_embedding":[[1,2,3,4], [5,6,7,8]]}'
JSON.SET docs:01 $ '{"chunk1":{"doc_embedding":[1,2,3,4]}, "chunk2":{"doc_embedding":[5,6,7,8]}}'

Additional information and examples are available in the Indexing JSON documents section.

Search with vectors

You can run vector search queries with the FT.SEARCH or FT.AGGREGATE commands.

To issue a vector search query with FT.SEARCH, you must set the DIALECT option to >= 2. See the dialects documentation for more information.

KNN vector search

KNN vector search finds the top k nearest neighbors to a query vector. It has the following syntax:

Syntax

FT.SEARCH <index_name>
  <primary_filter_query>=>[KNN <top_k> @<vector_field> $<vector_blob_param> $<vector_query_params> AS <distance_field>]
  PARAMS <query_params_count> [$<vector_blob_param> <vector_blob> <query_param_name> <query_param_value> ...]
  SORTBY <distance_field>
  DIALECT 4

Parameters

Example

FT.SEARCH documents "*=>[KNN 10 @doc_embedding $BLOB]" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" DIALECT 4

Use query attributes

Alternatively, as of v2.6, <vector_query_params> and <distance_field> name can be specified in runtime query attributes as shown below.

[KNN <top_k> @<vector_field> $<vector_blob_param>]=>{$yield_distance_as: <distance_field>}

Vector range queries

Vector range queries allow you to filter the index using a radius parameter representing the semantic distance between an input query vector and indexed vector fields. This is useful in scenarios when you don't know exactly how many nearest (top_k) neighbors to fetch, but you do know how similar the results should be.

For example, imagine a fraud or anomaly detection scenario where you aren't sure if there are any matches in the vector index. You can issue a vector range query to quickly check if there are any records of interest in the index within the specified radius.

Vector range queries operate slightly different than KNN vector queries:

• Vector range queries can appear multiple times in a query as filter criteria.
• Vector range queries can be a part of the <primary_filter_query> in KNN vector search.

Syntax

FT.SEARCH <index_name>
  @<vector_field>:[VECTOR_RANGE (<radius> | $<radius_param>) $<vector_blob_param> $<vector_query_params>]
  PARAMS <vector_query_params_count> [<vector_query_param_name> <vector_query_param_value> ...]
  SORTBY <distance_field>
  DIALECT 4

Use query attributes

A vector range query clause can be followed by a query attributes section as follows:

@<vector_field>: [VECTOR_RANGE (<radius> | $<radius_param>) $<vector_blob_param>]=>{$<param>: (<value> |
    $<value_attribute>); ... }

where the relevant parameters in that case are $yield_distance_as and $epsilon. Note that there is no default distance field name in range queries.

Filters

Redis supports vector searches that include filters to narrow the search space based on defined criteria. If your index contains searchable fields (for example, TEXT, TAG, NUMERIC, GEO, GEOSHAPE, and VECTOR), you can perform vector searches with filters.

Supported filter types

• Exact match
• Numeric range
• Full-text
• Geospatial

You can also combine multiple queries as a filter.

Syntax

Vector search queries with filters follow this basic structure:

FT.SEARCH <index_name> <primary_filter_query>=>[...]

where <primary_filter_query> defines document selection and filtering.

Example

FT.SEARCH documents "(@title:Sports @year:[2020 2022])=>[KNN 10 @doc_embedding $BLOB]" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" DIALECT 4

How filtering works

Redis uses internal algorithms to optimize the filtering computation for vector search. The runtime algorithm is determined by heuristics that aim to minimize query latency based on several factors derived from the query and the index.

Batches mode

Batches mode works by paginating through small batches of nearest neighbors from the index:

• A batch of high-scoring documents from the vector index is retrieved. These documents are yielded only if the <primary_filter_query> is satisfied. In other words, the document must contain a similar vector and meet the filter criteria.
• The iterative procedure terminates when <top_k> documents that pass the filter criteria are yielded, or after every vector in the index has been processed.
• The batch size is determined automatically by heuristics based on <top_k> and the ratio between the expected number of documents in the index that pass the <primary_filter_query> and the vector index size.
• The goal is to minimize the total number of batches required to get the <top_k> results while preserving the smallest batch size possible. Note that the batch size may change dynamically in each iteration based on the number of results that pass the filter in previous batches.

Ad-hoc brute force mode

• The score of every vector corresponding to a document that passes the filter is computed, and the <top_k> results are selected and returned.
• This approach is preferable when the number of documents passing the <primary_filter_query> is relatively small.
• The results of the KNN query will always be accurate in this mode, even if the underlying vector index algorithm is an approximate one.

The execution mode may switch from batch mode to ad-hoc brute-force mode during the run, based on updated estimations of relevant factors from one batch to another.

Runtime query parameters

Filter mode

By default, Redis selects the best filter mode to optimize query execution. You can override the auto-selected policy using these optional parameters:

Index-specific query parameters

FLAT

Currently, there are no runtime parameters available for FLAT indexes.

HNSW

Optional runtime parameters for HNSW indexes are:

Important notes

Vector search examples

Below are a number of examples to help you get started. For more comprehensive walkthroughs, see the Redis vector quickstart guide and the Redis AI Resources Github repo.

KNN vector search examples

Return the 10 nearest neighbor documents for which the doc_embedding vector field is the closest to the query vector represented by the following 4-byte blob:

FT.SEARCH documents "*=>[KNN 10 @doc_embedding $BLOB]" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" SORTBY __vector_score DIALECT 4

Return the top 10 nearest neighbors and customize the K and EF_RUNTIME parameters using query parameters. See the "Optional arguments" section in FT.SEARCH command. Set the EF_RUNTIME value to 150, assuming doc_embedding is an HNSW index:

FT.SEARCH documents "*=>[KNN $K @doc_embedding $BLOB EF_RUNTIME $EF]" PARAMS 6 BLOB "\x12\xa9\xf5\x6c" K 10 EF 150 DIALECT 4

Assign a custom name to the distance field (vector_distance) and then sort using that name:

FT.SEARCH documents "*=>[KNN 10 @doc_embedding $BLOB AS vector_distance]" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" SORTBY vector_distance DIALECT 4

Use query attributes syntax to specify optional parameters and the distance field name:

FT.SEARCH documents "*=>[KNN 10 @doc_embedding $BLOB]=>{$EF_RUNTIME: $EF; $YIELD_DISTANCE_AS: vector_distance}" PARAMS 4 EF 150 BLOB "\x12\xa9\xf5\x6c" SORTBY vector_distance DIALECT 4

To explore additional Python vector search examples, review recipes for the Redis Python client library and the Redis Vector Library.

Filter examples

For these examples, assume you created an index named movies with records of different movies and their metadata.

Among the movies that have 'Dune' in the title field and year between [2020, 2022], return the top 10 nearest neighbors, sorted by movie_distance:

FT.SEARCH movies "(@title:Dune @year:[2020 2022])=>[KNN 10 @movie_embedding $BLOB AS movie_distance]" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" SORTBY movie_distance DIALECT 4

Among the movies that have action as a category tag, but not drama, return the top 10 nearest neighbors, sorted by movie_distance:

FT.SEARCH movies "(@category:{action} ~@category:{drama})=>[KNN 10 @doc_embedding $BLOB AS movie_distance]" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" SORTBY movie_distance DIALECT 4

Among the movies that have drama or action as a category tag, return the top 10 nearest neighbors and explicitly set the filter mode (hybrid policy) to "ad-hoc brute force" rather than it being auto-selected:

FT.SEARCH movies "(@category:{drama | action})=>[KNN 10 @doc_embedding $BLOB HYBRID_POLICY ADHOC_BF]" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" SORTBY __vec_score DIALECT 4

Among the movies that have action as a category tag, return the top 10 nearest neighbors and explicitly set the filter mode (hybrid policy) to "batches" and batch size 50 using a query parameter:

FT.SEARCH movies "(@category:{action})=>[KNN 10 @doc_embedding $BLOB HYBRID_POLICY BATCHES BATCH_SIZE $BATCH_SIZE]" PARAMS 4 BLOB "\x12\xa9\xf5\x6c" BATCH_SIZE 50 DIALECT 4

Run the same query as above and use the query attributes syntax to specify optional parameters:

FT.SEARCH movies "(@category:{action})=>[KNN 10 @doc_embedding $BLOB]=>{$HYBRID_POLICY: BATCHES; $BATCH_SIZE: 50}" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" DIALECT 4

To explore additional Python vector search examples, review recipes for the Redis Python client library and the Redis Vector Library.

Range query examples

For these examples, assume you created an index named products with records of different products and metadata from an ecommerce site.

Return 100 products for which the distance between the description_vector field and the specified query vector blob is at most 5:

FT.SEARCH products "@description_vector:[VECTOR_RANGE 5 $BLOB]" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" LIMIT 0 100 DIALECT 4

Run the same query as above and set the EPSILON parameter to 0.5, assuming description_vector is HNSW index, yield the vector distance between description_vector and the query result in a field named vector_distance, and sort the results by that distance.

FT.SEARCH products "@description_vector:[VECTOR_RANGE 5 $BLOB]=>{$EPSILON:0.5; $YIELD_DISTANCE_AS: vector_distance}" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" SORTBY vector_distance LIMIT 0 100 DIALECT 4

Use the vector range query as a filter: return all the documents that contain either 'shirt' in their type tag with their year value in the range [2020, 2022] or a vector stored in description_vector whose distance from the query vector is no more than 0.8, then sort the results by their vector distance, if it is in the range:

FT.SEARCH products "(@type:{shirt} @year:[2020 2022]) | @description_vector:[VECTOR_RANGE 0.8 $BLOB]=>{$YIELD_DISTANCE_AS: vector_distance}" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" SORTBY vector_distance DIALECT 4

To explore additional Python vector search examples, review recipes for the Redis Python client library and the Redis Vector Library.

Next steps

Vector embeddings and vector search are not new concepts. Many of the largest companies have used vectors to represent products in ecommerce catalogs or content in advertising pipelines for well over a decade.

With the emergence of Large Language Models (LLMs) and the proliferation of applications that require advanced information retrieval techniques, Redis is well positioned to serve as your high performance query engine for semantic search and more.

Here are some additonal resources that apply vector search for different use cases:

• Retrieval augmented generation from scratch
• Semantic caching