User Guides on Docs

Getting Started with RedisVL

Mon, 01 Jan 0001 00:00:00 +0000

redisvl is a versatile Python library with an integrated CLI, designed to enhance AI applications using Redis. This guide will walk you through the following steps:

Defining an IndexSchema
Preparing a sample dataset
Creating a SearchIndex object
Testing rvl CLI functionality
Loading the sample data
Building VectorQuery objects and executing searches
Updating a SearchIndex object

...and more!

Prerequisites:

Ensure redisvl is installed in your Python environment.
Have a running instance of Redis Stack or Redis Cloud.

Define an `IndexSchema`

The IndexSchema maintains crucial index configuration and field definitions to enable search with Redis. For ease of use, the schema can be constructed from a python dictionary or yaml file.

Querying with RedisVL

Mon, 01 Jan 0001 00:00:00 +0000

In this notebook, we will explore more complex queries that can be performed with redisvl

Before running this notebook, be sure to

Have installed redisvl and have that environment active for this notebook.
Have a running Redis instance with RediSearch > 2.4 running.

import pickle
from jupyterutils import table_print, result_print

# load in the example data and printing utils
data = pickle.load(open("hybrid_example_data.pkl", "rb"))
table_print(data)

user	age	job	credit_score	office_location	user_embedding	last_updated
john	18	engineer	high	-122.4194,37.7749	b'\xcd\xcc\xcc=\xcd\xcc\xcc=\x00\x00\x00?'	1741627789
derrick	14	doctor	low	-122.4194,37.7749	b'\xcd\xcc\xcc=\xcd\xcc\xcc=\x00\x00\x00?'	1741627789
nancy	94	doctor	high	-122.4194,37.7749	b'333?\xcd\xcc\xcc=\x00\x00\x00?'	1710696589
tyler	100	engineer	high	-122.0839,37.3861	b'\xcd\xcc\xcc=\xcd\xcc\xcc>\x00\x00\x00?'	1742232589
tim	12	dermatologist	high	-122.0839,37.3861	b'\xcd\xcc\xcc>\xcd\xcc\xcc>\x00\x00\x00?'	1739644189
taimur	15	CEO	low	-122.0839,37.3861	b'\x9a\x99\x19?\xcd\xcc\xcc=\x00\x00\x00?'	1742232589
joe	35	dentist	medium	-122.0839,37.3861	b'fff?fff?\xcd\xcc\xcc='	1742232589

schema = {
    "index": {
        "name": "user_queries",
        "prefix": "user_queries_docs",
        "storage_type": "hash", # default setting -- HASH
    },
    "fields": [
        {"name": "user", "type": "tag"},
        {"name": "credit_score", "type": "tag"},
        {"name": "job", "type": "text"},
        {"name": "age", "type": "numeric"},
        {"name": "last_updated", "type": "numeric"},
        {"name": "office_location", "type": "geo"},
        {
            "name": "user_embedding",
            "type": "vector",
            "attrs": {
                "dims": 3,
                "distance_metric": "cosine",
                "algorithm": "flat",
                "datatype": "float32"
            }

        }
    ],
}

from redisvl.index import SearchIndex

# construct a search index from the schema
index = SearchIndex.from_dict(schema, redis_url="redis://localhost:6379")

# create the index (no data yet)
index.create(overwrite=True)

# use the CLI to see the created index
!rvl index listall

13:00:56 [RedisVL] INFO   Indices:
13:00:56 [RedisVL] INFO   1. user_queries

# load data to redis
keys = index.load(data)

index.info()['num_docs']

Hybrid Queries

Hybrid queries are queries that combine multiple types of filters. For example, you may want to search for a user that is a certain age, has a certain job, and is within a certain distance of a location. This is a hybrid query that combines numeric, tag, and geographic filters.

LLM Caching

Mon, 01 Jan 0001 00:00:00 +0000

This notebook demonstrates how to use RedisVL's SemanticCache to cache LLM responses based on semantic similarity. Semantic caching can significantly reduce API costs and latency by retrieving cached responses for semantically similar prompts instead of making redundant API calls.

Key features covered:

Basic cache operations (store, check, clear)
Customizing semantic similarity thresholds
TTL policies for cache expiration
Performance benchmarking
Access controls with tags and filters for multi-user scenarios

Prerequisites:

Ensure redisvl is installed in your Python environment
Have a running instance of Redis Stack or Redis Cloud
OpenAI API key for the examples

First, we will import OpenAI to use their API for responding to user prompts. We will also create a simple ask_openai helper method to assist.

Vectorizers

Mon, 01 Jan 0001 00:00:00 +0000

In this notebook, we will show how to use RedisVL to create embeddings using the built-in text embedding vectorizers. Today RedisVL supports:

OpenAI
HuggingFace
Vertex AI
Cohere
Mistral AI
Amazon Bedrock
Bringing your own vectorizer
VoyageAI

Before running this notebook, be sure to

Have installed redisvl and have that environment active for this notebook.
Have a running Redis Stack instance with RediSearch > 2.4 active.

For example, you can run Redis Stack locally with Docker:

Hash vs JSON Storage

Mon, 01 Jan 0001 00:00:00 +0000

Out of the box, Redis provides a variety of data structures that can adapt to your domain specific applications and use cases. In this notebook, we will demonstrate how to use RedisVL with both Hash and JSON data.

Before running this notebook, be sure to

Have installed redisvl and have that environment active for this notebook.
Have a running Redis Stack or Redis Enterprise instance with RediSearch > 2.4 activated.

For example, you can run Redis Stack locally with Docker:

Rerankers

Mon, 01 Jan 0001 00:00:00 +0000

In this notebook, we will show how to use RedisVL to rerank search results (documents or chunks or records) based on the input query. Today RedisVL supports reranking through:

A re-ranker that uses pre-trained Cross-Encoders which can use models from Hugging Face cross encoder models or Hugging Face models that implement a cross encoder function (example: BAAI/bge-reranker-base).
The Cohere /rerank API.
The VoyageAI /rerank API.

Before running this notebook, be sure to:

Have installed redisvl and have that environment active for this notebook.
Have a running Redis Stack instance with RediSearch > 2.4 active.

For example, you can run Redis Stack locally with Docker:

LLM Message History

Mon, 01 Jan 0001 00:00:00 +0000

Large Language Models are inherently stateless and have no knowledge of previous interactions with a user, or even of previous parts of the current conversation. While this may not be noticeable when asking simple questions, it becomes a hindrance when engaging in long running conversations that rely on conversational context.

The solution to this problem is to append the previous conversation history to each subsequent call to the LLM.

This notebook will show how to use Redis to structure and store and retrieve this conversational message history.

Semantic Routing

Mon, 01 Jan 0001 00:00:00 +0000

RedisVL provides a SemanticRouter interface to utilize Redis' built-in search & aggregation in order to perform KNN-style classification over a set of Route references to determine the best match.

This notebook will go over how to use Redis as a Semantic Router for your applications

Define the Routes

Below we define 3 different routes. One for technology, one for sports, and another for entertainment. Now for this example, the goal here is surely topic "classification". But you can create routes and references for almost anything.

SVS-VAMANA Vector Search

Mon, 01 Jan 0001 00:00:00 +0000

In this notebook, we will explore SVS-VAMANA (Scalable Vector Search with VAMANA graph algorithm), a graph-based vector search algorithm that is optimized to work with compression methods to reduce memory usage. It combines the Vamana graph algorithm with advanced compression techniques (LVQ and LeanVec) and is optimized for Intel hardware.

How it works

Vamana builds a single-layer proximity graph and prunes edges during construction based on tunable parameters, similar to HNSW but with a simpler structure. The compression methods apply per-vector normalization and scalar quantization, learning parameters directly from the data to enable fast, on-the-fly distance computations with SIMD-optimized layout Vector quantization and compression.

Caching Embeddings

Mon, 01 Jan 0001 00:00:00 +0000

RedisVL provides an EmbeddingsCache that makes it easy to store and retrieve embedding vectors with their associated text and metadata. This cache is particularly useful for applications that frequently compute the same embeddings, enabling you to:

Reduce computational costs by reusing previously computed embeddings
Decrease latency in applications that rely on embeddings
Store additional metadata alongside embeddings for richer applications

This notebook will show you how to use the EmbeddingsCache effectively in your applications.

Advanced Query Types

Mon, 01 Jan 0001 00:00:00 +0000

In this notebook, we will explore advanced query types available in RedisVL:

TextQuery: Full text search with advanced scoring
AggregateHybridQuery and HybridQuery: Combines text and vector search for hybrid retrieval
MultiVectorQuery: Search over multiple vector fields simultaneously

These query types are powerful tools for building sophisticated search applications that go beyond simple vector similarity search.

Prerequisites:

Ensure redisvl is installed in your Python environment.
Have a running instance of Redis Stack or Redis Cloud.
For HybridQuery, we will need Redis >= 8.4.0 and redis-py >= 7.1.0.

Setup and Data Preparation

First, let's create a schema and prepare sample data that includes text fields, numeric fields, and vector fields.

User Guides on Docs

Getting Started with RedisVL

Define an IndexSchema

Querying with RedisVL

Hybrid Queries

LLM Caching

Vectorizers

Hash vs JSON Storage

Rerankers

LLM Message History

Semantic Routing

Define the Routes

SVS-VAMANA Vector Search

Caching Embeddings

Advanced Query Types

Setup and Data Preparation

Define an `IndexSchema`