Index and query vectors

Summary: Learn how to index and query vector embeddings with Redis

Initialize

Install [`redis-py`]() if you
have not already done so. Also, install `sentence-transformers` with the
following command:

[code example]

In a new Python source file, start by importing the required classes:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

The first of these imports is the
`SentenceTransformer` class, which generates an embedding from a section of text.
Here, we create an instance of `SentenceTransformer` that uses the
[`all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
model for the embeddings. This model generates vectors with 384 dimensions, regardless
of the length of the input text, but note that the input is truncated to 256
tokens (see
[Word piece tokenization](https://huggingface.co/learn/nlp-course/en/chapter6/6)
at the [Hugging Face](https://huggingface.co/) docs to learn more about the way tokens
are related to the original text).

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Create the index

Connect to Redis and delete any index previously created with the
name `vector_idx`. (The `dropindex()` call throws an exception if
the index doesn't already exist, which is why you need the
`try: except:` block.)

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Next, create the index.
The schema in the example below specifies hash objects for storage and includes
three fields: the text content to index, a
[tag]()
field to represent the "genre" of the text, and the embedding vector generated from
the original text content. The `embedding` field specifies
[HNSW]()
indexing, the
[L2]()
vector distance metric, `Float32` values to represent the vector's components,
and 384 dimensions, as required by the `all-MiniLM-L6-v2` embedding model.

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Add data

You can now supply the data objects, which will be indexed automatically
when you add them with [`hset()`](), as long as
you use the `doc:` prefix specified in the index definition.

Use the `model.encode()` method of `SentenceTransformer`
as shown below to create the embedding that represents the `content` field.
The `astype()` option that follows the `model.encode()` call specifies that
we want a vector of `float32` values. The `tobytes()` option encodes the
vector components together as a single binary string.
Use the binary string representation when you are indexing hashes
or running a query (but use a list of `float` for
[JSON documents](#differences-with-json-documents)).

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Run a query

After you have created the index and added the data, you are ready to run a query.
To do this, you must create another embedding vector from your chosen query
text. Redis calculates the similarity between the query vector and each
embedding vector in the index as it runs the query. It then ranks the
results in order of this numeric similarity value.

The code below creates the query embedding using `model.encode()`, as with
the indexing, and passes it as a parameter when the query executes
(see
[Vector search]()
for more information about using query parameters with embeddings).

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

The code is now ready to run, but note that it may take a while to complete when
you run it for the first time (which happens because RedisVL must download the
`all-MiniLM-L6-v2` model data before it can
generate the embeddings). When you run the code, it outputs the following result
object (slightly formatted here for clarity):

[code example]

Note that the results are ordered according to the value of the `vector_distance`
field, with the lowest distance indicating the greatest similarity to the query.
As you would expect, the result for `doc:0` with the content text *"That is a very happy person"*
is the result that is most similar in meaning to the query text
*"That is a happy person"*.

Differences with JSON documents

Indexing JSON documents is similar to hash indexing, but there are some
important differences. JSON allows much richer data modelling with nested fields, so
you must supply a [path]() in the schema
to identify each field you want to index. However, you can declare a short alias for each
of these paths (using the `as_name` keyword argument) to avoid typing it in full for
every query. Also, you must specify `IndexType.JSON` when you create the index.

The code below shows these differences, but the index is otherwise very similar to
the one created previously for hashes:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Use [`json().set()`]() to add the data
instead of [`hset()`](). The dictionaries
that specify the fields have the same structure as the ones used for `hset()`
but `json().set()` receives them in a positional argument instead of
the `mapping` keyword argument.

An important difference with JSON indexing is that the vectors are
specified using lists instead of binary strings. Generate the list
using the `tolist()` method instead of `tobytes()` as you would with a
hash.

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

The query is almost identical to the one for the hash documents. This
demonstrates how the right choice of aliases for the JSON paths can
save you having to write complex queries. An important thing to notice
is that the vector parameter for the query is still specified as a
binary string (using the `tobytes()` method), even though the data for
the `embedding` field of the JSON was specified as a list.

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Apart from the `jdoc:` prefixes for the keys, the result from the JSON
query is the same as for hash:

[code example]

Learn more

See
[Vector search]()
for more information about the indexing options, distance metrics, and query format
for vectors.

from sentence_transformers import SentenceTransformer
from redis.commands.search.query import Query
from redis.commands.search.field import TextField, TagField, VectorField
from redis.commands.search.indexDefinition import IndexDefinition, IndexType
from redis.commands.json.path import Path

import numpy as np
import redis

package main

import (
	"context"
	"encoding/binary"
	"fmt"
	"math"

	"github.com/knights-analytics/hugot"
	"github.com/redis/go-redis/v9"
)

func floatsToBytes(fs []float32) []byte {
	buf := make([]byte, len(fs)*4)

	for i, f := range fs {
		u := math.Float32bits(f)
		binary.NativeEndian.PutUint32(buf[i*4:], u)
	}

	return buf
}

func main() {
	ctx := context.Background()
	rdb := redis.NewClient(&redis.Options{
		Addr:     "localhost:6379",
		Password: "", // no password docs
		DB:       0,  // use default DB
		Protocol: 2,
	})

	rdb.FTDropIndexWithArgs(ctx,
		"vector_idx",
		&redis.FTDropIndexOptions{
			DeleteDocs: true,
		},
	)

	_, err := rdb.FTCreate(ctx,
		"vector_idx",
		&redis.FTCreateOptions{
			OnHash: true,
			Prefix: []any{"doc:"},
		},
		&redis.FieldSchema{
			FieldName: "content",
			FieldType: redis.SearchFieldTypeText,
		},
		&redis.FieldSchema{
			FieldName: "genre",
			FieldType: redis.SearchFieldTypeTag,
		},
		&redis.FieldSchema{
			FieldName: "embedding",
			FieldType: redis.SearchFieldTypeVector,
			VectorArgs: &redis.FTVectorArgs{
				HNSWOptions: &redis.FTHNSWOptions{
					Dim:            384,
					DistanceMetric: "L2",
					Type:           "FLOAT32",
				},
			},
		},
	).Result()
	if err != nil {
		panic(err)
	}

	// Create a Hugot session
	session, err := hugot.NewGoSession()
	if err != nil {
		panic(err)
	}
	defer func() {
		err := session.Destroy()
		if err != nil {
			panic(err)
		}
	}()

	// Download the model
	downloadOptions := hugot.NewDownloadOptions()
	downloadOptions.OnnxFilePath = "onnx/model.onnx" // Specify which ONNX file to use
	modelPath, err := hugot.DownloadModel("sentence-transformers/all-MiniLM-L6-v2", "./models/", downloadOptions)
	if err != nil {
		panic(err)
	}

	// Create feature extraction pipeline configuration
	config := hugot.FeatureExtractionConfig{
		ModelPath: modelPath,
		Name:      "embeddingPipeline",
	}

	// Create the feature extraction pipeline
	embeddingPipeline, err := hugot.NewPipeline(session, config)
	if err != nil {
		panic(err)
	}

	sentences := []string{
		"That is a very happy person",
		"That is a happy dog",
		"Today is a sunny day",
	}

	tags := []string{
		"persons", "pets", "weather",
	}

	// Generate embeddings using Hugot
	embeddingResult, err := embeddingPipeline.RunPipeline(sentences)
	if err != nil {
		panic(err)
	}

	// Extract the embeddings from the result
	embeddings := embeddingResult.Embeddings

	for i, emb := range embeddings {
		buffer := floatsToBytes(emb)

		_, err = rdb.HSet(ctx,
			fmt.Sprintf("doc:%v", i),
			map[string]any{
				"content":   sentences[i],
				"genre":     tags[i],
				"embedding": buffer,
			},
		).Result()

		if err != nil {
			panic(err)
		}
	}

	// Generate query embedding using Hugot
	queryResult, err := embeddingPipeline.RunPipeline([]string{
		"That is a happy person",
	})

	if err != nil {
		panic(err)
	}

	buffer := floatsToBytes(queryResult.Embeddings[0])

	results, err := rdb.FTSearchWithArgs(ctx,
		"vector_idx",
		"*=>[KNN 3 @embedding $vec AS vector_distance]",
		&redis.FTSearchOptions{
			Return: []redis.FTSearchReturn{
				{FieldName: "vector_distance"},
				{FieldName: "content"},
			},
			DialectVersion: 2,
			Params: map[string]any{
				"vec": buffer,
			},
		},
	).Result()

	if err != nil {
		panic(err)
	}

	for _, doc := range results.Docs {
		fmt.Printf(
			"ID: %v, Distance:%v, Content:'%v'\n",
			doc.ID, doc.Fields["vector_distance"], doc.Fields["content"],
		)
	}

	rdb.FTDropIndexWithArgs(ctx,
		"vector_json_idx",
		&redis.FTDropIndexOptions{
			DeleteDocs: true,
		},
	)

	_, err = rdb.FTCreate(ctx,
		"vector_json_idx",
		&redis.FTCreateOptions{
			OnJSON: true,
			Prefix: []any{"jdoc:"},
		},
		&redis.FieldSchema{
			FieldName: "$.content",
			As:        "content",
			FieldType: redis.SearchFieldTypeText,
		},
		&redis.FieldSchema{
			FieldName: "$.genre",
			As:        "genre",
			FieldType: redis.SearchFieldTypeTag,
		},
		&redis.FieldSchema{
			FieldName: "$.embedding",
			As:        "embedding",
			FieldType: redis.SearchFieldTypeVector,
			VectorArgs: &redis.FTVectorArgs{
				HNSWOptions: &redis.FTHNSWOptions{
					Dim:            384,
					DistanceMetric: "L2",
					Type:           "FLOAT32",
				},
			},
		},
	).Result()
	if err != nil {
		panic(err)
	}

	for i, emb := range embeddings {
		_, err = rdb.JSONSet(ctx,
			fmt.Sprintf("jdoc:%v", i),
			"$",
			map[string]any{
				"content":   sentences[i],
				"genre":     tags[i],
				"embedding": emb,
			},
		).Result()

		if err != nil {
			panic(err)
		}
	}

	// Generate query embedding for JSON search using Hugot
	jsonQueryResult, err := embeddingPipeline.RunPipeline([]string{
		"That is a happy person",
	})

	if err != nil {
		panic(err)
	}

	jsonBuffer := floatsToBytes(jsonQueryResult.Embeddings[0])

	jsonResults, err := rdb.FTSearchWithArgs(ctx,
		"vector_json_idx",
		"*=>[KNN 3 @embedding $vec AS vector_distance]",
		&redis.FTSearchOptions{
			Return: []redis.FTSearchReturn{
				{FieldName: "vector_distance"},
				{FieldName: "content"},
			},
			DialectVersion: 2,
			Params: map[string]any{
				"vec": jsonBuffer,
			},
		},
	).Result()

	if err != nil {
		panic(err)
	}

	for _, doc := range jsonResults.Docs {
		fmt.Printf(
			"ID: %v, Distance:%v, Content:'%v'\n",
			doc.ID, doc.Fields["vector_distance"], doc.Fields["content"],
		)
	}
}

import * as transformers from '@xenova/transformers';
import {
    VectorAlgorithms,
    createClient,
    SchemaFieldTypes,
} from 'redis';

let pipe = await transformers.pipeline(
    'feature-extraction', 'Xenova/all-distilroberta-v1'
);

const pipeOptions = {
    pooling: 'mean',
    normalize: true,
};

const client = createClient({url: 'redis://localhost:6379'});
await client.connect();

try {
    await client.ft.dropIndex('vector_idx');
} catch (e) {
    // Index doesn't exist, which is fine
}

await client.ft.create('vector_idx', {
    'content': {
        type: SchemaFieldTypes.TEXT,
    },
    'genre': {
        type: SchemaFieldTypes.TAG,
    },
    'embedding': {
        type: SchemaFieldTypes.VECTOR,
        TYPE: 'FLOAT32',
        ALGORITHM: VectorAlgorithms.HNSW,
        DISTANCE_METRIC: 'L2',
        DIM: 768,
    }
}, {
    ON: 'HASH',
    PREFIX: 'doc:'
});

const sentence1 = 'That is a very happy person';
const doc1 = {
    'content': sentence1,
    'genre': 'persons',
    'embedding': Buffer.from(
        (await pipe(sentence1, pipeOptions)).data.buffer
    ),
};

const sentence2 = 'That is a happy dog';
const doc2 = {
    'content': sentence2,
    'genre': 'pets',
    'embedding': Buffer.from(
        (await pipe(sentence2, pipeOptions)).data.buffer
    )
};

const sentence3 = 'Today is a sunny day';
const doc3 = {
    'content': sentence3,
    'genre': 'weather',
    'embedding': Buffer.from(
        (await pipe(sentence3, pipeOptions)).data.buffer
    )
};

await Promise.all([
    client.hSet('doc:1', doc1),
    client.hSet('doc:2', doc2),
    client.hSet('doc:3', doc3)
]);

const similar = await client.ft.search(
    'vector_idx',
    '*=>[KNN 3 @embedding $B AS score]',
    {
        'PARAMS': {
            B: Buffer.from(
                (await pipe('That is a happy person', pipeOptions)).data.buffer
            ),
        },
        'RETURN': ['score', 'content'],
        'DIALECT': '2'
    },
);

for (const doc of similar.documents) {
    console.log(`${doc.id}: '${doc.value.content}', Score: ${doc.value.score}`);
}

try {
    await client.ft.dropIndex('vector_json_idx');
} catch (e) {
    // Index doesn't exist, which is fine
}

await client.ft.create('vector_json_idx', {
    '$.content': {
        type: SchemaFieldTypes.TEXT,
        AS: 'content',
    },
    '$.genre': {
        type: SchemaFieldTypes.TAG,
        AS: 'genre',
    },
    '$.embedding': {
        type: SchemaFieldTypes.VECTOR,
        TYPE: 'FLOAT32',
        ALGORITHM: VectorAlgorithms.HNSW,
        DISTANCE_METRIC: 'L2',
        DIM: 768,
        AS: 'embedding',
    }
}, {
    ON: 'JSON',
    PREFIX: 'jdoc:'
});

const jSentence1 = 'That is a very happy person';
const jdoc1 = {
    'content': jSentence1,
    'genre': 'persons',
    'embedding': [...(await pipe(jSentence1, pipeOptions)).data],
};

const jSentence2 = 'That is a happy dog';
const jdoc2 = {
    'content': jSentence2,
    'genre': 'pets',
    'embedding': [...(await pipe(jSentence2, pipeOptions)).data],
};

const jSentence3 = 'Today is a sunny day';
const jdoc3 = {
    'content': jSentence3,
    'genre': 'weather',
    'embedding': [...(await pipe(jSentence3, pipeOptions)).data],
};

await Promise.all([
    client.json.set('jdoc:1', '$', jdoc1),
    client.json.set('jdoc:2', '$', jdoc2),
    client.json.set('jdoc:3', '$', jdoc3)
]);

const jsons = await client.ft.search(
    'vector_json_idx',
    '*=>[KNN 3 @embedding $B AS score]',
    {
        "PARAMS": {
            B: Buffer.from(
                (await pipe('That is a happy person', pipeOptions)).data.buffer
            ),
        },
        'RETURN': ['score', 'content'],
        'DIALECT': '2'
    },
);

await client.quit();

r = redis.Redis(decode_responses=True)

try:
    r.ft("vector_idx").dropindex(True)
except redis.exceptions.ResponseError:
    pass

schema = (
    TextField("content"),
    TagField("genre"),
    VectorField("embedding", "HNSW", {
        "TYPE": "FLOAT32",
        "DIM": 384,
        "DISTANCE_METRIC":"L2"
    })
)

r.ft("vector_idx").create_index(
    schema,
    definition=IndexDefinition(
        prefix=["doc:"], index_type=IndexType.HASH
    )
)

content = "That is a very happy person"

r.hset("doc:0", mapping={
    "content": content,
    "genre": "persons",
    "embedding": model.encode(content).astype(np.float32).tobytes(),
})

content = "That is a happy dog"

r.hset("doc:1", mapping={
    "content": content,
    "genre": "pets",
    "embedding": model.encode(content).astype(np.float32).tobytes(),
})

content = "Today is a sunny day"

r.hset("doc:2", mapping={
    "content": content,
    "genre": "weather",
    "embedding": model.encode(content).astype(np.float32).tobytes(),
})

q = Query(
    "*=>[KNN 3 @embedding $vec AS vector_distance]"
).return_field("vector_distance").return_field("content").dialect(2)

query_text = "That is a happy person"

res = r.ft("vector_idx").search(
    q, query_params={
        "vec": model.encode(query_text).astype(np.float32).tobytes()
    }
)

print(res)

Result{
    3 total,
    docs: [
        Document {
            'id': 'doc:0',
            'payload': None,
            'vector_distance': '0.114169985056',
            'content': 'That is a very happy person'
        },
        Document {
            'id': 'doc:1',
            'payload': None,
            'vector_distance': '0.610845386982',
            'content': 'That is a happy dog'
        },
        Document {
            'id': 'doc:2',
            'payload': None,
            'vector_distance': '1.48624813557',
            'content': 'Today is a sunny day'
        }
    ]
}

package io.redis.examples.async;

// Lettuce client and Redis Search classes.
import io.lettuce.core.*;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.api.async.RedisAsyncCommands;
import io.lettuce.core.search.arguments.*;
import io.lettuce.core.search.SearchReply;
import io.lettuce.core.json.JsonParser;
import io.lettuce.core.json.JsonObject;
import io.lettuce.core.json.JsonPath;

// Standard library classes for data manipulation and
// asynchronous programming.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.StandardCharsets;
import java.util.*;
import java.util.concurrent.CompletableFuture;

// DJL classes for model loading and inference.
import ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory;
import ai.djl.inference.Predictor;
import ai.djl.repository.zoo.Criteria;
import ai.djl.training.util.ProgressBar;

public class HomeQueryVecExample {

    private ByteBuffer floatArrayToByteBuffer(float[] vector) {
        ByteBuffer buffer = ByteBuffer.allocate(vector.length * 4).order(ByteOrder.LITTLE_ENDIAN);
        for (float value : vector) {
            buffer.putFloat(value);
        }
        return (ByteBuffer) buffer.flip();
    }

    public void run() {
        Predictor<String, float[]> predictor = null;

        try {
            Criteria<String, float[]> criteria = Criteria.builder().setTypes(String.class, float[].class)
                    .optModelUrls("djl://ai.djl.huggingface.pytorch/sentence-transformers/all-MiniLM-L6-v2")
                    .optEngine("PyTorch").optTranslatorFactory(new TextEmbeddingTranslatorFactory())
                    .optProgress(new ProgressBar()).build();

            predictor = criteria.loadModel().newPredictor();
        } catch (Exception e) {
            // ...
        }

        RedisClient redisClient = RedisClient.create("redis://localhost:6379");

        try (StatefulRedisConnection<String, String> connection = redisClient.connect();
                StatefulRedisConnection<ByteBuffer, ByteBuffer> binConnection = redisClient.connect(new ByteBufferCodec())) {
            RedisAsyncCommands<String, String> asyncCommands = connection.async();
            RedisAsyncCommands<ByteBuffer, ByteBuffer> binAsyncCommands = binConnection.async();
            // ...

            List<FieldArgs<String>> schema = Arrays.asList(TextFieldArgs.<String> builder().name("content").build(),
                    TagFieldArgs.<String> builder().name("genre").build(),
                    VectorFieldArgs.<String> builder().name("embedding").hnsw().type(VectorFieldArgs.VectorType.FLOAT32)
                            .dimensions(384).distanceMetric(VectorFieldArgs.DistanceMetric.L2).build());

            CreateArgs<String, String> createArgs = CreateArgs.<String, String> builder().on(CreateArgs.TargetType.HASH)
                    .withPrefix("doc:").build();

            CompletableFuture<Void> createIndex = asyncCommands.ftCreate("vector_idx", createArgs, schema)
                    .thenAccept(System.out::println).toCompletableFuture();
            createIndex.join();

            String sentence1 = "That is a very happy person";

            Map<ByteBuffer, ByteBuffer> doc1 = new HashMap<>();
            doc1.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence1.getBytes()));
            doc1.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("persons".getBytes()));

            try {
                doc1.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence1)));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<Long> addDoc1 = binAsyncCommands.hset(ByteBuffer.wrap("doc:1".getBytes()), doc1)
                    .thenApply(result -> {
                        System.out.println(result); // >>> 3
                        return result;
                    }).toCompletableFuture();

            String sentence2 = "That is a happy dog";

            Map<ByteBuffer, ByteBuffer> doc2 = new HashMap<>();
            doc2.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence2.getBytes()));
            doc2.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("pets".getBytes()));

            try {
                doc2.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence2)));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<Long> addDoc2 = binAsyncCommands.hset(ByteBuffer.wrap("doc:2".getBytes()), doc2)
                    .thenApply(result -> {
                        System.out.println(result); // >>> 3
                        return result;
                    }).toCompletableFuture();

            String sentence3 = "Today is a sunny day";

            Map<ByteBuffer, ByteBuffer> doc3 = new HashMap<>();
            doc3.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence3.getBytes()));
            doc3.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("weather".getBytes()));

            try {
                doc3.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence3)));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<Long> addDoc3 = binAsyncCommands.hset(ByteBuffer.wrap("doc:3".getBytes()), doc3)
                    .thenApply(result -> {
                        System.out.println(result); // >>> 3
                        return result;
                    }).toCompletableFuture();
            CompletableFuture.allOf(addDoc1, addDoc2, addDoc3).join();

            String query = "That is a happy person";
            float[] queryEmbedding = null;

            try {
                queryEmbedding = predictor.predict(query);
            } catch (Exception e) {
                // ...
            }

            SearchArgs<ByteBuffer, ByteBuffer> searchArgs = SearchArgs.<ByteBuffer, ByteBuffer> builder()
                    .param(ByteBuffer.wrap("vec".getBytes()), floatArrayToByteBuffer(queryEmbedding))
                    .returnField(ByteBuffer.wrap("content".getBytes()))
                    .returnField(ByteBuffer.wrap("vector_distance".getBytes()))
                    .sortBy(SortByArgs.<ByteBuffer> builder().attribute(ByteBuffer.wrap("vector_distance".getBytes())).build())
                    .build();

            CompletableFuture<SearchReply<ByteBuffer, ByteBuffer>> hashQuery = binAsyncCommands
                    .ftSearch(ByteBuffer.wrap("vector_idx".getBytes()),
                            ByteBuffer.wrap("*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes()), searchArgs)
                    .thenApply(result -> {
                        List<SearchReply.SearchResult<ByteBuffer, ByteBuffer>> results = result.getResults();

                        results.forEach(r -> {
                            String id = StandardCharsets.UTF_8.decode(r.getId()).toString();
                            String content = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("content".getBytes()))).toString();
                            String distance = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("vector_distance".getBytes()))).toString();

                            System.out.println("ID: " + id + ", Content: " + content + ", Distance: " + distance);
                        });
                        // >>> ID: doc:1, Content: That is a very happy person, Distance: 0.114169836044
                        // >>> ID: doc:2, Content: That is a happy dog, Distance: 0.610845506191
                        // >>> ID: doc:3, Content: Today is a sunny day, Distance: 1.48624765873

                        return result;
                    }).toCompletableFuture();
            hashQuery.join();

            List<FieldArgs<String>> jsonSchema = Arrays.asList(
                    TextFieldArgs.<String> builder().name("$.content").as("content").build(),
                    TagFieldArgs.<String> builder().name("$.genre").as("genre").build(),
                    VectorFieldArgs.<String> builder().name("$.embedding").as("embedding").hnsw()
                            .type(VectorFieldArgs.VectorType.FLOAT32).dimensions(384)
                            .distanceMetric(VectorFieldArgs.DistanceMetric.L2).build());

            CreateArgs<String, String> jsonCreateArgs = CreateArgs.<String, String> builder().on(CreateArgs.TargetType.JSON)
                    .withPrefix("jdoc:").build();

            CompletableFuture<Void> jsonCreateIndex = asyncCommands.ftCreate("vector_json_idx", jsonCreateArgs, jsonSchema)
                    .thenAccept(System.out::println).toCompletableFuture();
            jsonCreateIndex.join();

            JsonParser parser = asyncCommands.getJsonParser();

            String jSentence1 = "\"That is a very happy person\"";

            JsonObject jDoc1 = parser.createJsonObject();
            jDoc1.put("content", parser.createJsonValue(jSentence1));
            jDoc1.put("genre", parser.createJsonValue("\"persons\""));

            try {
                jDoc1.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence1))));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<String> jsonAddDoc1 = asyncCommands.jsonSet("jdoc:1", JsonPath.ROOT_PATH, jDoc1)
                    .thenApply(result -> {
                        System.out.println(result); // >>> OK
                        return result;
                    }).toCompletableFuture();

            String jSentence2 = "\"That is a happy dog\"";

            JsonObject jDoc2 = parser.createJsonObject();
            jDoc2.put("content", parser.createJsonValue(jSentence2));
            jDoc2.put("genre", parser.createJsonValue("\"pets\""));

            try {
                jDoc2.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence2))));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<String> jsonAddDoc2 = asyncCommands.jsonSet("jdoc:2", JsonPath.ROOT_PATH, jDoc2)
                    .thenApply(result -> {
                        System.out.println(result); // >>> OK
                        return result;
                    }).toCompletableFuture();

            String jSentence3 = "\"Today is a sunny day\"";

            JsonObject jDoc3 = parser.createJsonObject();
            jDoc3.put("content", parser.createJsonValue(jSentence3));
            jDoc3.put("genre", parser.createJsonValue("\"weather\""));

            try {
                jDoc3.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence3))));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<String> jsonAddDoc3 = asyncCommands.jsonSet("jdoc:3", JsonPath.ROOT_PATH, jDoc3)
                    .thenApply(result -> {
                        System.out.println(result); // >>> OK
                        return result;
                    }).toCompletableFuture();
            CompletableFuture.allOf(jsonAddDoc1, jsonAddDoc2, jsonAddDoc3).join();

            String jQuery = "That is a happy person";
            float[] jsonQueryEmbedding = null;

            try {
                jsonQueryEmbedding = predictor.predict(jQuery);
            } catch (Exception e) {
                // ...
            }

            SearchArgs<ByteBuffer, ByteBuffer> jsonSearchArgs = SearchArgs.<ByteBuffer, ByteBuffer> builder()
                    .param(ByteBuffer.wrap("vec".getBytes()), floatArrayToByteBuffer(jsonQueryEmbedding))
                    .returnField(ByteBuffer.wrap("content".getBytes()))
                    .returnField(ByteBuffer.wrap("vector_distance".getBytes()))
                    .sortBy(SortByArgs.<ByteBuffer> builder().attribute(ByteBuffer.wrap("vector_distance".getBytes())).build())
                    .build();

            CompletableFuture<SearchReply<ByteBuffer, ByteBuffer>> jsonQuery = binAsyncCommands
                    .ftSearch(ByteBuffer.wrap("vector_json_idx".getBytes()),
                            ByteBuffer.wrap("*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes()), jsonSearchArgs)
                    .thenApply(result -> {
                        List<SearchReply.SearchResult<ByteBuffer, ByteBuffer>> results = result.getResults();

                        results.forEach(r -> {
                            String id = StandardCharsets.UTF_8.decode(r.getId()).toString();
                            String content = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("content".getBytes()))).toString();
                            String distance = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("vector_distance".getBytes()))).toString();

                            System.out.println("ID: " + id + ", Content: " + content + ", Distance: " + distance);
                        });
                        // >>> ID: jdoc:1, Content: "That is a very happy person", Distance:0.628328084946
                        // >>> ID: jdoc:2, Content: That is a happy dog, Distance: 0.895147025585
                        // >>> ID: jdoc:3, Content: "Today is a sunny day", Distance: 1.49569523335

                        return result;
                    }).toCompletableFuture();
            jsonQuery.join();
        } finally {
            redisClient.shutdown();
        }
    }

}

package io.redis.examples.reactive;

// Lettuce client and Redis Search classes.
import io.lettuce.core.*;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.api.reactive.RedisReactiveCommands;
import io.lettuce.core.search.arguments.*;
import io.lettuce.core.search.SearchReply;
import io.lettuce.core.json.JsonParser;
import io.lettuce.core.json.JsonObject;
import io.lettuce.core.json.JsonPath;

// Standard library classes for data manipulation and
// reactive programming.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.StandardCharsets;
import java.util.*;
import reactor.core.publisher.Mono;

// DJL classes for model loading and inference.
import ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory;
import ai.djl.inference.Predictor;
import ai.djl.repository.zoo.Criteria;
import ai.djl.training.util.ProgressBar;

public class HomeQueryVecExample {

    private ByteBuffer floatArrayToByteBuffer(float[] vector) {
        ByteBuffer buffer = ByteBuffer.allocate(vector.length * 4).order(ByteOrder.LITTLE_ENDIAN);
        for (float value : vector) {
            buffer.putFloat(value);
        }
        return (ByteBuffer) buffer.flip();
    }

    public void run() {
        Predictor<String, float[]> predictor = null;

        try {
            Criteria<String, float[]> criteria = Criteria.builder().setTypes(String.class, float[].class)
                    .optModelUrls("djl://ai.djl.huggingface.pytorch/sentence-transformers/all-MiniLM-L6-v2")
                    .optEngine("PyTorch").optTranslatorFactory(new TextEmbeddingTranslatorFactory())
                    .optProgress(new ProgressBar()).build();

            predictor = criteria.loadModel().newPredictor();
        } catch (Exception e) {
            // ...
        }

        RedisClient redisClient = RedisClient.create("redis://localhost:6379");

        try (StatefulRedisConnection<String, String> connection = redisClient.connect();
                StatefulRedisConnection<ByteBuffer, ByteBuffer> binConnection = redisClient.connect(new ByteBufferCodec())) {
            RedisReactiveCommands<String, String> reactiveCommands = connection.reactive();
            RedisReactiveCommands<ByteBuffer, ByteBuffer> binReactiveCommands = binConnection.reactive();
            // ...

            List<FieldArgs<String>> schema = Arrays.asList(TextFieldArgs.<String> builder().name("content").build(),
                    TagFieldArgs.<String> builder().name("genre").build(),
                    VectorFieldArgs.<String> builder().name("embedding").hnsw().type(VectorFieldArgs.VectorType.FLOAT32)
                            .dimensions(384).distanceMetric(VectorFieldArgs.DistanceMetric.L2).build());

            CreateArgs<String, String> createArgs = CreateArgs.<String, String> builder().on(CreateArgs.TargetType.HASH)
                    .withPrefix("doc:").build();

            Mono<Void> createIndex = reactiveCommands.ftCreate("vector_idx", createArgs, schema).doOnNext(result -> {
                System.out.println(result); // >>> OK
            }).then();
            createIndex.block();

            String sentence1 = "That is a very happy person";

            Map<ByteBuffer, ByteBuffer> doc1 = new HashMap<>();
            doc1.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence1.getBytes()));
            doc1.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("persons".getBytes()));

            try {
                doc1.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence1)));
            } catch (Exception e) {
                // ...
            }

            Mono<Long> addDoc1 = binReactiveCommands.hset(ByteBuffer.wrap("doc:1".getBytes()), doc1).doOnNext(result -> {
                System.out.println(result); // >>> 3
            });

            String sentence2 = "That is a happy dog";

            Map<ByteBuffer, ByteBuffer> doc2 = new HashMap<>();
            doc2.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence2.getBytes()));
            doc2.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("pets".getBytes()));

            try {
                doc2.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence2)));
            } catch (Exception e) {
                // ...
            }

            Mono<Long> addDoc2 = binReactiveCommands.hset(ByteBuffer.wrap("doc:2".getBytes()), doc2).doOnNext(result -> {
                System.out.println(result); // >>> 3
            });

            String sentence3 = "Today is a sunny day";

            Map<ByteBuffer, ByteBuffer> doc3 = new HashMap<>();
            doc3.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence3.getBytes()));
            doc3.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("weather".getBytes()));

            try {
                doc3.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence3)));
            } catch (Exception e) {
                // ...
            }

            Mono<Long> addDoc3 = binReactiveCommands.hset(ByteBuffer.wrap("doc:3".getBytes()), doc3).doOnNext(result -> {
                System.out.println(result); // >>> 3
            });
            Mono.when(addDoc1, addDoc2, addDoc3).block();

            String query = "That is a happy person";
            float[] queryEmbedding = null;

            try {
                queryEmbedding = predictor.predict(query);
            } catch (Exception e) {
                // ...
            }

            SearchArgs<ByteBuffer, ByteBuffer> searchArgs = SearchArgs.<ByteBuffer, ByteBuffer> builder()
                    .param(ByteBuffer.wrap("vec".getBytes()), floatArrayToByteBuffer(queryEmbedding))
                    .returnField(ByteBuffer.wrap("content".getBytes()))
                    .returnField(ByteBuffer.wrap("vector_distance".getBytes()))
                    .sortBy(SortByArgs.<ByteBuffer> builder().attribute(ByteBuffer.wrap("vector_distance".getBytes())).build())
                    .build();

            Mono<SearchReply<ByteBuffer, ByteBuffer>> hashQuery = binReactiveCommands
                    .ftSearch(ByteBuffer.wrap("vector_idx".getBytes()),
                            ByteBuffer.wrap("*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes()), searchArgs)
                    .doOnNext(result -> {
                        List<SearchReply.SearchResult<ByteBuffer, ByteBuffer>> results = result.getResults();

                        results.forEach(r -> {
                            String id = StandardCharsets.UTF_8.decode(r.getId()).toString();
                            String content = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("content".getBytes()))).toString();
                            String distance = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("vector_distance".getBytes()))).toString();

                            System.out.println("ID: " + id + ", Content: " + content + ", Distance: " + distance);
                        });
                        // >>> ID: doc:1, Content: That is a very happy person, Distance: 0.114169836044
                        // >>> ID: doc:2, Content: That is a happy dog, Distance: 0.610845506191
                        // >>> ID: doc:3, Content: Today is a sunny day, Distance: 1.48624765873
                    });
            hashQuery.block();

            List<FieldArgs<String>> jsonSchema = Arrays.asList(
                    TextFieldArgs.<String> builder().name("$.content").as("content").build(),
                    TagFieldArgs.<String> builder().name("$.genre").as("genre").build(),
                    VectorFieldArgs.<String> builder().name("$.embedding").as("embedding").hnsw()
                            .type(VectorFieldArgs.VectorType.FLOAT32).dimensions(384)
                            .distanceMetric(VectorFieldArgs.DistanceMetric.L2).build());

            CreateArgs<String, String> jsonCreateArgs = CreateArgs.<String, String> builder().on(CreateArgs.TargetType.JSON)
                    .withPrefix("jdoc:").build();

            Mono<Void> jsonCreateIndex = reactiveCommands.ftCreate("vector_json_idx", jsonCreateArgs, jsonSchema)
                    .doOnNext(result -> {
                        System.out.println(result); // >>> OK
                    }).then();
            jsonCreateIndex.block();

            JsonParser parser = reactiveCommands.getJsonParser();

            String jSentence1 = "\"That is a very happy person\"";
            JsonObject jDoc1 = parser.createJsonObject();
            jDoc1.put("content", parser.createJsonValue(jSentence1));
            jDoc1.put("genre", parser.createJsonValue("\"persons\""));

            try {
                jDoc1.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence1))));
            } catch (Exception e) {
                // ...
            }

            Mono<Void> jsonAddDoc1 = reactiveCommands.jsonSet("jdoc:1", JsonPath.ROOT_PATH, jDoc1).doOnNext(result -> {
                System.out.println(result); // >>> OK
            }).then();

            String jSentence2 = "\"That is a happy dog\"";
            JsonObject jDoc2 = parser.createJsonObject();
            jDoc2.put("content", parser.createJsonValue(jSentence2));
            jDoc2.put("genre", parser.createJsonValue("\"pets\""));

            try {
                jDoc2.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence2))));
            } catch (Exception e) {
                // ...
            }

            Mono<Void> jsonAddDoc2 = reactiveCommands.jsonSet("jdoc:2", JsonPath.ROOT_PATH, jDoc2).doOnNext(result -> {
                System.out.println(result); // >>> OK
            }).then();

            String jSentence3 = "\"Today is a sunny day\"";
            JsonObject jDoc3 = parser.createJsonObject();
            jDoc3.put("content", parser.createJsonValue(jSentence3));
            jDoc3.put("genre", parser.createJsonValue("\"weather\""));

            try {
                jDoc3.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence3))));
            } catch (Exception e) {
                // ...
            }

            Mono<Void> jsonAddDoc3 = reactiveCommands.jsonSet("jdoc:3", JsonPath.ROOT_PATH, jDoc3).doOnNext(result -> {
                System.out.println(result); // >>> OK
            }).then();
            Mono.when(jsonAddDoc1, jsonAddDoc2, jsonAddDoc3).block();

            String jQuery = "That is a happy person";
            float[] jsonQueryEmbedding = null;

            try {
                jsonQueryEmbedding = predictor.predict(jQuery);
            } catch (Exception e) {
                // ...
            }

            SearchArgs<ByteBuffer, ByteBuffer> jsonSearchArgs = SearchArgs.<ByteBuffer, ByteBuffer> builder()
                    .param(ByteBuffer.wrap("vec".getBytes()), floatArrayToByteBuffer(jsonQueryEmbedding))
                    .returnField(ByteBuffer.wrap("content".getBytes()))
                    .returnField(ByteBuffer.wrap("vector_distance".getBytes()))
                    .sortBy(SortByArgs.<ByteBuffer> builder().attribute(ByteBuffer.wrap("vector_distance".getBytes())).build())
                    .build();

            Mono<SearchReply<ByteBuffer, ByteBuffer>> jsonQuery = binReactiveCommands
                    .ftSearch(ByteBuffer.wrap("vector_json_idx".getBytes()),
                            ByteBuffer.wrap("*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes()), jsonSearchArgs)
                    .doOnNext(result -> {
                        List<SearchReply.SearchResult<ByteBuffer, ByteBuffer>> results = result.getResults();

                        results.forEach(r -> {
                            String id = StandardCharsets.UTF_8.decode(r.getId()).toString();
                            String content = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("content".getBytes()))).toString();
                            String distance = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("vector_distance".getBytes()))).toString();

                            System.out.println("ID: " + id + ", Content: " + content + ", Distance: " + distance);
                        });
                        // >>> ID: jdoc:1, Content: "That is a very happy person", Distance: 0.628328084946
                        // >>> ID: jdoc:2, Content: "That is a happy dog", Distance: 0.895147025585
                        // >>> ID: jdoc:3, Content: "Today is a sunny day", Distance: 1.49569523335
                    });
            jsonQuery.block();
        } finally {
            redisClient.shutdown();
        }
    }

}

schema = (
    TextField("$.content", as_name="content"),
    TagField("$.genre", as_name="genre"),
    VectorField(
        "$.embedding", "HNSW", {
            "TYPE": "FLOAT32",
            "DIM": 384,
            "DISTANCE_METRIC": "L2"
        },
        as_name="embedding"
    )
)

r.ft("vector_json_idx").create_index(
    schema,
    definition=IndexDefinition(
        prefix=["jdoc:"], index_type=IndexType.JSON
    )
)

content = "That is a very happy person"

r.json().set("jdoc:0", Path.root_path(), {
    "content": content,
    "genre": "persons",
    "embedding": model.encode(content).astype(np.float32).tolist(),
})

content = "That is a happy dog"

r.json().set("jdoc:1", Path.root_path(), {
    "content": content,
    "genre": "pets",
    "embedding": model.encode(content).astype(np.float32).tolist(),
})

content = "Today is a sunny day"

r.json().set("jdoc:2", Path.root_path(), {
    "content": content,
    "genre": "weather",
    "embedding": model.encode(content).astype(np.float32).tolist(),
})

q = Query(
    "*=>[KNN 3 @embedding $vec AS vector_distance]"
).return_field("vector_distance").return_field("content").dialect(2)

query_text = "That is a happy person"

res = r.ft("vector_json_idx").search(
    q, query_params={
        "vec": model.encode(query_text).astype(np.float32).tobytes()
    }
)

print(repr(res))

Result{
    3 total,
    docs: [
        Document {
            'id': 'jdoc:0',
            'payload': None,
            'vector_distance': '0.114169985056',
            'content': 'That is a very happy person'
        },
            .
            .
            .


Vector set embeddings

Summary: Index and query embeddings with Redis vector sets

Initialize

Start by installing the preview version of `redis-py` with the following
command:

[code example]

Also, install `sentence-transformers`:

[code example]

In a new Python file, import the required classes:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

The first of these imports is the
`SentenceTransformer` class, which generates an embedding from a section of text.
This example uses an instance of `SentenceTransformer` with the
[`all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
model for the embeddings. This model generates vectors with 384 dimensions, regardless
of the length of the input text, but note that the input is truncated to 256
tokens (see
[Word piece tokenization](https://huggingface.co/learn/nlp-course/en/chapter6/6)
at the [Hugging Face](https://huggingface.co/) docs to learn more about the way tokens
are related to the original text).

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Create the data

The example data is contained a dictionary with some brief
descriptions of famous people:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Add the data to a vector set

The next step is to connect to Redis and add the data to a new vector set.

The code below uses the dictionary's
[`items()`](https://docs.python.org/3/library/stdtypes.html#dict.items)
view to iterate through all the key-value pairs and add corresponding
elements to a vector set called `famousPeople`.

Use the
[`encode()`](https://sbert.net/docs/package_reference/sentence_transformer/SentenceTransformer.html#sentence_transformers.SentenceTransformer.encode)
method of `SentenceTransformer` to generate the
embedding as an array of `float32` values. The `tobytes()` method converts
the array to a byte string that you can pass to the
[`vadd()`]() command to set the embedding.
Note that `vadd()` can also accept a list of `float` values to set the
vector, but the byte string format is more compact and saves a little
transmission time. If you later use
[`vemb()`]() to retrieve the embedding,
it will return the vector as an array rather than the original byte
string (note that this is different from the behavior of byte strings in
[hash vector indexing]()).

The call to `vadd()` also adds the `born` and `died` values from the
original dictionary as attribute data. You can access this during a query
or by using the [`vgetattr()`]() method.

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Query the vector set

You can now query the data in the set. The basic approach is to use the
`encode()` method to generate another embedding vector for the query text.
(This is the same method used to add the elements to the set.) Then, pass
the query vector to [`vsim()`]() to return elements
of the set, ranked in order of similarity to the query.

Start with a simple query for "actors":

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

This returns the following list of elements (formatted slightly for clarity):

[code example]

The first two people in the list are the two actors, as expected, but none of the
people from Linus Pauling onward was especially well-known for acting (and there certainly
isn't any information about that in the short description text).
As it stands, the search attempts to rank all the elements in the set, based
on the information contained in the embedding model.
You can use the `count` parameter of `vsim()` to limit the list of elements
to just the most relevant few items:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

The reason for using text embeddings rather than simple text search
is that the embeddings represent semantic information. This allows a query
to find elements with a similar meaning even if the text is
different. For example, the word "entertainer" doesn't appear in any of the
descriptions but if you use it as a query, the actors and musicians are ranked
highest in the results list:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Similarly, if you use "science" as a query, you get the following results:

[code example]

The scientists are ranked highest but they are then followed by the
mathematicians. This seems reasonable given the connection between mathematics
and science.

You can also use
[filter expressions]()
with `vsim()` to restrict the search further. For example,
repeat the "science" query, but this time limit the results to people
who died before the year 2000:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Note that the boolean filter expression is applied to items in the list
before the vector distance calculation is performed. Items that don't
pass the filter test are removed from the results completely, rather
than just reduced in rank. This can help to improve the performance of the
search because there is no need to calculate the vector distance for
elements that have already been filtered out of the search.

More information

See the [vector sets]()
docs for more information and code examples. See the
[Redis for AI]() section for more details
about text embeddings and other AI techniques you can use with Redis.

You may also be interested in
[vector search]().
This is a feature of
[Redis Search]()
that lets you retrieve
[JSON]() and
[hash]() documents based on
vector data stored in their fields.

// Suppress experimental API warnings for VectorSet
#pragma warning disable SER001

using StackExchange.Redis;

using Microsoft.ML;
using Microsoft.ML.Transforms.Text;

package main

import (
	"context"
	"fmt"
	"strings"

	"github.com/knights-analytics/hugot"
	"github.com/redis/go-redis/v9"
)

// Lettuce client and Redis Search classes.
import io.lettuce.core.*;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.api.async.RedisAsyncCommands;

// Standard library classes for data manipulation and
// asynchronous programming.
import java.util.*;
import java.util.concurrent.CompletableFuture;

// DJL classes for model loading and inference.
import ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory;
import ai.djl.inference.Predictor;
import ai.djl.repository.zoo.Criteria;
import ai.djl.training.util.ProgressBar;

// Lettuce client and Redis Search classes.
import io.lettuce.core.*;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.api.reactive.RedisReactiveCommands;

// Standard library classes for data manipulation and
// reactive programming.
import java.util.*;
import reactor.core.publisher.Mono;

// DJL classes for model loading and inference.
import ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory;
import ai.djl.inference.Predictor;
import ai.djl.repository.zoo.Criteria;
import ai.djl.training.util.ProgressBar;

import redis.clients.jedis.RedisClient;
import redis.clients.jedis.params.VAddParams;
import redis.clients.jedis.params.VSimParams;

import java.util.*;

// DJL classes for model loading and inference.
import ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory;
import ai.djl.inference.Predictor;
import ai.djl.repository.zoo.Criteria;
import ai.djl.training.util.ProgressBar;

import * as transformers from '@xenova/transformers';
import { createClient } from 'redis';

from sentence_transformers import SentenceTransformer

import redis
import numpy as np

static PredictionEngine<TextData, TransformedTextData> GetPredictionEngine()
{
    // Create a new ML context, for ML.NET operations. It can be used for
    // exception tracking and logging, as well as the source of randomness.
    var mlContext = new MLContext();

    // Create an empty list as the dataset
    var emptySamples = new List<TextData>();

    // Convert sample list to an empty IDataView.
    var emptyDataView = mlContext.Data.LoadFromEnumerable(emptySamples);

    // A pipeline for converting text into a 150-dimension embedding vector
    var textPipeline = mlContext.Transforms.Text.NormalizeText("Text")
        .Append(mlContext.Transforms.Text.TokenizeIntoWords("Tokens",
            "Text"))
        .Append(mlContext.Transforms.Text.ApplyWordEmbedding("Features",
            "Tokens", WordEmbeddingEstimator.PretrainedModelKind
            .SentimentSpecificWordEmbedding));

    // Fit to data.
    var textTransformer = textPipeline.Fit(emptyDataView);

    // Create the prediction engine to get the embedding vector from the input text/string.
    var predictionEngine = mlContext.Model.CreatePredictionEngine<TextData,
        TransformedTextData>(textTransformer);

    return predictionEngine;
}

// Create a new Hugot session
	session, err := hugot.NewGoSession()
	if err != nil {
		panic(err)
	}
	defer func() {
		err := session.Destroy()
		if err != nil {
			panic(err)
		}
	}()

	// Download the model.
	downloadOptions := hugot.NewDownloadOptions()
	downloadOptions.OnnxFilePath = "onnx/model.onnx"
	modelPath, err := hugot.DownloadModel(
		"sentence-transformers/all-MiniLM-L6-v2",
		"./models/",
		downloadOptions,
	)
	if err != nil {
		panic(err)
	}

	// Create feature extraction pipeline configuration
	config := hugot.FeatureExtractionConfig{
		ModelPath: modelPath,
		Name:      "embeddingPipeline",
	}

	// Create the feature extraction pipeline
	embeddingPipeline, err := hugot.NewPipeline(session, config)
	if err != nil {
		panic(err)
	}

Predictor<String, float[]> predictor = null;

        try {
            Criteria<String, float[]> criteria = Criteria.builder().setTypes(String.class, float[].class)
                    .optModelUrls("djl://ai.djl.huggingface.pytorch/sentence-transformers/all-MiniLM-L6-v2")
                    .optEngine("PyTorch").optTranslatorFactory(new TextEmbeddingTranslatorFactory())
                    .optProgress(new ProgressBar()).build();

            predictor = criteria.loadModel().newPredictor();
        } catch (Exception e) {
            // ...
        }

const pipe = await transformers.pipeline(
    'feature-extraction', 'Xenova/all-MiniLM-L6-v2'
);

const pipeOptions = {
    pooling: 'mean',
    normalize: true,
};

Dictionary<string, dynamic> peopleData = new Dictionary<string, dynamic>
{
    ["Marie Curie"] = new
    {
        born = 1867,
        died = 1934,
        description = @"
        Polish-French chemist and physicist. The only person ever to win
        two Nobel prizes for two different sciences.
        "
    },
    ["Linus Pauling"] = new
    {
        born = 1901,
        died = 1994,
        description = @"
        American chemist and peace activist. One of only two people to win two
        Nobel prizes in different fields (chemistry and peace).
        "
    },
    ["Freddie Mercury"] = new
    {
        born = 1946,
        died = 1991,
        description = @"
        British musician, best known as the lead singer of the rock band
        Queen.
        "
    },
    ["Marie Fredriksson"] = new
    {
        born = 1958,
        died = 2019,
        description = @"
        Swedish multi-instrumentalist, mainly known as the lead singer and
        keyboardist of the band Roxette.
        "
    },
    ["Paul Erdos"] = new
    {
        born = 1913,
        died = 1996,
        description = @"
        Hungarian mathematician, known for his eccentric personality almost
        as much as his contributions to many different fields of mathematics.
        "
    },
    ["Maryam Mirzakhani"] = new
    {
        born = 1977,
        died = 2017,
        description = @"
        Iranian mathematician. The first woman ever to win the Fields medal
        for her contributions to mathematics.
        "
    },
    ["Masako Natsume"] = new
    {
        born = 1957,
        died = 1985,
        description = @"
        Japanese actress. She was very famous in Japan but was primarily
        known elsewhere in the world for her portrayal of Tripitaka in the
        TV series Monkey.
        "
    },
    ["Chaim Topol"] = new
    {
        born = 1935,
        died = 2023,
        description = @"
        Israeli actor and singer, usually credited simply as 'Topol'. He was
        best known for his many appearances as Tevye in the musical Fiddler
        on the Roof.
        "
    }
};

type PersonData struct {
	Born        int
	Died        int
	Description string
}

var peopleData = map[string]PersonData{
	"Marie Curie": {
		Born: 1867, Died: 1934,
		Description: `Polish-French chemist and physicist. The only person ever to win
		two Nobel prizes for two different sciences.
		`,
	},
	"Linus Pauling": {
		Born: 1901, Died: 1994,
		Description: `American chemist and peace activist. One of only two people to win two
		Nobel prizes in different fields (chemistry and peace).
		`,
	},
	"Freddie Mercury": {
		Born: 1946, Died: 1991,
		Description: `British musician, best known as the lead singer of the rock band
		Queen.
		`,
	},
	"Marie Fredriksson": {
		Born: 1958, Died: 2019,
		Description: `Swedish multi-instrumentalist, mainly known as the lead singer and
		keyboardist of the band Roxette.
		`,
	},
	"Paul Erdos": {
		Born: 1913, Died: 1996,
		Description: `Hungarian mathematician, known for his eccentric personality almost
		as much as his contributions to many different fields of mathematics.
		`,
	},
	"Maryam Mirzakhani": {
		Born: 1977, Died: 2017,
		Description: `Iranian mathematician. The first woman ever to win the Fields medal
		for her contributions to mathematics.
		`,
	},
	"Masako Natsume": {
		Born: 1957, Died: 1985,
		Description: `Japanese actress. She was very famous in Japan but was primarily
		known elsewhere in the world for her portrayal of Tripitaka in the
		TV series Monkey.
		`,
	},
	"Chaim Topol": {
		Born: 1935, Died: 2023,
		Description: `Israeli actor and singer, usually credited simply as 'Topol'. He was
		best known for his many appearances as Tevye in the musical Fiddler
		on the Roof.
		`,
	},
}

final class Person {
            final String name;
            final int born;
            final int died;
            final String description;
            Person(String name, int born, int died, String description) {
                this.name = name; this.born = born; this.died = died; this.description = description;
            }
        }

        List<Person> people = Arrays.asList(
            new Person(
                "Marie Curie",
                1867, 1934,
                "Polish-French chemist and physicist. The only person ever to win" +
                " two Nobel prizes for two different sciences."
            ),
            new Person(
                "Linus Pauling",
                1901, 1994,
                "American chemist and peace activist. One of only two people to" +
                " win two Nobel prizes in different fields (chemistry and peace)."
            ),
            new Person(
                "Freddie Mercury",
                1946, 1991,
                "British musician, best known as the lead singer of the rock band Queen."
            ),
            new Person(
                "Marie Fredriksson",
                1958, 2019,
                "Swedish multi-instrumentalist, mainly known as the lead singer and" +
                " keyboardist of the band Roxette."
            ),
            new Person(
                "Paul Erdos",
                1913, 1996,
                "Hungarian mathematician, known for his eccentric personality almost" +
                " as much as his contributions to many different fields of mathematics."
            ),
            new Person(
                "Maryam Mirzakhani",
                1977, 2017,
                "Iranian mathematician. The first woman ever to win the Fields medal" +
                " for her contributions to mathematics."
            ),
            new Person(
                "Masako Natsume",
                1957, 1985,
                "Japanese actress. She was very famous in Japan but was primarily" +
                " known elsewhere in the world for her portrayal of Tripitaka in the" +
                " TV series Monkey."
            ),
            new Person(
                "Chaim Topol",
                1935, 2023,
                "Israeli actor and singer, usually credited simply as 'Topol'. He was" +
                " best known for his many appearances as Tevye in the musical Fiddler" +
                " on the Roof."
            )
        );

const peopleData = {
    "Marie Curie": {
        "born": 1867, "died": 1934,
        "description": `
        Polish-French chemist and physicist. The only person ever to win
        two Nobel prizes for two different sciences.
        `
    },
    "Linus Pauling": {
        "born": 1901, "died": 1994,
        "description": `
        American chemist and peace activist. One of only two people to win two
        Nobel prizes in different fields (chemistry and peace).
        `
    },
    "Freddie Mercury": {
        "born": 1946, "died": 1991,
        "description": `
        British musician, best known as the lead singer of the rock band
        Queen.
        `
    },
    "Marie Fredriksson": {
        "born": 1958, "died": 2019,
        "description": `
        Swedish multi-instrumentalist, mainly known as the lead singer and
        keyboardist of the band Roxette.
        `
    },
    "Paul Erdos": {
        "born": 1913, "died": 1996,
        "description": `
        Hungarian mathematician, known for his eccentric personality almost
        as much as his contributions to many different fields of mathematics.
        `
    },
    "Maryam Mirzakhani": {
        "born": 1977, "died": 2017,
        "description": `
        Iranian mathematician. The first woman ever to win the Fields medal
        for her contributions to mathematics.
        `
    },
    "Masako Natsume": {
        "born": 1957, "died": 1985,
        "description": `
        Japanese actress. She was very famous in Japan but was primarily
        known elsewhere in the world for her portrayal of Tripitaka in the
        TV series Monkey.
        `
    },
    "Chaim Topol": {
        "born": 1935, "died": 2023,
        "description": `
        Israeli actor and singer, usually credited simply as 'Topol'. He was
        best known for his many appearances as Tevye in the musical Fiddler
        on the Roof.
        `
    }
};

peopleData = {
    "Marie Curie": {
        "born": 1867, "died": 1934,
        "description": """
        Polish-French chemist and physicist. The only person ever to win
        two Nobel prizes for two different sciences.
        """
    },
    "Linus Pauling": {
        "born": 1901, "died": 1994,
        "description": """
        American chemist and peace activist. One of only two people to win two
        Nobel prizes in different fields (chemistry and peace).
        """
    },
    "Freddie Mercury": {
        "born": 1946, "died": 1991,
        "description": """
        British musician, best known as the lead singer of the rock band
        Queen.
        """
    },
    "Marie Fredriksson": {
        "born": 1958, "died": 2019,
        "description": """
        Swedish multi-instrumentalist, mainly known as the lead singer and
        keyboardist of the band Roxette.
        """
    },
    "Paul Erdos": {
        "born": 1913, "died": 1996,
        "description": """
        Hungarian mathematician, known for his eccentric personality almost
        as much as his contributions to many different fields of mathematics.
        """
    },
    "Maryam Mirzakhani": {
        "born": 1977, "died": 2017,
        "description": """
        Iranian mathematician. The first woman ever to win the Fields medal
        for her contributions to mathematics.
        """
    },
    "Masako Natsume": {
        "born": 1957, "died": 1985,
        "description": """
        Japanese actress. She was very famous in Japan but was primarily
        known elsewhere in the world for her portrayal of Tripitaka in the
        TV series Monkey.
        """
    },
    "Chaim Topol": {
        "born": 1935, "died": 2023,
        "description": """
        Israeli actor and singer, usually credited simply as 'Topol'. He was
        best known for his many appearances as Tevye in the musical Fiddler
        on the Roof.
        """
    }
}

ConnectionMultiplexer muxer = ConnectionMultiplexer.Connect("localhost:6379");
IDatabase db = muxer.GetDatabase();

PredictionEngine<TextData, TransformedTextData> model = GetPredictionEngine();

foreach (KeyValuePair<string, dynamic> person in peopleData)
{
    string name = person.Key;
    dynamic details = person.Value;
    float[] embedding = GetEmbedding(model, details.description);

    VectorSetAddRequest addRequest = VectorSetAddRequest.Member(name, embedding, null);
    db.VectorSetAdd("famousPeople", addRequest);

    // Set attributes separately
    string attributesJson = $"{{\"born\": {details.born}, \"died\": {details.died}}}";
    db.VectorSetSetAttributesJson("famousPeople", name, attributesJson);
}

for name, details := range peopleData {
		// Generate embeddings using Hugot
		result, err := embeddingPipeline.RunPipeline([]string{details.Description})
		if err != nil {
			panic(err)
		}

		// Convert embedding to float64 slice
		embFloat32 := result.Embeddings[0]
		embFloat64 := make([]float64, len(embFloat32))
		for i, v := range embFloat32 {
			embFloat64[i] = float64(v)
		}

		// Add vector to vector set
		_, err = rdb.VAdd(ctx, "famousPeople", name, &redis.VectorValues{Val: embFloat64}).Result()
		if err != nil {
			panic(err)
		}

		// Set attributes for the element
		_, err = rdb.VSetAttr(ctx, "famousPeople", name, map[string]interface{}{
			"born": details.Born,
			"died": details.Died,
		}).Result()
		if err != nil {
			panic(err)
		}
	}

RedisClient redisClient = RedisClient.create("redis://localhost:6379");

        try (StatefulRedisConnection<String, String> connection = redisClient.connect()) {
            RedisAsyncCommands<String, String> asyncCommands = connection.async();

            CompletableFuture<?>[] vaddFutures = new CompletableFuture[people.size()];

            for (int i = 0; i < people.size(); i++) {
                Person person = people.get(i);
                Double[] embedding = null;

                try {
                    embedding = convertFloatsToDoubles(predictor.predict(person.description));
                } catch (Exception e) {
                    // ...
                }

                VAddArgs personArgs = VAddArgs.Builder.attributes(String.format("{\"born\": %d, \"died\": %d}", person.born, person.died));

                vaddFutures[i] = asyncCommands.vadd("famousPeople", person.name, personArgs, embedding)
                        .thenApply(result -> {
                            System.out.println(result); // >>> true
                            return result;
                        }).toCompletableFuture();
            }

            CompletableFuture.allOf(vaddFutures).join();

RedisClient redisClient = RedisClient.create("redis://localhost:6379");

        try (StatefulRedisConnection<String, String> connection = redisClient.connect()) {
            RedisReactiveCommands<String, String> reactiveCommands = connection.reactive();

            Mono<?>[] vaddFutures = new Mono[people.size()];

            for (int i = 0; i < people.size(); i++) {
                Person person = people.get(i);
                Double[] embedding = null;

                try {
                    embedding = convertFloatsToDoubles(predictor.predict(person.description));
                } catch (Exception e) {
                    // ...
                }

                VAddArgs personArgs = VAddArgs.Builder.attributes(String.format("{\"born\": %d, \"died\": %d}", person.born, person.died));

                vaddFutures[i] = reactiveCommands.vadd("famousPeople", person.name, personArgs, embedding)
                        .doOnNext(result -> {
                            System.out.println(result); // >>> true
                        });
            }

            Mono.when(vaddFutures).block();

RedisClient jedis = new RedisClient("redis://localhost:6379");

        for (Person person : people) {
            float[] embedding;
            try {
                embedding = predictor.predict(person.description);
            } catch (Exception e) {
                // This just allows the code to compile without errors.
                // In a real-world scenario, you would handle the exception properly.
                embedding = new float[384];
            }

            // Add element with attributes using VAddParams
            String attrs = String.format("{\"born\": %d, \"died\": %d}", person.born, person.died);
            boolean added = jedis.vadd("famousPeople", embedding, person.name, new VAddParams().setAttr(attrs));
            System.out.println(added); // >>> true
        }

const client = createClient({ url: 'redis://localhost:6379' });

client.on('error', err => console.log('Redis Client Error', err));
await client.connect();

for (const [name, details] of Object.entries(peopleData)) {
    const embedding = await pipe(details.description, pipeOptions);
    const embeddingArray = Array.from(embedding.data);

    await client.vAdd('famousPeople', embeddingArray, name);
    await client.vSetAttr('famousPeople', name, JSON.stringify({
        born: details.born,
        died: details.died
    }));
}

r = redis.Redis(decode_responses=True)

for name, details in peopleData.items():
    emb = model.encode(details["description"]).astype(np.float32).tobytes()

    r.vset().vadd(
        "famousPeople",
        emb,
        name,
        attributes={
            "born": details["born"],
            "died": details["died"]
        }
    )

string queryValue = "actors";

float[] queryEmbedding = GetEmbedding(model, queryValue);

VectorSetSimilaritySearchRequest basicQuery = VectorSetSimilaritySearchRequest.ByVector(queryEmbedding);
using (Lease<VectorSetSimilaritySearchResult>? actorsResults = db.VectorSetSimilaritySearch("famousPeople", basicQuery))
{
    IEnumerable<string> resultIds = actorsResults!.Span.ToArray()
        .Select(r => (string?)r.Member)
        .Where(id => id != null)
        .Select(id => $"'{id!}'");
    Console.WriteLine($"'actors': [{string.Join(", ", resultIds)}]");
}

queryValue := "actors"

	queryResult, err := embeddingPipeline.RunPipeline([]string{queryValue})
	if err != nil {
		panic(err)
	}

	// Convert embedding to float64 slice
	queryFloat32 := queryResult.Embeddings[0]
	queryFloat64 := make([]float64, len(queryFloat32))
	for i, v := range queryFloat32 {
		queryFloat64[i] = float64(v)
	}

	actorsResults, err := rdb.VSim(ctx, "famousPeople", &redis.VectorValues{Val: queryFloat64}).Result()
	if err != nil {
		panic(err)
	}

	fmt.Printf("'actors': %v\n", strings.Join(actorsResults, ", "))
	// >>> 'actors': Masako Natsume, Chaim Topol, Linus Pauling,
	// Marie Fredriksson, Maryam Mirzakhani, Marie Curie, Freddie Mercury,
	// Paul Erdos

Double[] actorsEmbedding = null;

            try {
                actorsEmbedding = convertFloatsToDoubles(predictor.predict("actors"));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<List<String>> basicQuery = asyncCommands.vsim("famousPeople", actorsEmbedding)
                    .thenApply(result -> {
                        System.out.println(result);
                        // >>> [Masako Natsume, Chaim Topol, Linus Pauling, Marie Fredriksson, Maryam Mirzakhani, Marie Curie, Freddie Mercury, Paul Erdos]
                        return result;
                    }).toCompletableFuture();

Double[] actorsEmbedding = null;

            try {
                actorsEmbedding = convertFloatsToDoubles(predictor.predict("actors"));
            } catch (Exception e) {
                // ...
            }

            Mono<java.util.List<String>> basicQuery = reactiveCommands.vsim("famousPeople", actorsEmbedding)
                    .collectList().doOnNext(result -> {
                        System.out.println(result);
                        // >>> [Masako Natsume, Chaim Topol, Linus Pauling, Marie Fredriksson, Maryam Mirzakhani, Marie Curie, Freddie Mercury, Paul Erdos]
                    });

float[] actorsEmbedding;
        try {
            actorsEmbedding = predictor.predict("actors");
        } catch (Exception e) {
            actorsEmbedding = new float[384];
        }

        List<String> basicResults = jedis.vsim("famousPeople", actorsEmbedding);
        System.out.println(basicResults);
        // >>> [Masako Natsume, Chaim Topol, Linus Pauling, Marie Fredriksson,
        // >>> Maryam Mirzakhani, Marie Curie, Freddie Mercury, Paul Erdos]

const queryValue = "actors";

const queryEmbedding = await pipe(queryValue, pipeOptions);
const queryArray = Array.from(queryEmbedding.data);

const actorsResults = await client.vSim('famousPeople', queryArray);

console.log(`'actors': ${JSON.stringify(actorsResults)}`);
// >>> 'actors': ["Masako Natsume","Chaim Topol","Linus Pauling",
// "Marie Fredriksson","Maryam Mirzakhani","Freddie Mercury",
// "Marie Curie","Paul Erdos"]

query_value = "actors"

actors_results = r.vset().vsim(
    "famousPeople",
    model.encode(query_value).astype(np.float32).tobytes(),
)

print(f"'actors': {actors_results}")

'actors': ['Masako Natsume', 'Chaim Topol', 'Linus Pauling',
'Marie Fredriksson', 'Maryam Mirzakhani', 'Marie Curie',
'Freddie Mercury', 'Paul Erdos']

queryValue = "actors";

queryEmbedding = GetEmbedding(model, queryValue);

VectorSetSimilaritySearchRequest limitedQuery = VectorSetSimilaritySearchRequest.ByVector(queryEmbedding);
limitedQuery.Count = 2;
using (Lease<VectorSetSimilaritySearchResult>? twoActorsResults = db.VectorSetSimilaritySearch("famousPeople", limitedQuery))
{
    IEnumerable<string> resultIds = twoActorsResults!.Span.ToArray()
        .Select(r => (string?)r.Member)
        .Where(id => id != null)
        .Select(id => $"'{id!}'");
    Console.WriteLine($"'actors (2)': [{string.Join(", ", resultIds)}]");
    // >>> 'actors (2)': ['Masako Natsume', 'Chaim Topol']
}

queryValue = "actors"

	queryResult, err = embeddingPipeline.RunPipeline([]string{queryValue})
	if err != nil {
		panic(err)
	}

	// Convert embedding to float64 slice
	queryFloat32 = queryResult.Embeddings[0]
	queryFloat64 = make([]float64, len(queryFloat32))
	for i, v := range queryFloat32 {
		queryFloat64[i] = float64(v)
	}

	twoActorsResults, err := rdb.VSimWithArgs(ctx, "famousPeople",
		&redis.VectorValues{Val: queryFloat64},
		&redis.VSimArgs{Count: 2}).Result()
	if err != nil {
		panic(err)
	}

	fmt.Printf("'actors (2)': %v\n", strings.Join(twoActorsResults, ", "))
	// >>> 'actors (2)': Masako Natsume, Chaim Topol

VSimArgs limitArgs = VSimArgs.Builder.count(2L);

            CompletableFuture<List<String>> limitedQuery = asyncCommands.vsim("famousPeople", limitArgs, actorsEmbedding)
                    .thenApply(result -> {
                        System.out.println(result);
                        // >>> [Masako Natsume, Chaim Topol]
                        return result;
                    }).toCompletableFuture();

VSimArgs limitArgs = VSimArgs.Builder.count(2L);

            Mono<java.util.List<String>> limitedQuery = reactiveCommands.vsim("famousPeople", limitArgs, actorsEmbedding)
                    .collectList().doOnNext(result -> {
                        System.out.println(result);
                        // >>> [Masako Natsume, Chaim Topol]
                    });

VSimParams limitParams = new VSimParams().count(2);
        List<String> limitedResults = jedis.vsim("famousPeople", actorsEmbedding, limitParams);
        System.out.println(limitedResults);
        // >>> [Masako Natsume, Chaim Topol]

const queryValue2 = "actors";

const queryEmbedding2 = await pipe(queryValue2, pipeOptions);
const queryArray2 = Array.from(queryEmbedding2.data);

const twoActorsResults = await client.vSim('famousPeople', queryArray2, {
    COUNT: 2
});

console.log(`'actors (2)': ${JSON.stringify(twoActorsResults)}`);
// >>> 'actors (2)': ["Masako Natsume","Chaim Topol"]

query_value = "actors"

two_actors_results = r.vset().vsim(
    "famousPeople",
    model.encode(query_value).astype(np.float32).tobytes(),
    count=2
)

print(f"'actors (2)': {two_actors_results}")
# >>> 'actors (2)': ['Masako Natsume', 'Chaim Topol']

queryValue = "entertainer";

queryEmbedding = GetEmbedding(model, queryValue);

VectorSetSimilaritySearchRequest entertainerQuery = VectorSetSimilaritySearchRequest.ByVector(queryEmbedding);
using (Lease<VectorSetSimilaritySearchResult>? entertainerResults = db.VectorSetSimilaritySearch("famousPeople", entertainerQuery))
{
    IEnumerable<string> resultIds = entertainerResults!.Span.ToArray()
        .Select(r => (string?)r.Member)
        .Where(id => id != null)
        .Select(id => $"'{id!}'");
    Console.WriteLine($"'entertainer': [{string.Join(", ", resultIds)}]");
    // >>> 'entertainer': ['Chaim Topol', 'Freddie Mercury',
    // 'Marie Fredriksson', 'Masako Natsume', 'Linus Pauling',
    // 'Paul Erdos', 'Maryam Mirzakhani', 'Marie Curie']
}

queryValue = "entertainer"

	queryResult, err = embeddingPipeline.RunPipeline([]string{queryValue})
	if err != nil {
		panic(err)
	}

	// Convert embedding to float64 slice
	queryFloat32 = queryResult.Embeddings[0]
	queryFloat64 = make([]float64, len(queryFloat32))
	for i, v := range queryFloat32 {
		queryFloat64[i] = float64(v)
	}

	entertainerResults, err := rdb.VSim(ctx, "famousPeople", &redis.VectorValues{Val: queryFloat64}).Result()
	if err != nil {
		panic(err)
	}

	fmt.Printf("'entertainer': %v\n", strings.Join(entertainerResults, ", "))
	// >>> 'entertainer': Chaim Topol, Freddie Mercury, Marie Fredriksson,
	// Linus Pauling, Masako Natsume, Paul Erdos,
	// Maryam Mirzakhani, Marie Curie

Double[] entertainerEmbedding = null;

            try {
                entertainerEmbedding = convertFloatsToDoubles(predictor.predict("entertainers"));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<List<String>> entertainerQuery = asyncCommands.vsim("famousPeople", entertainerEmbedding)
                    .thenApply(result -> {
                        System.out.println(result);
                        // >>> [Freddie Mercury, Chaim Topol, Linus Pauling, Marie Fredriksson, Masako Natsume, Paul Erdos, Maryam Mirzakhani, Marie Curie]
                        return result;
                    }).toCompletableFuture();

Double[] entertainerEmbedding = null;

            try {
                entertainerEmbedding = convertFloatsToDoubles(predictor.predict("entertainers"));
            } catch (Exception e) {
                // ...
            }

            Mono<java.util.List<String>> entertainerQuery = reactiveCommands.vsim("famousPeople", entertainerEmbedding)
                    .collectList().doOnNext(result -> {
                        System.out.println(result);
                        // >>> [Freddie Mercury, Chaim Topol, Linus Pauling, Marie Fredriksson, Masako Natsume, Paul Erdos, Maryam Mirzakhani, Marie Curie]
                    });

float[] entertainerEmbedding;
        try {
            entertainerEmbedding = predictor.predict("entertainers");
        } catch (Exception e) {
            entertainerEmbedding = new float[384];
        }

        List<String> entertainerResults = jedis.vsim("famousPeople", entertainerEmbedding);
        System.out.println(entertainerResults);
        // >>> [Freddie Mercury, Chaim Topol, Linus Pauling, Marie Fredriksson,
        // >>> Masako Natsume, Paul Erdos, Maryam Mirzakhani, Marie Curie]

const queryValue3 = "entertainer";

const queryEmbedding3 = await pipe(queryValue3, pipeOptions);
const queryArray3 = Array.from(queryEmbedding3.data);

const entertainerResults = await client.vSim('famousPeople', queryArray3);

console.log(`'entertainer': ${JSON.stringify(entertainerResults)}`);
// >>> 'actors': ["Masako Natsume","Chaim Topol","Linus Pauling",
// "Marie Fredriksson","Maryam Mirzakhani","Freddie Mercury",
// "Marie Curie","Paul Erdos"]

query_value = "entertainer"

entertainer_results = r.vset().vsim(
    "famousPeople",
    model.encode(query_value).astype(np.float32).tobytes()
)

print(f"'entertainer': {entertainer_results}")
# >>> 'entertainer': ['Chaim Topol', 'Freddie Mercury',
# 'Marie Fredriksson', 'Masako Natsume', 'Linus Pauling',
# 'Paul Erdos', 'Maryam Mirzakhani', 'Marie Curie']

'science': ['Marie Curie', 'Linus Pauling', 'Maryam Mirzakhani',
'Paul Erdos', 'Marie Fredriksson', 'Freddie Mercury', 'Masako Natsume',
'Chaim Topol']

queryValue = "science";

queryEmbedding = GetEmbedding(model, queryValue);

VectorSetSimilaritySearchRequest filteredQuery = VectorSetSimilaritySearchRequest.ByVector(queryEmbedding);
filteredQuery.FilterExpression = ".died < 2000";
using (Lease<VectorSetSimilaritySearchResult>? science2000Results = db.VectorSetSimilaritySearch("famousPeople", filteredQuery))
{
    IEnumerable<string> resultIds = science2000Results!.Span.ToArray()
        .Select(r => (string?)r.Member)
        .Where(id => id != null)
        .Select(id => $"'{id!}'");
    Console.WriteLine($"'science2000': [{string.Join(", ", resultIds)}]");
    // >>> 'science2000': ['Marie Curie', 'Linus Pauling',
    // 'Paul Erdos', 'Freddie Mercury', 'Masako Natsume']
}

queryValue = "science"

	queryResult, err = embeddingPipeline.RunPipeline([]string{queryValue})
	if err != nil {
		panic(err)
	}

	// Convert embedding to float64 slice
	queryFloat32 = queryResult.Embeddings[0]
	queryFloat64 = make([]float64, len(queryFloat32))
	for i, v := range queryFloat32 {
		queryFloat64[i] = float64(v)
	}

	science2000Results, err := rdb.VSimWithArgs(ctx, "famousPeople",
		&redis.VectorValues{Val: queryFloat64},
		&redis.VSimArgs{Filter: ".died < 2000"}).Result()
	if err != nil {
		panic(err)
	}

	fmt.Printf("'science2000': %v\n", strings.Join(science2000Results, ", "))
	// >>> 'science2000': Marie Curie, Linus Pauling, Paul Erdos, Freddie Mercury,
	// Masako Natsume

Double[] science2000Embedding = null;

            try {
                science2000Embedding = convertFloatsToDoubles(predictor.predict("science"));
            } catch (Exception e) {
                // ...
            }

            VSimArgs filterArgs = VSimArgs.Builder.filter(".died < 2000");

            CompletableFuture<List<String>> filteredQuery = asyncCommands.vsim("famousPeople", filterArgs, science2000Embedding)
                    .thenApply(result -> {
                        System.out.println(result);
                        // >>> [Marie Curie, Linus Pauling, Paul Erdos, Freddie Mercury, Masako Natsume]
                        return result;
                    }).toCompletableFuture();

Double[] science2000Embedding = null;

            try {
                science2000Embedding = convertFloatsToDoubles(predictor.predict("science"));
            } catch (Exception e) {
                // ...
            }

            VSimArgs filterArgs = VSimArgs.Builder.filter(".died < 2000");

            Mono<java.util.List<String>> filteredQuery = reactiveCommands.vsim("famousPeople", filterArgs, science2000Embedding)
                    .collectList().doOnNext(result -> {
                        System.out.println(result);
                        // >>> [Marie Curie, Linus Pauling, Paul Erdos, Freddie Mercury, Masako Natsume]
                    });

float[] science2000Embedding;
        try {
            science2000Embedding = predictor.predict("science");
        } catch (Exception e) {
            science2000Embedding = new float[384];
        }

        VSimParams filterParams = new VSimParams().filter(".died < 2000");
        List<String> filteredResults = jedis.vsim("famousPeople", science2000Embedding, filterParams);
        System.out.println(filteredResults);
        // >>> [Marie Curie, Linus Pauling, Paul Erdos, Freddie Mercury, Masako Natsume]

const queryValue5 = "science";

const queryEmbedding5 = await pipe(queryValue5, pipeOptions);
const queryArray5 = Array.from(queryEmbedding5.data);

const science2000Results = await client.vSim('famousPeople', queryArray5, {
    FILTER: '.died < 2000'
});

console.log(`'science2000': ${JSON.stringify(science2000Results)}`);
// >>> 'science2000': ["Linus Pauling","Marie Curie","Paul Erdos",
// "Masako Natsume","Freddie Mercury"]

query_value = "science"

science2000_results = r.vset().vsim(
    "famousPeople",
    model.encode(query_value).astype(np.float32).tobytes(),
    filter=".died < 2000"
)

print(f"'science2000': {science2000_results}")
# >>> 'science2000': ['Marie Curie', 'Linus Pauling',
# 'Paul Erdos', 'Freddie Mercury', 'Masako Natsume']


Redis vector library guide (Python)

Summary: Connect your Python vector application to a Redis vector database

Content

See the [RedisVL Guide]() for more information.