title: Index and query vectors

title: Initialize

text:
Install the required dependencies:

1. Install [`node-redis`]() if you haven't already.
2. Install `@xenova/transformers`:

[code example]

In your JavaScript source file, import the required classes:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

The `@xenova/transformers` module handles embedding models. This example uses the
[`all-distilroberta-v1`](https://huggingface.co/sentence-transformers/all-distilroberta-v1)
model, which:
- Generates 768-dimensional vectors
- Truncates input to 128 tokens
- Uses word piece tokenization (see [Word piece tokenization](https://huggingface.co/learn/nlp-course/en/chapter6/6)
  at the [Hugging Face](https://huggingface.co/) docs for details)

The `pipe` function generates embeddings. The `pipeOptions` object specifies how to generate sentence embeddings from token embeddings (see the
[`all-distilroberta-v1`](https://huggingface.co/sentence-transformers/all-distilroberta-v1)
documentation for details):

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

title: Create the index

text:
First, connect to Redis and remove any existing index named `vector_idx`:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Next, create the index with the following schema:
-   `content`: Text field for the content to index
-   `genre`: [Tag]()
    field representing the text's genre
-   `embedding`: [Vector]()
    field with:
    -   [HNSW]()
        indexing
    -   [L2]()
        distance metric
    -   Float32 values
    -   768 dimensions (matching the embedding model)

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

title: Add data

text:
Add data objects to the index using `hSet()`. The index automatically processes objects with the `doc:` prefix.

For each document:
1. Generate an embedding using the `pipe()` function and `pipeOptions`
2. Convert the embedding to a binary string using `Buffer.from()`
3. Store the document with `hSet()`

Use `Promise.all()` to batch the commands and reduce network round trips:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

title: Run a query

text:
To query the index:
1. Generate an embedding for your query text
2. Pass the embedding as a parameter to the search
3. Redis calculates vector distances and ranks results

The query returns an array of document objects. Each object contains:
- `id`: The document's key
- `value`: An object with fields specified in the `RETURN` option

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

The first run may take longer as it downloads the model data. The output shows results ordered by score (vector distance), with lower scores indicating greater similarity:

[code example]

title: Differences with JSON documents

text:
JSON documents support richer data modeling with nested fields. Key differences from hash documents:

1. Use paths in the schema to identify fields
2. Declare aliases for paths using the `AS` option
3. Set `ON` to `JSON` when creating the index
4. Use arrays instead of binary strings for vectors
5. Use `json.set()` instead of `hSet()`

Create the index with path aliases:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Add data using `json.set()`. Convert the `Float32Array` to a standard JavaScript array using the spread operator:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Query JSON documents using the same syntax, but note that the vector parameter must still be a binary string:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), JavaScript (Node.js), Python

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

The results are identical to the hash document query, except for the `jdoc:` prefix:

[code example]

title: Learn more

text:
See
[Vector search]()
for more information about indexing options, distance metrics, and query format.

code: npm install @xenova/transformers

code:
import * as transformers from '@xenova/transformers';
import {
    VectorAlgorithms,
    createClient,
    SchemaFieldTypes,
} from 'redis';

code:
package main

import (
	"context"
	"encoding/binary"
	"fmt"
	"math"

	"github.com/knights-analytics/hugot"
	"github.com/redis/go-redis/v9"
)

func floatsToBytes(fs []float32) []byte {
	buf := make([]byte, len(fs)*4)

	for i, f := range fs {
		u := math.Float32bits(f)
		binary.NativeEndian.PutUint32(buf[i*4:], u)
	}

	return buf
}

func main() {
	ctx := context.Background()
	rdb := redis.NewClient(&redis.Options{
		Addr:     "localhost:6379",
		Password: "", // no password docs
		DB:       0,  // use default DB
		Protocol: 2,
	})

	rdb.FTDropIndexWithArgs(ctx,
		"vector_idx",
		&redis.FTDropIndexOptions{
			DeleteDocs: true,
		},
	)

	_, err := rdb.FTCreate(ctx,
		"vector_idx",
		&redis.FTCreateOptions{
			OnHash: true,
			Prefix: []any{"doc:"},
		},
		&redis.FieldSchema{
			FieldName: "content",
			FieldType: redis.SearchFieldTypeText,
		},
		&redis.FieldSchema{
			FieldName: "genre",
			FieldType: redis.SearchFieldTypeTag,
		},
		&redis.FieldSchema{
			FieldName: "embedding",
			FieldType: redis.SearchFieldTypeVector,
			VectorArgs: &redis.FTVectorArgs{
				HNSWOptions: &redis.FTHNSWOptions{
					Dim:            384,
					DistanceMetric: "L2",
					Type:           "FLOAT32",
				},
			},
		},
	).Result()
	if err != nil {
		panic(err)
	}

	// Create a Hugot session
	session, err := hugot.NewGoSession()
	if err != nil {
		panic(err)
	}
	defer func() {
		err := session.Destroy()
		if err != nil {
			panic(err)
		}
	}()

	// Download the model
	downloadOptions := hugot.NewDownloadOptions()
	downloadOptions.OnnxFilePath = "onnx/model.onnx" // Specify which ONNX file to use
	modelPath, err := hugot.DownloadModel("sentence-transformers/all-MiniLM-L6-v2", "./models/", downloadOptions)
	if err != nil {
		panic(err)
	}

	// Create feature extraction pipeline configuration
	config := hugot.FeatureExtractionConfig{
		ModelPath: modelPath,
		Name:      "embeddingPipeline",
	}

	// Create the feature extraction pipeline
	embeddingPipeline, err := hugot.NewPipeline(session, config)
	if err != nil {
		panic(err)
	}

	sentences := []string{
		"That is a very happy person",
		"That is a happy dog",
		"Today is a sunny day",
	}

	tags := []string{
		"persons", "pets", "weather",
	}

	// Generate embeddings using Hugot
	embeddingResult, err := embeddingPipeline.RunPipeline(sentences)
	if err != nil {
		panic(err)
	}

	// Extract the embeddings from the result
	embeddings := embeddingResult.Embeddings

	for i, emb := range embeddings {
		buffer := floatsToBytes(emb)

		_, err = rdb.HSet(ctx,
			fmt.Sprintf("doc:%v", i),
			map[string]any{
				"content":   sentences[i],
				"genre":     tags[i],
				"embedding": buffer,
			},
		).Result()

		if err != nil {
			panic(err)
		}
	}

	// Generate query embedding using Hugot
	queryResult, err := embeddingPipeline.RunPipeline([]string{
		"That is a happy person",
	})

	if err != nil {
		panic(err)
	}

	buffer := floatsToBytes(queryResult.Embeddings[0])

	results, err := rdb.FTSearchWithArgs(ctx,
		"vector_idx",
		"*=>[KNN 3 @embedding $vec AS vector_distance]",
		&redis.FTSearchOptions{
			Return: []redis.FTSearchReturn{
				{FieldName: "vector_distance"},
				{FieldName: "content"},
			},
			DialectVersion: 2,
			Params: map[string]any{
				"vec": buffer,
			},
		},
	).Result()

	if err != nil {
		panic(err)
	}

	for _, doc := range results.Docs {
		fmt.Printf(
			"ID: %v, Distance:%v, Content:'%v'\n",
			doc.ID, doc.Fields["vector_distance"], doc.Fields["content"],
		)
	}

	rdb.FTDropIndexWithArgs(ctx,
		"vector_json_idx",
		&redis.FTDropIndexOptions{
			DeleteDocs: true,
		},
	)

	_, err = rdb.FTCreate(ctx,
		"vector_json_idx",
		&redis.FTCreateOptions{
			OnJSON: true,
			Prefix: []any{"jdoc:"},
		},
		&redis.FieldSchema{
			FieldName: "$.content",
			As:        "content",
			FieldType: redis.SearchFieldTypeText,
		},
		&redis.FieldSchema{
			FieldName: "$.genre",
			As:        "genre",
			FieldType: redis.SearchFieldTypeTag,
		},
		&redis.FieldSchema{
			FieldName: "$.embedding",
			As:        "embedding",
			FieldType: redis.SearchFieldTypeVector,
			VectorArgs: &redis.FTVectorArgs{
				HNSWOptions: &redis.FTHNSWOptions{
					Dim:            384,
					DistanceMetric: "L2",
					Type:           "FLOAT32",
				},
			},
		},
	).Result()
	if err != nil {
		panic(err)
	}

	for i, emb := range embeddings {
		_, err = rdb.JSONSet(ctx,
			fmt.Sprintf("jdoc:%v", i),
			"$",
			map[string]any{
				"content":   sentences[i],
				"genre":     tags[i],
				"embedding": emb,
			},
		).Result()

		if err != nil {
			panic(err)
		}
	}

	// Generate query embedding for JSON search using Hugot
	jsonQueryResult, err := embeddingPipeline.RunPipeline([]string{
		"That is a happy person",
	})

	if err != nil {
		panic(err)
	}

	jsonBuffer := floatsToBytes(jsonQueryResult.Embeddings[0])

	jsonResults, err := rdb.FTSearchWithArgs(ctx,
		"vector_json_idx",
		"*=>[KNN 3 @embedding $vec AS vector_distance]",
		&redis.FTSearchOptions{
			Return: []redis.FTSearchReturn{
				{FieldName: "vector_distance"},
				{FieldName: "content"},
			},
			DialectVersion: 2,
			Params: map[string]any{
				"vec": jsonBuffer,
			},
		},
	).Result()

	if err != nil {
		panic(err)
	}

	for _, doc := range jsonResults.Docs {
		fmt.Printf(
			"ID: %v, Distance:%v, Content:'%v'\n",
			doc.ID, doc.Fields["vector_distance"], doc.Fields["content"],
		)
	}
}

code:
package io.redis.examples.async;

// Lettuce client and Redis Search classes.
import io.lettuce.core.*;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.api.async.RedisAsyncCommands;
import io.lettuce.core.search.arguments.*;
import io.lettuce.core.search.SearchReply;
import io.lettuce.core.json.JsonParser;
import io.lettuce.core.json.JsonObject;
import io.lettuce.core.json.JsonPath;

// Standard library classes for data manipulation and
// asynchronous programming.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.StandardCharsets;
import java.util.*;
import java.util.concurrent.CompletableFuture;

// DJL classes for model loading and inference.
import ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory;
import ai.djl.inference.Predictor;
import ai.djl.repository.zoo.Criteria;
import ai.djl.training.util.ProgressBar;

public class HomeQueryVecExample {

    private ByteBuffer floatArrayToByteBuffer(float[] vector) {
        ByteBuffer buffer = ByteBuffer.allocate(vector.length * 4).order(ByteOrder.LITTLE_ENDIAN);
        for (float value : vector) {
            buffer.putFloat(value);
        }
        return (ByteBuffer) buffer.flip();
    }

    public void run() {
        Predictor<String, float[]> predictor = null;

        try {
            Criteria<String, float[]> criteria = Criteria.builder().setTypes(String.class, float[].class)
                    .optModelUrls("djl://ai.djl.huggingface.pytorch/sentence-transformers/all-MiniLM-L6-v2")
                    .optEngine("PyTorch").optTranslatorFactory(new TextEmbeddingTranslatorFactory())
                    .optProgress(new ProgressBar()).build();

            predictor = criteria.loadModel().newPredictor();
        } catch (Exception e) {
            // ...
        }

        RedisClient redisClient = RedisClient.create("redis://localhost:6379");

        try (StatefulRedisConnection<String, String> connection = redisClient.connect();
                StatefulRedisConnection<ByteBuffer, ByteBuffer> binConnection = redisClient.connect(new ByteBufferCodec())) {
            RedisAsyncCommands<String, String> asyncCommands = connection.async();
            RedisAsyncCommands<ByteBuffer, ByteBuffer> binAsyncCommands = binConnection.async();
            // ...

            List<FieldArgs<String>> schema = Arrays.asList(TextFieldArgs.<String> builder().name("content").build(),
                    TagFieldArgs.<String> builder().name("genre").build(),
                    VectorFieldArgs.<String> builder().name("embedding").hnsw().type(VectorFieldArgs.VectorType.FLOAT32)
                            .dimensions(384).distanceMetric(VectorFieldArgs.DistanceMetric.L2).build());

            CreateArgs<String, String> createArgs = CreateArgs.<String, String> builder().on(CreateArgs.TargetType.HASH)
                    .withPrefix("doc:").build();

            CompletableFuture<Void> createIndex = asyncCommands.ftCreate("vector_idx", createArgs, schema)
                    .thenAccept(System.out::println).toCompletableFuture();
            createIndex.join();

            String sentence1 = "That is a very happy person";

            Map<ByteBuffer, ByteBuffer> doc1 = new HashMap<>();
            doc1.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence1.getBytes()));
            doc1.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("persons".getBytes()));

            try {
                doc1.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence1)));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<Long> addDoc1 = binAsyncCommands.hset(ByteBuffer.wrap("doc:1".getBytes()), doc1)
                    .thenApply(result -> {
                        System.out.println(result); // >>> 3
                        return result;
                    }).toCompletableFuture();

            String sentence2 = "That is a happy dog";

            Map<ByteBuffer, ByteBuffer> doc2 = new HashMap<>();
            doc2.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence2.getBytes()));
            doc2.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("pets".getBytes()));

            try {
                doc2.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence2)));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<Long> addDoc2 = binAsyncCommands.hset(ByteBuffer.wrap("doc:2".getBytes()), doc2)
                    .thenApply(result -> {
                        System.out.println(result); // >>> 3
                        return result;
                    }).toCompletableFuture();

            String sentence3 = "Today is a sunny day";

            Map<ByteBuffer, ByteBuffer> doc3 = new HashMap<>();
            doc3.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence3.getBytes()));
            doc3.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("weather".getBytes()));

            try {
                doc3.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence3)));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<Long> addDoc3 = binAsyncCommands.hset(ByteBuffer.wrap("doc:3".getBytes()), doc3)
                    .thenApply(result -> {
                        System.out.println(result); // >>> 3
                        return result;
                    }).toCompletableFuture();
            CompletableFuture.allOf(addDoc1, addDoc2, addDoc3).join();

            String query = "That is a happy person";
            float[] queryEmbedding = null;

            try {
                queryEmbedding = predictor.predict(query);
            } catch (Exception e) {
                // ...
            }

            SearchArgs<ByteBuffer, ByteBuffer> searchArgs = SearchArgs.<ByteBuffer, ByteBuffer> builder()
                    .param(ByteBuffer.wrap("vec".getBytes()), floatArrayToByteBuffer(queryEmbedding))
                    .returnField(ByteBuffer.wrap("content".getBytes()))
                    .returnField(ByteBuffer.wrap("vector_distance".getBytes()))
                    .sortBy(SortByArgs.<ByteBuffer> builder().attribute(ByteBuffer.wrap("vector_distance".getBytes())).build())
                    .build();

            CompletableFuture<SearchReply<ByteBuffer, ByteBuffer>> hashQuery = binAsyncCommands
                    .ftSearch(ByteBuffer.wrap("vector_idx".getBytes()),
                            ByteBuffer.wrap("*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes()), searchArgs)
                    .thenApply(result -> {
                        List<SearchReply.SearchResult<ByteBuffer, ByteBuffer>> results = result.getResults();

                        results.forEach(r -> {
                            String id = StandardCharsets.UTF_8.decode(r.getId()).toString();
                            String content = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("content".getBytes()))).toString();
                            String distance = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("vector_distance".getBytes()))).toString();

                            System.out.println("ID: " + id + ", Content: " + content + ", Distance: " + distance);
                        });
                        // >>> ID: doc:1, Content: That is a very happy person, Distance: 0.114169836044
                        // >>> ID: doc:2, Content: That is a happy dog, Distance: 0.610845506191
                        // >>> ID: doc:3, Content: Today is a sunny day, Distance: 1.48624765873

                        return result;
                    }).toCompletableFuture();
            hashQuery.join();

            List<FieldArgs<String>> jsonSchema = Arrays.asList(
                    TextFieldArgs.<String> builder().name("$.content").as("content").build(),
                    TagFieldArgs.<String> builder().name("$.genre").as("genre").build(),
                    VectorFieldArgs.<String> builder().name("$.embedding").as("embedding").hnsw()
                            .type(VectorFieldArgs.VectorType.FLOAT32).dimensions(384)
                            .distanceMetric(VectorFieldArgs.DistanceMetric.L2).build());

            CreateArgs<String, String> jsonCreateArgs = CreateArgs.<String, String> builder().on(CreateArgs.TargetType.JSON)
                    .withPrefix("jdoc:").build();

            CompletableFuture<Void> jsonCreateIndex = asyncCommands.ftCreate("vector_json_idx", jsonCreateArgs, jsonSchema)
                    .thenAccept(System.out::println).toCompletableFuture();
            jsonCreateIndex.join();

            JsonParser parser = asyncCommands.getJsonParser();

            String jSentence1 = "\"That is a very happy person\"";

            JsonObject jDoc1 = parser.createJsonObject();
            jDoc1.put("content", parser.createJsonValue(jSentence1));
            jDoc1.put("genre", parser.createJsonValue("\"persons\""));

            try {
                jDoc1.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence1))));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<String> jsonAddDoc1 = asyncCommands.jsonSet("jdoc:1", JsonPath.ROOT_PATH, jDoc1)
                    .thenApply(result -> {
                        System.out.println(result); // >>> OK
                        return result;
                    }).toCompletableFuture();

            String jSentence2 = "\"That is a happy dog\"";

            JsonObject jDoc2 = parser.createJsonObject();
            jDoc2.put("content", parser.createJsonValue(jSentence2));
            jDoc2.put("genre", parser.createJsonValue("\"pets\""));

            try {
                jDoc2.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence2))));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<String> jsonAddDoc2 = asyncCommands.jsonSet("jdoc:2", JsonPath.ROOT_PATH, jDoc2)
                    .thenApply(result -> {
                        System.out.println(result); // >>> OK
                        return result;
                    }).toCompletableFuture();

            String jSentence3 = "\"Today is a sunny day\"";

            JsonObject jDoc3 = parser.createJsonObject();
            jDoc3.put("content", parser.createJsonValue(jSentence3));
            jDoc3.put("genre", parser.createJsonValue("\"weather\""));

            try {
                jDoc3.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence3))));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<String> jsonAddDoc3 = asyncCommands.jsonSet("jdoc:3", JsonPath.ROOT_PATH, jDoc3)
                    .thenApply(result -> {
                        System.out.println(result); // >>> OK
                        return result;
                    }).toCompletableFuture();
            CompletableFuture.allOf(jsonAddDoc1, jsonAddDoc2, jsonAddDoc3).join();

            String jQuery = "That is a happy person";
            float[] jsonQueryEmbedding = null;

            try {
                jsonQueryEmbedding = predictor.predict(jQuery);
            } catch (Exception e) {
                // ...
            }

            SearchArgs<ByteBuffer, ByteBuffer> jsonSearchArgs = SearchArgs.<ByteBuffer, ByteBuffer> builder()
                    .param(ByteBuffer.wrap("vec".getBytes()), floatArrayToByteBuffer(jsonQueryEmbedding))
                    .returnField(ByteBuffer.wrap("content".getBytes()))
                    .returnField(ByteBuffer.wrap("vector_distance".getBytes()))
                    .sortBy(SortByArgs.<ByteBuffer> builder().attribute(ByteBuffer.wrap("vector_distance".getBytes())).build())
                    .build();

            CompletableFuture<SearchReply<ByteBuffer, ByteBuffer>> jsonQuery = binAsyncCommands
                    .ftSearch(ByteBuffer.wrap("vector_json_idx".getBytes()),
                            ByteBuffer.wrap("*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes()), jsonSearchArgs)
                    .thenApply(result -> {
                        List<SearchReply.SearchResult<ByteBuffer, ByteBuffer>> results = result.getResults();

                        results.forEach(r -> {
                            String id = StandardCharsets.UTF_8.decode(r.getId()).toString();
                            String content = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("content".getBytes()))).toString();
                            String distance = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("vector_distance".getBytes()))).toString();

                            System.out.println("ID: " + id + ", Content: " + content + ", Distance: " + distance);
                        });
                        // >>> ID: jdoc:1, Content: "That is a very happy person", Distance:0.628328084946
                        // >>> ID: jdoc:2, Content: That is a happy dog, Distance: 0.895147025585
                        // >>> ID: jdoc:3, Content: "Today is a sunny day", Distance: 1.49569523335

                        return result;
                    }).toCompletableFuture();
            jsonQuery.join();
        } finally {
            redisClient.shutdown();
        }
    }

}

code:
package io.redis.examples.reactive;

// Lettuce client and Redis Search classes.
import io.lettuce.core.*;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.api.reactive.RedisReactiveCommands;
import io.lettuce.core.search.arguments.*;
import io.lettuce.core.search.SearchReply;
import io.lettuce.core.json.JsonParser;
import io.lettuce.core.json.JsonObject;
import io.lettuce.core.json.JsonPath;

// Standard library classes for data manipulation and
// reactive programming.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.StandardCharsets;
import java.util.*;
import reactor.core.publisher.Mono;

// DJL classes for model loading and inference.
import ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory;
import ai.djl.inference.Predictor;
import ai.djl.repository.zoo.Criteria;
import ai.djl.training.util.ProgressBar;

public class HomeQueryVecExample {

    private ByteBuffer floatArrayToByteBuffer(float[] vector) {
        ByteBuffer buffer = ByteBuffer.allocate(vector.length * 4).order(ByteOrder.LITTLE_ENDIAN);
        for (float value : vector) {
            buffer.putFloat(value);
        }
        return (ByteBuffer) buffer.flip();
    }

    public void run() {
        Predictor<String, float[]> predictor = null;

        try {
            Criteria<String, float[]> criteria = Criteria.builder().setTypes(String.class, float[].class)
                    .optModelUrls("djl://ai.djl.huggingface.pytorch/sentence-transformers/all-MiniLM-L6-v2")
                    .optEngine("PyTorch").optTranslatorFactory(new TextEmbeddingTranslatorFactory())
                    .optProgress(new ProgressBar()).build();

            predictor = criteria.loadModel().newPredictor();
        } catch (Exception e) {
            // ...
        }

        RedisClient redisClient = RedisClient.create("redis://localhost:6379");

        try (StatefulRedisConnection<String, String> connection = redisClient.connect();
                StatefulRedisConnection<ByteBuffer, ByteBuffer> binConnection = redisClient.connect(new ByteBufferCodec())) {
            RedisReactiveCommands<String, String> reactiveCommands = connection.reactive();
            RedisReactiveCommands<ByteBuffer, ByteBuffer> binReactiveCommands = binConnection.reactive();
            // ...

            List<FieldArgs<String>> schema = Arrays.asList(TextFieldArgs.<String> builder().name("content").build(),
                    TagFieldArgs.<String> builder().name("genre").build(),
                    VectorFieldArgs.<String> builder().name("embedding").hnsw().type(VectorFieldArgs.VectorType.FLOAT32)
                            .dimensions(384).distanceMetric(VectorFieldArgs.DistanceMetric.L2).build());

            CreateArgs<String, String> createArgs = CreateArgs.<String, String> builder().on(CreateArgs.TargetType.HASH)
                    .withPrefix("doc:").build();

            Mono<Void> createIndex = reactiveCommands.ftCreate("vector_idx", createArgs, schema).doOnNext(result -> {
                System.out.println(result); // >>> OK
            }).then();
            createIndex.block();

            String sentence1 = "That is a very happy person";

            Map<ByteBuffer, ByteBuffer> doc1 = new HashMap<>();
            doc1.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence1.getBytes()));
            doc1.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("persons".getBytes()));

            try {
                doc1.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence1)));
            } catch (Exception e) {
                // ...
            }

            Mono<Long> addDoc1 = binReactiveCommands.hset(ByteBuffer.wrap("doc:1".getBytes()), doc1).doOnNext(result -> {
                System.out.println(result); // >>> 3
            });

            String sentence2 = "That is a happy dog";

            Map<ByteBuffer, ByteBuffer> doc2 = new HashMap<>();
            doc2.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence2.getBytes()));
            doc2.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("pets".getBytes()));

            try {
                doc2.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence2)));
            } catch (Exception e) {
                // ...
            }

            Mono<Long> addDoc2 = binReactiveCommands.hset(ByteBuffer.wrap("doc:2".getBytes()), doc2).doOnNext(result -> {
                System.out.println(result); // >>> 3
            });

            String sentence3 = "Today is a sunny day";

            Map<ByteBuffer, ByteBuffer> doc3 = new HashMap<>();
            doc3.put(ByteBuffer.wrap("content".getBytes()), ByteBuffer.wrap(sentence3.getBytes()));
            doc3.put(ByteBuffer.wrap("genre".getBytes()), ByteBuffer.wrap("weather".getBytes()));

            try {
                doc3.put(ByteBuffer.wrap("embedding".getBytes()), floatArrayToByteBuffer(predictor.predict(sentence3)));
            } catch (Exception e) {
                // ...
            }

            Mono<Long> addDoc3 = binReactiveCommands.hset(ByteBuffer.wrap("doc:3".getBytes()), doc3).doOnNext(result -> {
                System.out.println(result); // >>> 3
            });
            Mono.when(addDoc1, addDoc2, addDoc3).block();

            String query = "That is a happy person";
            float[] queryEmbedding = null;

            try {
                queryEmbedding = predictor.predict(query);
            } catch (Exception e) {
                // ...
            }

            SearchArgs<ByteBuffer, ByteBuffer> searchArgs = SearchArgs.<ByteBuffer, ByteBuffer> builder()
                    .param(ByteBuffer.wrap("vec".getBytes()), floatArrayToByteBuffer(queryEmbedding))
                    .returnField(ByteBuffer.wrap("content".getBytes()))
                    .returnField(ByteBuffer.wrap("vector_distance".getBytes()))
                    .sortBy(SortByArgs.<ByteBuffer> builder().attribute(ByteBuffer.wrap("vector_distance".getBytes())).build())
                    .build();

            Mono<SearchReply<ByteBuffer, ByteBuffer>> hashQuery = binReactiveCommands
                    .ftSearch(ByteBuffer.wrap("vector_idx".getBytes()),
                            ByteBuffer.wrap("*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes()), searchArgs)
                    .doOnNext(result -> {
                        List<SearchReply.SearchResult<ByteBuffer, ByteBuffer>> results = result.getResults();

                        results.forEach(r -> {
                            String id = StandardCharsets.UTF_8.decode(r.getId()).toString();
                            String content = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("content".getBytes()))).toString();
                            String distance = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("vector_distance".getBytes()))).toString();

                            System.out.println("ID: " + id + ", Content: " + content + ", Distance: " + distance);
                        });
                        // >>> ID: doc:1, Content: That is a very happy person, Distance: 0.114169836044
                        // >>> ID: doc:2, Content: That is a happy dog, Distance: 0.610845506191
                        // >>> ID: doc:3, Content: Today is a sunny day, Distance: 1.48624765873
                    });
            hashQuery.block();

            List<FieldArgs<String>> jsonSchema = Arrays.asList(
                    TextFieldArgs.<String> builder().name("$.content").as("content").build(),
                    TagFieldArgs.<String> builder().name("$.genre").as("genre").build(),
                    VectorFieldArgs.<String> builder().name("$.embedding").as("embedding").hnsw()
                            .type(VectorFieldArgs.VectorType.FLOAT32).dimensions(384)
                            .distanceMetric(VectorFieldArgs.DistanceMetric.L2).build());

            CreateArgs<String, String> jsonCreateArgs = CreateArgs.<String, String> builder().on(CreateArgs.TargetType.JSON)
                    .withPrefix("jdoc:").build();

            Mono<Void> jsonCreateIndex = reactiveCommands.ftCreate("vector_json_idx", jsonCreateArgs, jsonSchema)
                    .doOnNext(result -> {
                        System.out.println(result); // >>> OK
                    }).then();
            jsonCreateIndex.block();

            JsonParser parser = reactiveCommands.getJsonParser();

            String jSentence1 = "\"That is a very happy person\"";
            JsonObject jDoc1 = parser.createJsonObject();
            jDoc1.put("content", parser.createJsonValue(jSentence1));
            jDoc1.put("genre", parser.createJsonValue("\"persons\""));

            try {
                jDoc1.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence1))));
            } catch (Exception e) {
                // ...
            }

            Mono<Void> jsonAddDoc1 = reactiveCommands.jsonSet("jdoc:1", JsonPath.ROOT_PATH, jDoc1).doOnNext(result -> {
                System.out.println(result); // >>> OK
            }).then();

            String jSentence2 = "\"That is a happy dog\"";
            JsonObject jDoc2 = parser.createJsonObject();
            jDoc2.put("content", parser.createJsonValue(jSentence2));
            jDoc2.put("genre", parser.createJsonValue("\"pets\""));

            try {
                jDoc2.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence2))));
            } catch (Exception e) {
                // ...
            }

            Mono<Void> jsonAddDoc2 = reactiveCommands.jsonSet("jdoc:2", JsonPath.ROOT_PATH, jDoc2).doOnNext(result -> {
                System.out.println(result); // >>> OK
            }).then();

            String jSentence3 = "\"Today is a sunny day\"";
            JsonObject jDoc3 = parser.createJsonObject();
            jDoc3.put("content", parser.createJsonValue(jSentence3));
            jDoc3.put("genre", parser.createJsonValue("\"weather\""));

            try {
                jDoc3.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence3))));
            } catch (Exception e) {
                // ...
            }

            Mono<Void> jsonAddDoc3 = reactiveCommands.jsonSet("jdoc:3", JsonPath.ROOT_PATH, jDoc3).doOnNext(result -> {
                System.out.println(result); // >>> OK
            }).then();
            Mono.when(jsonAddDoc1, jsonAddDoc2, jsonAddDoc3).block();

            String jQuery = "That is a happy person";
            float[] jsonQueryEmbedding = null;

            try {
                jsonQueryEmbedding = predictor.predict(jQuery);
            } catch (Exception e) {
                // ...
            }

            SearchArgs<ByteBuffer, ByteBuffer> jsonSearchArgs = SearchArgs.<ByteBuffer, ByteBuffer> builder()
                    .param(ByteBuffer.wrap("vec".getBytes()), floatArrayToByteBuffer(jsonQueryEmbedding))
                    .returnField(ByteBuffer.wrap("content".getBytes()))
                    .returnField(ByteBuffer.wrap("vector_distance".getBytes()))
                    .sortBy(SortByArgs.<ByteBuffer> builder().attribute(ByteBuffer.wrap("vector_distance".getBytes())).build())
                    .build();

            Mono<SearchReply<ByteBuffer, ByteBuffer>> jsonQuery = binReactiveCommands
                    .ftSearch(ByteBuffer.wrap("vector_json_idx".getBytes()),
                            ByteBuffer.wrap("*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes()), jsonSearchArgs)
                    .doOnNext(result -> {
                        List<SearchReply.SearchResult<ByteBuffer, ByteBuffer>> results = result.getResults();

                        results.forEach(r -> {
                            String id = StandardCharsets.UTF_8.decode(r.getId()).toString();
                            String content = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("content".getBytes()))).toString();
                            String distance = StandardCharsets.UTF_8
                                    .decode(r.getFields().get(ByteBuffer.wrap("vector_distance".getBytes()))).toString();

                            System.out.println("ID: " + id + ", Content: " + content + ", Distance: " + distance);
                        });
                        // >>> ID: jdoc:1, Content: "That is a very happy person", Distance: 0.628328084946
                        // >>> ID: jdoc:2, Content: "That is a happy dog", Distance: 0.895147025585
                        // >>> ID: jdoc:3, Content: "Today is a sunny day", Distance: 1.49569523335
                    });
            jsonQuery.block();
        } finally {
            redisClient.shutdown();
        }
    }

}

code:
let pipe = await transformers.pipeline(
    'feature-extraction', 'Xenova/all-distilroberta-v1'
);

const pipeOptions = {
    pooling: 'mean',
    normalize: true,
};

code:
from sentence_transformers import SentenceTransformer
from redis.commands.search.query import Query
from redis.commands.search.field import TextField, TagField, VectorField
from redis.commands.search.indexDefinition import IndexDefinition, IndexType
from redis.commands.json.path import Path

import numpy as np
import redis

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

r = redis.Redis(decode_responses=True)

try:
    r.ft("vector_idx").dropindex(True)
except redis.exceptions.ResponseError:
    pass

schema = (
    TextField("content"),
    TagField("genre"),
    VectorField("embedding", "HNSW", {
        "TYPE": "FLOAT32",
        "DIM": 384,
        "DISTANCE_METRIC":"L2"
    })
)

r.ft("vector_idx").create_index(
    schema,
    definition=IndexDefinition(
        prefix=["doc:"], index_type=IndexType.HASH
    )
)

content = "That is a very happy person"

r.hset("doc:0", mapping={
    "content": content,
    "genre": "persons",
    "embedding": model.encode(content).astype(np.float32).tobytes(),
})

content = "That is a happy dog"

r.hset("doc:1", mapping={
    "content": content,
    "genre": "pets",
    "embedding": model.encode(content).astype(np.float32).tobytes(),
})

content = "Today is a sunny day"

r.hset("doc:2", mapping={
    "content": content,
    "genre": "weather",
    "embedding": model.encode(content).astype(np.float32).tobytes(),
})

q = Query(
    "*=>[KNN 3 @embedding $vec AS vector_distance]"
).return_field("vector_distance").return_field("content").dialect(2)

query_text = "That is a happy person"

res = r.ft("vector_idx").search(
    q, query_params={
        "vec": model.encode(query_text).astype(np.float32).tobytes()
    }
)

print(res)

try:
    r.ft("vector_json_idx").dropindex(True)
except redis.exceptions.ResponseError:
    pass

schema = (
    TextField("$.content", as_name="content"),
    TagField("$.genre", as_name="genre"),
    VectorField(
        "$.embedding", "HNSW", {
            "TYPE": "FLOAT32",
            "DIM": 384,
            "DISTANCE_METRIC": "L2"
        },
        as_name="embedding"
    )
)

r.ft("vector_json_idx").create_index(
    schema,
    definition=IndexDefinition(
        prefix=["jdoc:"], index_type=IndexType.JSON
    )
)

content = "That is a very happy person"

r.json().set("jdoc:0", Path.root_path(), {
    "content": content,
    "genre": "persons",
    "embedding": model.encode(content).astype(np.float32).tolist(),
})

content = "That is a happy dog"

r.json().set("jdoc:1", Path.root_path(), {
    "content": content,
    "genre": "pets",
    "embedding": model.encode(content).astype(np.float32).tolist(),
})

content = "Today is a sunny day"

r.json().set("jdoc:2", Path.root_path(), {
    "content": content,
    "genre": "weather",
    "embedding": model.encode(content).astype(np.float32).tolist(),
})

q = Query(
    "*=>[KNN 3 @embedding $vec AS vector_distance]"
).return_field("vector_distance").return_field("content").dialect(2)

query_text = "That is a happy person"

res = r.ft("vector_json_idx").search(
    q, query_params={
        "vec": model.encode(query_text).astype(np.float32).tobytes()
    }
)

print(repr(res))

section_id: create-the-index

code:
const client = createClient({url: 'redis://localhost:6379'});
await client.connect();

try {
    await client.ft.dropIndex('vector_idx');
} catch (e) {
    // Index doesn't exist, which is fine
}

code:
await client.ft.create('vector_idx', {
    'content': {
        type: SchemaFieldTypes.TEXT,
    },
    'genre': {
        type: SchemaFieldTypes.TAG,
    },
    'embedding': {
        type: SchemaFieldTypes.VECTOR,
        TYPE: 'FLOAT32',
        ALGORITHM: VectorAlgorithms.HNSW,
        DISTANCE_METRIC: 'L2',
        DIM: 768,
    }
}, {
    ON: 'HASH',
    PREFIX: 'doc:'
});

section_id: add-data

code:
const sentence1 = 'That is a very happy person';
const doc1 = {
    'content': sentence1,
    'genre': 'persons',
    'embedding': Buffer.from(
        (await pipe(sentence1, pipeOptions)).data.buffer
    ),
};

const sentence2 = 'That is a happy dog';
const doc2 = {
    'content': sentence2,
    'genre': 'pets',
    'embedding': Buffer.from(
        (await pipe(sentence2, pipeOptions)).data.buffer
    )
};

const sentence3 = 'Today is a sunny day';
const doc3 = {
    'content': sentence3,
    'genre': 'weather',
    'embedding': Buffer.from(
        (await pipe(sentence3, pipeOptions)).data.buffer
    )
};

await Promise.all([
    client.hSet('doc:1', doc1),
    client.hSet('doc:2', doc2),
    client.hSet('doc:3', doc3)
]);

section_id: run-a-query

code:
const similar = await client.ft.search(
    'vector_idx',
    '*=>[KNN 3 @embedding $B AS score]',
    {
        'PARAMS': {
            B: Buffer.from(
                (await pipe('That is a happy person', pipeOptions)).data.buffer
            ),
        },
        'RETURN': ['score', 'content'],
        'DIALECT': '2'
    },
);

for (const doc of similar.documents) {
    console.log(`${doc.id}: '${doc.value.content}', Score: ${doc.value.score}`);
}

code:
doc:1: 'That is a very happy person', Score: 0.127055495977
doc:2: 'That is a happy dog', Score: 0.836842417717
doc:3: 'Today is a sunny day', Score: 1.50889515877

section_id: differences-with-json-documents

code:
await client.ft.create('vector_json_idx', {
    '$.content': {
        type: SchemaFieldTypes.TEXT,
        AS: 'content',
    },
    '$.genre': {
        type: SchemaFieldTypes.TAG,
        AS: 'genre',
    },
    '$.embedding': {
        type: SchemaFieldTypes.VECTOR,
        TYPE: 'FLOAT32',
        ALGORITHM: VectorAlgorithms.HNSW,
        DISTANCE_METRIC: 'L2',
        DIM: 768,
        AS: 'embedding',
    }
}, {
    ON: 'JSON',
    PREFIX: 'jdoc:'
});

code:
const jSentence1 = 'That is a very happy person';
const jdoc1 = {
    'content': jSentence1,
    'genre': 'persons',
    'embedding': [...(await pipe(jSentence1, pipeOptions)).data],
};

const jSentence2 = 'That is a happy dog';
const jdoc2 = {
    'content': jSentence2,
    'genre': 'pets',
    'embedding': [...(await pipe(jSentence2, pipeOptions)).data],
};

const jSentence3 = 'Today is a sunny day';
const jdoc3 = {
    'content': jSentence3,
    'genre': 'weather',
    'embedding': [...(await pipe(jSentence3, pipeOptions)).data],
};

await Promise.all([
    client.json.set('jdoc:1', '$', jdoc1),
    client.json.set('jdoc:2', '$', jdoc2),
    client.json.set('jdoc:3', '$', jdoc3)
]);

code:
const jsons = await client.ft.search(
    'vector_json_idx',
    '*=>[KNN 3 @embedding $B AS score]',
    {
        "PARAMS": {
            B: Buffer.from(
                (await pipe('That is a happy person', pipeOptions)).data.buffer
            ),
        },
        'RETURN': ['score', 'content'],
        'DIALECT': '2'
    },
);

code:
jdoc:1: 'That is a very happy person', Score: 0.127055495977
jdoc:2: 'That is a happy dog', Score: 0.836842417717
jdoc:3: 'Today is a sunny day', Score: 1.50889515877


title: Vector set embeddings

summary: Index and query embeddings with Redis vector sets

page_type: content

title: Initialize

role: content

text:
Start by [installing]() `node-redis`
if you haven't already done so. Also, install `@xenova/transformers`:

[code example]

In your JavaScript source file, import the required classes:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

The first of these imports is the
`@xenova/transformers` class, which generates an embedding from a section of text.
This example uses `transformers.pipeline` with the
[`all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
model for the embeddings. This model generates vectors with 384 dimensions, regardless
of the length of the input text, but note that the input is truncated to 256
tokens (see
[Word piece tokenization](https://huggingface.co/learn/nlp-course/en/chapter6/6)
at the [Hugging Face](https://huggingface.co/) docs to learn more about the way tokens
are related to the original text).

The output from `transformers.pipeline` is a function (called `pipe` in the examples)
that you can call to generate embeddings. The `pipeOptions` object is a parameter for
`pipe` that specifies how to generate sentence embeddings from token embeddings (see the
[`all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
documentation for details).

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

title: Create the data

text:
The example data is contained in an object with some brief
descriptions of famous people:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

title: Add the data to a vector set

text:
The next step is to connect to Redis and add the data to a new vector set.

The code below iterates through all the key-value pairs in the `peopleData` object
and adds corresponding elements to a vector set called `famousPeople`.

Use the `pipe()` function created above to generate the
embedding and then use `Array.from()` to convert the embedding to an array
of `float32` values that you can pass to the
[`vAdd()`]() command to set the embedding.

The call to `vAdd()` also adds the `born` and `died` values from the
`peopleData` object as attribute data. You can access this during a query
or by using the [`vGetAttr()`]() method.

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

title: Query the vector set

text:
You can now query the data in the set. The basic approach is to use the
`pipe()` function to generate another embedding vector for the query text.
(This is the same method used to add the elements to the set.) Then, pass
the query vector to [`vSim()`]() to return elements
of the set, ranked in order of similarity to the query.

Start with a simple query for "actors":

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

This returns the following list of elements (formatted slightly for clarity):

[code example]

The first two people in the list are the two actors, as expected, but none of the
people from Linus Pauling onward was especially well-known for acting (and there certainly
isn't any information about that in the short description text).
As it stands, the search attempts to rank all the elements in the set, based
on the information contained in the embedding model.
You can use the `COUNT` parameter of `vSim()` to limit the list of elements
to just the most relevant few items:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

The reason for using text embeddings rather than simple text search
is that the embeddings represent semantic information. This allows a query
to find elements with a similar meaning even if the text is
different. For example, the word "entertainer" doesn't appear in any of the
descriptions but if you use it as a query, the actors and musicians are ranked
highest in the results list:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Similarly, if you use "science" as a query, you get the following results:

[code example]

The scientists are ranked highest but they are then followed by the
mathematicians. This seems reasonable given the connection between mathematics
and science.

You can also use
[filter expressions]()
with `vSim()` to restrict the search further. For example,
repeat the "science" query, but this time limit the results to people
who died before the year 2000:

#### Code Examples

**Redis CLI:**

[code example]

**Available in:** Redis CLI, C# (Synchronous), Go, Java (Asynchronous - Lettuce), Java (Reactive - Lettuce), Java (Synchronous - Jedis), JavaScript (Node.js), Python

**C# (Synchronous):**

[code example]

**Go:**

[code example]

**Java (Asynchronous - Lettuce):**

[code example]

**Java (Reactive - Lettuce):**

[code example]

**Java (Synchronous - Jedis):**

[code example]

**JavaScript (Node.js):**

[code example]

**Python:**

[code example]

Note that the boolean filter expression is applied to items in the list
before the vector distance calculation is performed. Items that don't
pass the filter test are removed from the results completely, rather
than just reduced in rank. This can help to improve the performance of the
search because there is no need to calculate the vector distance for
elements that have already been filtered out of the search.

title: More information

text:
See the [vector sets]()
docs for more information and code examples. See the
[Redis for AI]() section for more details
about text embeddings and other AI techniques you can use with Redis.

You may also be interested in
[vector search]().
This is a feature of
[Redis Search]()
that lets you retrieve
[JSON]() and
[hash]() documents based on
vector data stored in their fields.

section_id: initialize

code:
// Suppress experimental API warnings for VectorSet
#pragma warning disable SER001

using StackExchange.Redis;

using Microsoft.ML;
using Microsoft.ML.Transforms.Text;

code:
package main

import (
	"context"
	"fmt"
	"strings"

	"github.com/knights-analytics/hugot"
	"github.com/redis/go-redis/v9"
)

code:
// Lettuce client and Redis Search classes.
import io.lettuce.core.*;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.api.async.RedisAsyncCommands;

// Standard library classes for data manipulation and
// asynchronous programming.
import java.util.*;
import java.util.concurrent.CompletableFuture;

// DJL classes for model loading and inference.
import ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory;
import ai.djl.inference.Predictor;
import ai.djl.repository.zoo.Criteria;
import ai.djl.training.util.ProgressBar;

code:
// Lettuce client and Redis Search classes.
import io.lettuce.core.*;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.api.reactive.RedisReactiveCommands;

// Standard library classes for data manipulation and
// reactive programming.
import java.util.*;
import reactor.core.publisher.Mono;

// DJL classes for model loading and inference.
import ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory;
import ai.djl.inference.Predictor;
import ai.djl.repository.zoo.Criteria;
import ai.djl.training.util.ProgressBar;

code:
import redis.clients.jedis.RedisClient;
import redis.clients.jedis.params.VAddParams;
import redis.clients.jedis.params.VSimParams;

import java.util.*;

// DJL classes for model loading and inference.
import ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory;
import ai.djl.inference.Predictor;
import ai.djl.repository.zoo.Criteria;
import ai.djl.training.util.ProgressBar;

code:
import * as transformers from '@xenova/transformers';
import { createClient } from 'redis';

code:
from sentence_transformers import SentenceTransformer

import redis
import numpy as np

code:
static PredictionEngine<TextData, TransformedTextData> GetPredictionEngine()
{
    // Create a new ML context, for ML.NET operations. It can be used for
    // exception tracking and logging, as well as the source of randomness.
    var mlContext = new MLContext();

    // Create an empty list as the dataset
    var emptySamples = new List<TextData>();

    // Convert sample list to an empty IDataView.
    var emptyDataView = mlContext.Data.LoadFromEnumerable(emptySamples);

    // A pipeline for converting text into a 150-dimension embedding vector
    var textPipeline = mlContext.Transforms.Text.NormalizeText("Text")
        .Append(mlContext.Transforms.Text.TokenizeIntoWords("Tokens",
            "Text"))
        .Append(mlContext.Transforms.Text.ApplyWordEmbedding("Features",
            "Tokens", WordEmbeddingEstimator.PretrainedModelKind
            .SentimentSpecificWordEmbedding));

    // Fit to data.
    var textTransformer = textPipeline.Fit(emptyDataView);

    // Create the prediction engine to get the embedding vector from the input text/string.
    var predictionEngine = mlContext.Model.CreatePredictionEngine<TextData,
        TransformedTextData>(textTransformer);

    return predictionEngine;
}

code:
// Create a new Hugot session
	session, err := hugot.NewGoSession()
	if err != nil {
		panic(err)
	}
	defer func() {
		err := session.Destroy()
		if err != nil {
			panic(err)
		}
	}()

	// Download the model.
	downloadOptions := hugot.NewDownloadOptions()
	downloadOptions.OnnxFilePath = "onnx/model.onnx"
	modelPath, err := hugot.DownloadModel(
		"sentence-transformers/all-MiniLM-L6-v2",
		"./models/",
		downloadOptions,
	)
	if err != nil {
		panic(err)
	}

	// Create feature extraction pipeline configuration
	config := hugot.FeatureExtractionConfig{
		ModelPath: modelPath,
		Name:      "embeddingPipeline",
	}

	// Create the feature extraction pipeline
	embeddingPipeline, err := hugot.NewPipeline(session, config)
	if err != nil {
		panic(err)
	}

code:
Predictor<String, float[]> predictor = null;

        try {
            Criteria<String, float[]> criteria = Criteria.builder().setTypes(String.class, float[].class)
                    .optModelUrls("djl://ai.djl.huggingface.pytorch/sentence-transformers/all-MiniLM-L6-v2")
                    .optEngine("PyTorch").optTranslatorFactory(new TextEmbeddingTranslatorFactory())
                    .optProgress(new ProgressBar()).build();

            predictor = criteria.loadModel().newPredictor();
        } catch (Exception e) {
            // ...
        }

code:
const pipe = await transformers.pipeline(
    'feature-extraction', 'Xenova/all-MiniLM-L6-v2'
);

const pipeOptions = {
    pooling: 'mean',
    normalize: true,
};

code:
Dictionary<string, dynamic> peopleData = new Dictionary<string, dynamic>
{
    ["Marie Curie"] = new
    {
        born = 1867,
        died = 1934,
        description = @"
        Polish-French chemist and physicist. The only person ever to win
        two Nobel prizes for two different sciences.
        "
    },
    ["Linus Pauling"] = new
    {
        born = 1901,
        died = 1994,
        description = @"
        American chemist and peace activist. One of only two people to win two
        Nobel prizes in different fields (chemistry and peace).
        "
    },
    ["Freddie Mercury"] = new
    {
        born = 1946,
        died = 1991,
        description = @"
        British musician, best known as the lead singer of the rock band
        Queen.
        "
    },
    ["Marie Fredriksson"] = new
    {
        born = 1958,
        died = 2019,
        description = @"
        Swedish multi-instrumentalist, mainly known as the lead singer and
        keyboardist of the band Roxette.
        "
    },
    ["Paul Erdos"] = new
    {
        born = 1913,
        died = 1996,
        description = @"
        Hungarian mathematician, known for his eccentric personality almost
        as much as his contributions to many different fields of mathematics.
        "
    },
    ["Maryam Mirzakhani"] = new
    {
        born = 1977,
        died = 2017,
        description = @"
        Iranian mathematician. The first woman ever to win the Fields medal
        for her contributions to mathematics.
        "
    },
    ["Masako Natsume"] = new
    {
        born = 1957,
        died = 1985,
        description = @"
        Japanese actress. She was very famous in Japan but was primarily
        known elsewhere in the world for her portrayal of Tripitaka in the
        TV series Monkey.
        "
    },
    ["Chaim Topol"] = new
    {
        born = 1935,
        died = 2023,
        description = @"
        Israeli actor and singer, usually credited simply as 'Topol'. He was
        best known for his many appearances as Tevye in the musical Fiddler
        on the Roof.
        "
    }
};

section_id: create-the-data

code:
type PersonData struct {
	Born        int
	Died        int
	Description string
}

var peopleData = map[string]PersonData{
	"Marie Curie": {
		Born: 1867, Died: 1934,
		Description: `Polish-French chemist and physicist. The only person ever to win
		two Nobel prizes for two different sciences.
		`,
	},
	"Linus Pauling": {
		Born: 1901, Died: 1994,
		Description: `American chemist and peace activist. One of only two people to win two
		Nobel prizes in different fields (chemistry and peace).
		`,
	},
	"Freddie Mercury": {
		Born: 1946, Died: 1991,
		Description: `British musician, best known as the lead singer of the rock band
		Queen.
		`,
	},
	"Marie Fredriksson": {
		Born: 1958, Died: 2019,
		Description: `Swedish multi-instrumentalist, mainly known as the lead singer and
		keyboardist of the band Roxette.
		`,
	},
	"Paul Erdos": {
		Born: 1913, Died: 1996,
		Description: `Hungarian mathematician, known for his eccentric personality almost
		as much as his contributions to many different fields of mathematics.
		`,
	},
	"Maryam Mirzakhani": {
		Born: 1977, Died: 2017,
		Description: `Iranian mathematician. The first woman ever to win the Fields medal
		for her contributions to mathematics.
		`,
	},
	"Masako Natsume": {
		Born: 1957, Died: 1985,
		Description: `Japanese actress. She was very famous in Japan but was primarily
		known elsewhere in the world for her portrayal of Tripitaka in the
		TV series Monkey.
		`,
	},
	"Chaim Topol": {
		Born: 1935, Died: 2023,
		Description: `Israeli actor and singer, usually credited simply as 'Topol'. He was
		best known for his many appearances as Tevye in the musical Fiddler
		on the Roof.
		`,
	},
}

code:
final class Person {
            final String name;
            final int born;
            final int died;
            final String description;
            Person(String name, int born, int died, String description) {
                this.name = name; this.born = born; this.died = died; this.description = description;
            }
        }

        List<Person> people = Arrays.asList(
            new Person(
                "Marie Curie",
                1867, 1934,
                "Polish-French chemist and physicist. The only person ever to win" +
                " two Nobel prizes for two different sciences."
            ),
            new Person(
                "Linus Pauling",
                1901, 1994,
                "American chemist and peace activist. One of only two people to" +
                " win two Nobel prizes in different fields (chemistry and peace)."
            ),
            new Person(
                "Freddie Mercury",
                1946, 1991,
                "British musician, best known as the lead singer of the rock band Queen."
            ),
            new Person(
                "Marie Fredriksson",
                1958, 2019,
                "Swedish multi-instrumentalist, mainly known as the lead singer and" +
                " keyboardist of the band Roxette."
            ),
            new Person(
                "Paul Erdos",
                1913, 1996,
                "Hungarian mathematician, known for his eccentric personality almost" +
                " as much as his contributions to many different fields of mathematics."
            ),
            new Person(
                "Maryam Mirzakhani",
                1977, 2017,
                "Iranian mathematician. The first woman ever to win the Fields medal" +
                " for her contributions to mathematics."
            ),
            new Person(
                "Masako Natsume",
                1957, 1985,
                "Japanese actress. She was very famous in Japan but was primarily" +
                " known elsewhere in the world for her portrayal of Tripitaka in the" +
                " TV series Monkey."
            ),
            new Person(
                "Chaim Topol",
                1935, 2023,
                "Israeli actor and singer, usually credited simply as 'Topol'. He was" +
                " best known for his many appearances as Tevye in the musical Fiddler" +
                " on the Roof."
            )
        );

code:
const peopleData = {
    "Marie Curie": {
        "born": 1867, "died": 1934,
        "description": `
        Polish-French chemist and physicist. The only person ever to win
        two Nobel prizes for two different sciences.
        `
    },
    "Linus Pauling": {
        "born": 1901, "died": 1994,
        "description": `
        American chemist and peace activist. One of only two people to win two
        Nobel prizes in different fields (chemistry and peace).
        `
    },
    "Freddie Mercury": {
        "born": 1946, "died": 1991,
        "description": `
        British musician, best known as the lead singer of the rock band
        Queen.
        `
    },
    "Marie Fredriksson": {
        "born": 1958, "died": 2019,
        "description": `
        Swedish multi-instrumentalist, mainly known as the lead singer and
        keyboardist of the band Roxette.
        `
    },
    "Paul Erdos": {
        "born": 1913, "died": 1996,
        "description": `
        Hungarian mathematician, known for his eccentric personality almost
        as much as his contributions to many different fields of mathematics.
        `
    },
    "Maryam Mirzakhani": {
        "born": 1977, "died": 2017,
        "description": `
        Iranian mathematician. The first woman ever to win the Fields medal
        for her contributions to mathematics.
        `
    },
    "Masako Natsume": {
        "born": 1957, "died": 1985,
        "description": `
        Japanese actress. She was very famous in Japan but was primarily
        known elsewhere in the world for her portrayal of Tripitaka in the
        TV series Monkey.
        `
    },
    "Chaim Topol": {
        "born": 1935, "died": 2023,
        "description": `
        Israeli actor and singer, usually credited simply as 'Topol'. He was
        best known for his many appearances as Tevye in the musical Fiddler
        on the Roof.
        `
    }
};

code:
peopleData = {
    "Marie Curie": {
        "born": 1867, "died": 1934,
        "description": """
        Polish-French chemist and physicist. The only person ever to win
        two Nobel prizes for two different sciences.
        """
    },
    "Linus Pauling": {
        "born": 1901, "died": 1994,
        "description": """
        American chemist and peace activist. One of only two people to win two
        Nobel prizes in different fields (chemistry and peace).
        """
    },
    "Freddie Mercury": {
        "born": 1946, "died": 1991,
        "description": """
        British musician, best known as the lead singer of the rock band
        Queen.
        """
    },
    "Marie Fredriksson": {
        "born": 1958, "died": 2019,
        "description": """
        Swedish multi-instrumentalist, mainly known as the lead singer and
        keyboardist of the band Roxette.
        """
    },
    "Paul Erdos": {
        "born": 1913, "died": 1996,
        "description": """
        Hungarian mathematician, known for his eccentric personality almost
        as much as his contributions to many different fields of mathematics.
        """
    },
    "Maryam Mirzakhani": {
        "born": 1977, "died": 2017,
        "description": """
        Iranian mathematician. The first woman ever to win the Fields medal
        for her contributions to mathematics.
        """
    },
    "Masako Natsume": {
        "born": 1957, "died": 1985,
        "description": """
        Japanese actress. She was very famous in Japan but was primarily
        known elsewhere in the world for her portrayal of Tripitaka in the
        TV series Monkey.
        """
    },
    "Chaim Topol": {
        "born": 1935, "died": 2023,
        "description": """
        Israeli actor and singer, usually credited simply as 'Topol'. He was
        best known for his many appearances as Tevye in the musical Fiddler
        on the Roof.
        """
    }
}

code:
ConnectionMultiplexer muxer = ConnectionMultiplexer.Connect("localhost:6379");
IDatabase db = muxer.GetDatabase();

PredictionEngine<TextData, TransformedTextData> model = GetPredictionEngine();

foreach (KeyValuePair<string, dynamic> person in peopleData)
{
    string name = person.Key;
    dynamic details = person.Value;
    float[] embedding = GetEmbedding(model, details.description);

    VectorSetAddRequest addRequest = VectorSetAddRequest.Member(name, embedding, null);
    db.VectorSetAdd("famousPeople", addRequest);

    // Set attributes separately
    string attributesJson = $"{{\"born\": {details.born}, \"died\": {details.died}}}";
    db.VectorSetSetAttributesJson("famousPeople", name, attributesJson);
}

section_id: add-the-data-to-a-vector-set

code:
for name, details := range peopleData {
		// Generate embeddings using Hugot
		result, err := embeddingPipeline.RunPipeline([]string{details.Description})
		if err != nil {
			panic(err)
		}

		// Convert embedding to float64 slice
		embFloat32 := result.Embeddings[0]
		embFloat64 := make([]float64, len(embFloat32))
		for i, v := range embFloat32 {
			embFloat64[i] = float64(v)
		}

		// Add vector to vector set
		_, err = rdb.VAdd(ctx, "famousPeople", name, &redis.VectorValues{Val: embFloat64}).Result()
		if err != nil {
			panic(err)
		}

		// Set attributes for the element
		_, err = rdb.VSetAttr(ctx, "famousPeople", name, map[string]interface{}{
			"born": details.Born,
			"died": details.Died,
		}).Result()
		if err != nil {
			panic(err)
		}
	}

code:
RedisClient redisClient = RedisClient.create("redis://localhost:6379");

        try (StatefulRedisConnection<String, String> connection = redisClient.connect()) {
            RedisAsyncCommands<String, String> asyncCommands = connection.async();

            CompletableFuture<?>[] vaddFutures = new CompletableFuture[people.size()];

            for (int i = 0; i < people.size(); i++) {
                Person person = people.get(i);
                Double[] embedding = null;

                try {
                    embedding = convertFloatsToDoubles(predictor.predict(person.description));
                } catch (Exception e) {
                    // ...
                }

                VAddArgs personArgs = VAddArgs.Builder.attributes(String.format("{\"born\": %d, \"died\": %d}", person.born, person.died));

                vaddFutures[i] = asyncCommands.vadd("famousPeople", person.name, personArgs, embedding)
                        .thenApply(result -> {
                            System.out.println(result); // >>> true
                            return result;
                        }).toCompletableFuture();
            }

            CompletableFuture.allOf(vaddFutures).join();

code:
RedisClient redisClient = RedisClient.create("redis://localhost:6379");

        try (StatefulRedisConnection<String, String> connection = redisClient.connect()) {
            RedisReactiveCommands<String, String> reactiveCommands = connection.reactive();

            Mono<?>[] vaddFutures = new Mono[people.size()];

            for (int i = 0; i < people.size(); i++) {
                Person person = people.get(i);
                Double[] embedding = null;

                try {
                    embedding = convertFloatsToDoubles(predictor.predict(person.description));
                } catch (Exception e) {
                    // ...
                }

                VAddArgs personArgs = VAddArgs.Builder.attributes(String.format("{\"born\": %d, \"died\": %d}", person.born, person.died));

                vaddFutures[i] = reactiveCommands.vadd("famousPeople", person.name, personArgs, embedding)
                        .doOnNext(result -> {
                            System.out.println(result); // >>> true
                        });
            }

            Mono.when(vaddFutures).block();

code:
RedisClient jedis = new RedisClient("redis://localhost:6379");

        for (Person person : people) {
            float[] embedding;
            try {
                embedding = predictor.predict(person.description);
            } catch (Exception e) {
                // This just allows the code to compile without errors.
                // In a real-world scenario, you would handle the exception properly.
                embedding = new float[384];
            }

            // Add element with attributes using VAddParams
            String attrs = String.format("{\"born\": %d, \"died\": %d}", person.born, person.died);
            boolean added = jedis.vadd("famousPeople", embedding, person.name, new VAddParams().setAttr(attrs));
            System.out.println(added); // >>> true
        }

code:
const client = createClient({ url: 'redis://localhost:6379' });

client.on('error', err => console.log('Redis Client Error', err));
await client.connect();

for (const [name, details] of Object.entries(peopleData)) {
    const embedding = await pipe(details.description, pipeOptions);
    const embeddingArray = Array.from(embedding.data);

    await client.vAdd('famousPeople', embeddingArray, name);
    await client.vSetAttr('famousPeople', name, JSON.stringify({
        born: details.born,
        died: details.died
    }));
}

code:
r = redis.Redis(decode_responses=True)

for name, details in peopleData.items():
    emb = model.encode(details["description"]).astype(np.float32).tobytes()

    r.vset().vadd(
        "famousPeople",
        emb,
        name,
        attributes={
            "born": details["born"],
            "died": details["died"]
        }
    )

code:
string queryValue = "actors";

float[] queryEmbedding = GetEmbedding(model, queryValue);

VectorSetSimilaritySearchRequest basicQuery = VectorSetSimilaritySearchRequest.ByVector(queryEmbedding);
using (Lease<VectorSetSimilaritySearchResult>? actorsResults = db.VectorSetSimilaritySearch("famousPeople", basicQuery))
{
    IEnumerable<string> resultIds = actorsResults!.Span.ToArray()
        .Select(r => (string?)r.Member)
        .Where(id => id != null)
        .Select(id => $"'{id!}'");
    Console.WriteLine($"'actors': [{string.Join(", ", resultIds)}]");
}

section_id: query-the-vector-set

code:
queryValue := "actors"

	queryResult, err := embeddingPipeline.RunPipeline([]string{queryValue})
	if err != nil {
		panic(err)
	}

	// Convert embedding to float64 slice
	queryFloat32 := queryResult.Embeddings[0]
	queryFloat64 := make([]float64, len(queryFloat32))
	for i, v := range queryFloat32 {
		queryFloat64[i] = float64(v)
	}

	actorsResults, err := rdb.VSim(ctx, "famousPeople", &redis.VectorValues{Val: queryFloat64}).Result()
	if err != nil {
		panic(err)
	}

	fmt.Printf("'actors': %v\n", strings.Join(actorsResults, ", "))
	// >>> 'actors': Masako Natsume, Chaim Topol, Linus Pauling,
	// Marie Fredriksson, Maryam Mirzakhani, Marie Curie, Freddie Mercury,
	// Paul Erdos

code:
Double[] actorsEmbedding = null;

            try {
                actorsEmbedding = convertFloatsToDoubles(predictor.predict("actors"));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<List<String>> basicQuery = asyncCommands.vsim("famousPeople", actorsEmbedding)
                    .thenApply(result -> {
                        System.out.println(result);
                        // >>> [Masako Natsume, Chaim Topol, Linus Pauling, Marie Fredriksson, Maryam Mirzakhani, Marie Curie, Freddie Mercury, Paul Erdos]
                        return result;
                    }).toCompletableFuture();

code:
Double[] actorsEmbedding = null;

            try {
                actorsEmbedding = convertFloatsToDoubles(predictor.predict("actors"));
            } catch (Exception e) {
                // ...
            }

            Mono<java.util.List<String>> basicQuery = reactiveCommands.vsim("famousPeople", actorsEmbedding)
                    .collectList().doOnNext(result -> {
                        System.out.println(result);
                        // >>> [Masako Natsume, Chaim Topol, Linus Pauling, Marie Fredriksson, Maryam Mirzakhani, Marie Curie, Freddie Mercury, Paul Erdos]
                    });

code:
float[] actorsEmbedding;
        try {
            actorsEmbedding = predictor.predict("actors");
        } catch (Exception e) {
            actorsEmbedding = new float[384];
        }

        List<String> basicResults = jedis.vsim("famousPeople", actorsEmbedding);
        System.out.println(basicResults);
        // >>> [Masako Natsume, Chaim Topol, Linus Pauling, Marie Fredriksson,
        // >>> Maryam Mirzakhani, Marie Curie, Freddie Mercury, Paul Erdos]

code:
const queryValue = "actors";

const queryEmbedding = await pipe(queryValue, pipeOptions);
const queryArray = Array.from(queryEmbedding.data);

const actorsResults = await client.vSim('famousPeople', queryArray);

console.log(`'actors': ${JSON.stringify(actorsResults)}`);
// >>> 'actors': ["Masako Natsume","Chaim Topol","Linus Pauling",
// "Marie Fredriksson","Maryam Mirzakhani","Freddie Mercury",
// "Marie Curie","Paul Erdos"]

code:
query_value = "actors"

actors_results = r.vset().vsim(
    "famousPeople",
    model.encode(query_value).astype(np.float32).tobytes(),
)

print(f"'actors': {actors_results}")

code:
'actors': ["Masako Natsume","Chaim Topol","Linus Pauling",
    "Marie Fredriksson","Maryam Mirzakhani","Freddie Mercury",
    "Marie Curie","Paul Erdos"]

code:
queryValue = "actors";

queryEmbedding = GetEmbedding(model, queryValue);

VectorSetSimilaritySearchRequest limitedQuery = VectorSetSimilaritySearchRequest.ByVector(queryEmbedding);
limitedQuery.Count = 2;
using (Lease<VectorSetSimilaritySearchResult>? twoActorsResults = db.VectorSetSimilaritySearch("famousPeople", limitedQuery))
{
    IEnumerable<string> resultIds = twoActorsResults!.Span.ToArray()
        .Select(r => (string?)r.Member)
        .Where(id => id != null)
        .Select(id => $"'{id!}'");
    Console.WriteLine($"'actors (2)': [{string.Join(", ", resultIds)}]");
    // >>> 'actors (2)': ['Masako Natsume', 'Chaim Topol']
}

code:
queryValue = "actors"

	queryResult, err = embeddingPipeline.RunPipeline([]string{queryValue})
	if err != nil {
		panic(err)
	}

	// Convert embedding to float64 slice
	queryFloat32 = queryResult.Embeddings[0]
	queryFloat64 = make([]float64, len(queryFloat32))
	for i, v := range queryFloat32 {
		queryFloat64[i] = float64(v)
	}

	twoActorsResults, err := rdb.VSimWithArgs(ctx, "famousPeople",
		&redis.VectorValues{Val: queryFloat64},
		&redis.VSimArgs{Count: 2}).Result()
	if err != nil {
		panic(err)
	}

	fmt.Printf("'actors (2)': %v\n", strings.Join(twoActorsResults, ", "))
	// >>> 'actors (2)': Masako Natsume, Chaim Topol

code:
VSimArgs limitArgs = VSimArgs.Builder.count(2L);

            CompletableFuture<List<String>> limitedQuery = asyncCommands.vsim("famousPeople", limitArgs, actorsEmbedding)
                    .thenApply(result -> {
                        System.out.println(result);
                        // >>> [Masako Natsume, Chaim Topol]
                        return result;
                    }).toCompletableFuture();

code:
VSimArgs limitArgs = VSimArgs.Builder.count(2L);

            Mono<java.util.List<String>> limitedQuery = reactiveCommands.vsim("famousPeople", limitArgs, actorsEmbedding)
                    .collectList().doOnNext(result -> {
                        System.out.println(result);
                        // >>> [Masako Natsume, Chaim Topol]
                    });

code:
VSimParams limitParams = new VSimParams().count(2);
        List<String> limitedResults = jedis.vsim("famousPeople", actorsEmbedding, limitParams);
        System.out.println(limitedResults);
        // >>> [Masako Natsume, Chaim Topol]

code:
const queryValue2 = "actors";

const queryEmbedding2 = await pipe(queryValue2, pipeOptions);
const queryArray2 = Array.from(queryEmbedding2.data);

const twoActorsResults = await client.vSim('famousPeople', queryArray2, {
    COUNT: 2
});

console.log(`'actors (2)': ${JSON.stringify(twoActorsResults)}`);
// >>> 'actors (2)': ["Masako Natsume","Chaim Topol"]

code:
query_value = "actors"

two_actors_results = r.vset().vsim(
    "famousPeople",
    model.encode(query_value).astype(np.float32).tobytes(),
    count=2
)

print(f"'actors (2)': {two_actors_results}")
# >>> 'actors (2)': ['Masako Natsume', 'Chaim Topol']

code:
queryValue = "entertainer";

queryEmbedding = GetEmbedding(model, queryValue);

VectorSetSimilaritySearchRequest entertainerQuery = VectorSetSimilaritySearchRequest.ByVector(queryEmbedding);
using (Lease<VectorSetSimilaritySearchResult>? entertainerResults = db.VectorSetSimilaritySearch("famousPeople", entertainerQuery))
{
    IEnumerable<string> resultIds = entertainerResults!.Span.ToArray()
        .Select(r => (string?)r.Member)
        .Where(id => id != null)
        .Select(id => $"'{id!}'");
    Console.WriteLine($"'entertainer': [{string.Join(", ", resultIds)}]");
    // >>> 'entertainer': ['Chaim Topol', 'Freddie Mercury',
    // 'Marie Fredriksson', 'Masako Natsume', 'Linus Pauling',
    // 'Paul Erdos', 'Maryam Mirzakhani', 'Marie Curie']
}

code:
queryValue = "entertainer"

	queryResult, err = embeddingPipeline.RunPipeline([]string{queryValue})
	if err != nil {
		panic(err)
	}

	// Convert embedding to float64 slice
	queryFloat32 = queryResult.Embeddings[0]
	queryFloat64 = make([]float64, len(queryFloat32))
	for i, v := range queryFloat32 {
		queryFloat64[i] = float64(v)
	}

	entertainerResults, err := rdb.VSim(ctx, "famousPeople", &redis.VectorValues{Val: queryFloat64}).Result()
	if err != nil {
		panic(err)
	}

	fmt.Printf("'entertainer': %v\n", strings.Join(entertainerResults, ", "))
	// >>> 'entertainer': Chaim Topol, Freddie Mercury, Marie Fredriksson,
	// Linus Pauling, Masako Natsume, Paul Erdos,
	// Maryam Mirzakhani, Marie Curie

code:
Double[] entertainerEmbedding = null;

            try {
                entertainerEmbedding = convertFloatsToDoubles(predictor.predict("entertainers"));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<List<String>> entertainerQuery = asyncCommands.vsim("famousPeople", entertainerEmbedding)
                    .thenApply(result -> {
                        System.out.println(result);
                        // >>> [Freddie Mercury, Chaim Topol, Linus Pauling, Marie Fredriksson, Masako Natsume, Paul Erdos, Maryam Mirzakhani, Marie Curie]
                        return result;
                    }).toCompletableFuture();

code:
Double[] entertainerEmbedding = null;

            try {
                entertainerEmbedding = convertFloatsToDoubles(predictor.predict("entertainers"));
            } catch (Exception e) {
                // ...
            }

            Mono<java.util.List<String>> entertainerQuery = reactiveCommands.vsim("famousPeople", entertainerEmbedding)
                    .collectList().doOnNext(result -> {
                        System.out.println(result);
                        // >>> [Freddie Mercury, Chaim Topol, Linus Pauling, Marie Fredriksson, Masako Natsume, Paul Erdos, Maryam Mirzakhani, Marie Curie]
                    });

code:
float[] entertainerEmbedding;
        try {
            entertainerEmbedding = predictor.predict("entertainers");
        } catch (Exception e) {
            entertainerEmbedding = new float[384];
        }

        List<String> entertainerResults = jedis.vsim("famousPeople", entertainerEmbedding);
        System.out.println(entertainerResults);
        // >>> [Freddie Mercury, Chaim Topol, Linus Pauling, Marie Fredriksson,
        // >>> Masako Natsume, Paul Erdos, Maryam Mirzakhani, Marie Curie]

code:
const queryValue3 = "entertainer";

const queryEmbedding3 = await pipe(queryValue3, pipeOptions);
const queryArray3 = Array.from(queryEmbedding3.data);

const entertainerResults = await client.vSim('famousPeople', queryArray3);

console.log(`'entertainer': ${JSON.stringify(entertainerResults)}`);
// >>> 'actors': ["Masako Natsume","Chaim Topol","Linus Pauling",
// "Marie Fredriksson","Maryam Mirzakhani","Freddie Mercury",
// "Marie Curie","Paul Erdos"]

code:
query_value = "entertainer"

entertainer_results = r.vset().vsim(
    "famousPeople",
    model.encode(query_value).astype(np.float32).tobytes()
)

print(f"'entertainer': {entertainer_results}")
# >>> 'entertainer': ['Chaim Topol', 'Freddie Mercury',
# 'Marie Fredriksson', 'Masako Natsume', 'Linus Pauling',
# 'Paul Erdos', 'Maryam Mirzakhani', 'Marie Curie']

code:
'science': ["Linus Pauling","Marie Curie","Maryam Mirzakhani","Paul Erdos",
"Marie Fredriksson","Masako Natsume","Freddie Mercury","Chaim Topol"]

code:
queryValue = "science";

queryEmbedding = GetEmbedding(model, queryValue);

VectorSetSimilaritySearchRequest filteredQuery = VectorSetSimilaritySearchRequest.ByVector(queryEmbedding);
filteredQuery.FilterExpression = ".died < 2000";
using (Lease<VectorSetSimilaritySearchResult>? science2000Results = db.VectorSetSimilaritySearch("famousPeople", filteredQuery))
{
    IEnumerable<string> resultIds = science2000Results!.Span.ToArray()
        .Select(r => (string?)r.Member)
        .Where(id => id != null)
        .Select(id => $"'{id!}'");
    Console.WriteLine($"'science2000': [{string.Join(", ", resultIds)}]");
    // >>> 'science2000': ['Marie Curie', 'Linus Pauling',
    // 'Paul Erdos', 'Freddie Mercury', 'Masako Natsume']
}

code:
queryValue = "science"

	queryResult, err = embeddingPipeline.RunPipeline([]string{queryValue})
	if err != nil {
		panic(err)
	}

	// Convert embedding to float64 slice
	queryFloat32 = queryResult.Embeddings[0]
	queryFloat64 = make([]float64, len(queryFloat32))
	for i, v := range queryFloat32 {
		queryFloat64[i] = float64(v)
	}

	science2000Results, err := rdb.VSimWithArgs(ctx, "famousPeople",
		&redis.VectorValues{Val: queryFloat64},
		&redis.VSimArgs{Filter: ".died < 2000"}).Result()
	if err != nil {
		panic(err)
	}

	fmt.Printf("'science2000': %v\n", strings.Join(science2000Results, ", "))
	// >>> 'science2000': Marie Curie, Linus Pauling, Paul Erdos, Freddie Mercury,
	// Masako Natsume

code:
Double[] science2000Embedding = null;

            try {
                science2000Embedding = convertFloatsToDoubles(predictor.predict("science"));
            } catch (Exception e) {
                // ...
            }

            VSimArgs filterArgs = VSimArgs.Builder.filter(".died < 2000");

            CompletableFuture<List<String>> filteredQuery = asyncCommands.vsim("famousPeople", filterArgs, science2000Embedding)
                    .thenApply(result -> {
                        System.out.println(result);
                        // >>> [Marie Curie, Linus Pauling, Paul Erdos, Freddie Mercury, Masako Natsume]
                        return result;
                    }).toCompletableFuture();

code:
Double[] science2000Embedding = null;

            try {
                science2000Embedding = convertFloatsToDoubles(predictor.predict("science"));
            } catch (Exception e) {
                // ...
            }

            VSimArgs filterArgs = VSimArgs.Builder.filter(".died < 2000");

            Mono<java.util.List<String>> filteredQuery = reactiveCommands.vsim("famousPeople", filterArgs, science2000Embedding)
                    .collectList().doOnNext(result -> {
                        System.out.println(result);
                        // >>> [Marie Curie, Linus Pauling, Paul Erdos, Freddie Mercury, Masako Natsume]
                    });

code:
float[] science2000Embedding;
        try {
            science2000Embedding = predictor.predict("science");
        } catch (Exception e) {
            science2000Embedding = new float[384];
        }

        VSimParams filterParams = new VSimParams().filter(".died < 2000");
        List<String> filteredResults = jedis.vsim("famousPeople", science2000Embedding, filterParams);
        System.out.println(filteredResults);
        // >>> [Marie Curie, Linus Pauling, Paul Erdos, Freddie Mercury, Masako Natsume]

code:
const queryValue5 = "science";

const queryEmbedding5 = await pipe(queryValue5, pipeOptions);
const queryArray5 = Array.from(queryEmbedding5.data);

const science2000Results = await client.vSim('famousPeople', queryArray5, {
    FILTER: '.died < 2000'
});

console.log(`'science2000': ${JSON.stringify(science2000Results)}`);
// >>> 'science2000': ["Linus Pauling","Marie Curie","Paul Erdos",
// "Masako Natsume","Freddie Mercury"]

code:
query_value = "science"

science2000_results = r.vset().vsim(
    "famousPeople",
    model.encode(query_value).astype(np.float32).tobytes(),
    filter=".died < 2000"
)

print(f"'science2000': {science2000_results}")
# >>> 'science2000': ['Marie Curie', 'Linus Pauling',
# 'Paul Erdos', 'Freddie Mercury', 'Masako Natsume']


title: Observability

summary: Monitor your client's activity for optimization and debugging.

title: Overview

role: overview

text:
Some Redis client libraries implement the [OpenTelemetry](https://opentelemetry.io/) (OTel)
observability framework to let you gather performance metrics
for your application. This can help you optimize performance and pinpoint problems
quickly. Currently, the following clients support OTel:

- [redis-py]()
- [go-redis]()

<!--

title: Tracing overview

text:
An execution trace is a record of the sequence of steps that the Redis
client takes as it executes commands. Each step or *span* (in OTel terminology)
represents a specific operation. In the trace, each span is recorded using an identifier
to represent the type of operation along with its start and finish time, its
completion or error status, and other relevant information.

For example, a simple trace might look like this:

[code example]

This contains two child spans, each representing a specific operation
such as executing a command or connecting to a Redis server.
Each span is recorded with its start time, end time, and status information.

A span can sometimes be broken down into sub-tasks (such as steps taken
while calling an external service), each of which is a span in its own right.
The full trace is therefore best understood as a tree of nested spans.

For example, a more complex trace might look like this:

[code example]

This trace shows how the second and third spans are themselves broken down into more
granular operations.

By examining the sequence of spans in a trace, you can determine where an error
(if any) occurred and how long each step took to execute. Since the information
in each trace is recorded, you can use monitoring tools such as
[Grafana](https://grafana.com/) to aggregate the data from many traces over time.
This can help you find operations that are slow on average compared to others
(suggesting a performance bottleneck that could be optimized) or that have a high
error rate (suggesting a deeper problem that could be fixed to improve reliability).
-->

title: Metrics overview

text:
Metrics are quantitative measurements of the behavior of your application. They
provide information such as how often a certain operation occurs, how long it
takes to complete, or how many errors have occurred. By analyzing these metrics,
you can identify performance bottlenecks, errors, and other issues that need to
be addressed.

Redis clients follow the OTel push model. This uses the
[OLTP](https://opentelemetry.io/docs/specs/otlp/)
protocol to send metrics to a [collector](https://opentelemetry.io/docs/collector/)
(both [OLTP/gRPC](https://opentelemetry.io/docs/specs/otlp/#otlpgrpc) and
[OTLP/HTTP](https://opentelemetry.io/docs/specs/otlp/#otlphttp) are supported).
A storage layer (such as [Prometheus](https://prometheus.io/)) can then pull the
metrics from a collector endpoint, making the data available for monitoring tools
(such as [Grafana](https://grafana.com/)) to query and visualize. The basic
flow of data is shown below.

[code example]

The [Redis client observability demonstration](https://github.com/redis-developer/redis-client-observability)
on GitHub contains examples showing how to set up a local Grafana instance, then
connect it to a Redis client and visualize the metric data as it arrives.

title: Redis metric groups

text:
In Redis clients, the metrics collected by OTel are organized into the following
metric groups:

- [`resiliency`](#group-resiliency): data related to the availability and health of the Redis connection.
- [`connection-basic`](#group-connection-basic): minimal metrics about Redis connections made by the client.
- [`connection-advanced`](#group-connection-advanced): more detailed metrics about Redis connections.
- [`command`](#group-command): metrics about Redis commands executed by the client.
- [`client-side-caching`](#group-client-side-caching): metrics about
  [client-side caching]() operations.
- [`streaming`](#group-streaming): metrics about
  [stream]() operations.
- [`pubsub`](#group-pubsub): metrics about
  [pub/sub]() operations.

When you configure the client to activate OTel, you can select which metric groups
you are interested in. By default, only the `resiliency` and `connection-basic` groups are enabled.
The metrics in each group are described in the
[Metrics reference](#metrics-reference) below.

title: Metrics reference

text:
The metric groups and the metrics they contain are described below. The
name in parentheses after each group name is the group's identifier, which you
use when you configure the client to select which metrics to collect.

The metrics contain *attributes* that provide extra information (such as
the client library and server details) that you can use to filter and
aggregate the data. The attributes are described in the Attributes
section following the metric groups. The badge shown after the attribute
name can be any of the following:

- `required`: This attribute will always be present in the metrics.
- `optional`: This attribute may be present in the metrics.
- `conditionally required`: This attribute will be present in the metrics only if a certain condition is met,
  such as when a specific error occurs. The condition is described in the attribute description.
- `recommended`: Specific client libraries may not support this attribute in some situations.

code:
"Trace":
    "Span 1":
        _meta:
            description: "(Start time: 10000, End time: 10005, Status: OK)"
    "Span 2":
        _meta:
            description: "(Start time: 10005, End time: 10010, Status: OK)"

section_id: tracing-overview

code:
"Trace":
    "Span 1":
        _meta:
            description: "(Start time: 10000, End time: 10005, Status: OK)"
    "Span 2":
        "Span 2.1":
            _meta:
                description: "(Start time: 10005, End time: 10010, Status: OK)"
        "Span 2.2":
            _meta:
                description: "(Start time: 10010, End time: 10015, Status: OK)"
    "Span 3":
        "Span 3.1":
            _meta:
                description: "(Start time: 10015, End time: 10020, Status: OK)"
        "Span 3.2":
            _meta:
                description: "(Start time: 10020, End time: 10025, Status: OK)"
        "Span 3.3":
            _meta:
                description: "(Start time: 10025, End time: 10030, Status: OK)"

code:
sequenceDiagram
    participant App as Redis Client
    participant Collector as OTel Collector
    participant Storage as Storage Layer
(Prometheus)
    participant Monitor as Monitoring Tool
(Grafana)

    App->>Collector: Push metrics
    Note over App,Collector: OTel push model

    Storage->>Collector: Pull metrics from endpoint
    Collector-->>Storage: Return metrics

    Monitor->>Storage: Query metrics data
    Storage-->>Monitor: Return metrics
    Note over Monitor: Visualize

section_id: metrics-overview


title: Object-Mapper libraries

summary: Object-Mapper libraries for Redis Open Source

title: Content

text:
Redis OM (pronounced *REDiss OHM*) is a library that provides object mapping for Redis. With the help of Redis OM, you can map Redis data types, specifically Hashes and JSON documents, to objects of your preferred programming language or framework. Redis OM relies on the JSON and Redis Search features of Redis Open Source, allowing you to search and/or query for objects.

You can use Redis OM with the following four programming languages:

* [Python]()
* [C#/.NET]()
* [Node.js]()
* [Java/Spring]()