Index and query vectors

Learn how to index and query vector embeddings with Redis

Redis Search lets you index vector fields in hash or JSON objects (see the Vectors reference page for more information). Among other things, vector fields can store text embeddings, which are AI-generated vector representations of the semantic information in pieces of text. The vector distance between two embeddings indicates how similar they are semantically. By comparing the similarity of an embedding generated from some query text with embeddings stored in hash or JSON fields, Redis can retrieve documents that closely match the query in terms of their meaning.

The example below uses the HuggingFace model all-MiniLM-L6-v2 to generate the vector embeddings to store and index with Redis Search. The code is first demonstrated for hash documents with a separate section to explain the differences with JSON documents.

Initialize

If you are using Maven, add the following dependencies to your pom.xml file:

<dependency>
    <groupId>io.lettuce</groupId>
    <artifactId>lettuce-core</artifactId>
     <!-- Check for the latest version on Maven Central -->
    <version>7.7.0.RELEASE</version>
</dependency>

<dependency>
    <groupId>ai.djl.huggingface</groupId>
    <artifactId>tokenizers</artifactId>
    <version>0.33.0</version>
</dependency>

<dependency>
    <groupId>ai.djl.pytorch</groupId>
    <artifactId>pytorch-model-zoo</artifactId>
    <version>0.33.0</version>
</dependency>

<dependency>
    <groupId>ai.djl</groupId>
    <artifactId>api</artifactId>
    <version>0.33.0</version>
</dependency>

If you are using Gradle, add the following dependencies to your build.gradle file:

compileOnly 'io.lettuce:lettuce-core:7.7.0.RELEASE'
compileOnly 'ai.djl.huggingface:tokenizers:0.33.0'
compileOnly 'ai.djl.pytorch:pytorch-model-zoo:0.33.0'
compileOnly 'ai.djl:api:0.33.0'

Import dependencies

Import the following classes in your source file:

Language:

Foundational: Import Lettuce, DJL, and embedding model libraries for vector search

// Lettuce client and Redis Search classes.
import io.lettuce.core.*;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.codec.ByteArrayCodec;
import io.lettuce.core.codec.RedisCodec;
import io.lettuce.core.codec.StringCodec;
import io.lettuce.core.api.async.RedisAsyncCommands;
import io.lettuce.core.search.arguments.*;
import io.lettuce.core.search.SearchReply;
import io.lettuce.core.json.JsonParser;
import io.lettuce.core.json.JsonObject;
import io.lettuce.core.json.JsonPath;

// Standard library classes for data manipulation and
// asynchronous programming.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.StandardCharsets;
import java.util.*;
import java.util.concurrent.CompletableFuture;

// DJL classes for model loading and inference.
import ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory;
import ai.djl.inference.Predictor;
import ai.djl.repository.zoo.Criteria;
import ai.djl.training.util.ProgressBar;

// Lettuce client and Redis Search classes.
import io.lettuce.core.*;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.codec.ByteArrayCodec;
import io.lettuce.core.codec.RedisCodec;
import io.lettuce.core.codec.StringCodec;
import io.lettuce.core.api.reactive.RedisReactiveCommands;
import io.lettuce.core.search.arguments.*;
import io.lettuce.core.search.SearchReply;
import io.lettuce.core.json.JsonParser;
import io.lettuce.core.json.JsonObject;
import io.lettuce.core.json.JsonPath;

// Standard library classes for data manipulation and
// reactive programming.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.StandardCharsets;
import java.util.*;
import reactor.core.publisher.Mono;

// DJL classes for model loading and inference.
import ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory;
import ai.djl.inference.Predictor;
import ai.djl.repository.zoo.Criteria;
import ai.djl.training.util.ProgressBar;

Define a helper method

When you store vectors in a hash object, or pass them as query parameters, you must encode the float components of the vector array as a byte string. The helper method floatArrayToByteArray() shown below does this for you:

Language:

Foundational: Convert float arrays to byte arrays for storing vectors in hash objects

    private byte[] floatArrayToByteArray(float[] vector) {
        ByteBuffer buffer = ByteBuffer.allocate(vector.length * 4).order(ByteOrder.LITTLE_ENDIAN);
        for (float value : vector) {
            buffer.putFloat(value);
        }
        return buffer.array();
    }

    private byte[] floatArrayToByteArray(float[] vector) {
        ByteBuffer buffer = ByteBuffer.allocate(vector.length * 4).order(ByteOrder.LITTLE_ENDIAN);
        for (float value : vector) {
            buffer.putFloat(value);
        }
        return buffer.array();
    }

Create an embedding model instance

The example below uses the all-MiniLM-L6-v2 model to generate the embeddings. This model generates vectors with 384 dimensions, regardless of the length of the input text, but note that the input is truncated to 256 tokens (see Word piece tokenization at the Hugging Face docs to learn more about the way tokens are related to the original text).

The Predictor class implements the model to generate the embeddings. The code below creates an instance of Predictor that uses the all-MiniLM-L6-v2 model:

Language:

Practical pattern: Initialize a DJL Predictor with the all-MiniLM-L6-v2 embedding model

        Predictor<String, float[]> predictor = null;

        try {
            Criteria<String, float[]> criteria = Criteria.builder().setTypes(String.class, float[].class)
                    .optModelUrls("djl://ai.djl.huggingface.pytorch/sentence-transformers/all-MiniLM-L6-v2")
                    .optEngine("PyTorch").optTranslatorFactory(new TextEmbeddingTranslatorFactory())
                    .optProgress(new ProgressBar()).build();

            predictor = criteria.loadModel().newPredictor();
        } catch (Exception e) {
            // ...
        }

        Predictor<String, float[]> predictor = null;

        try {
            Criteria<String, float[]> criteria = Criteria.builder().setTypes(String.class, float[].class)
                    .optModelUrls("djl://ai.djl.huggingface.pytorch/sentence-transformers/all-MiniLM-L6-v2")
                    .optEngine("PyTorch").optTranslatorFactory(new TextEmbeddingTranslatorFactory())
                    .optProgress(new ProgressBar()).build();

            predictor = criteria.loadModel().newPredictor();
        } catch (Exception e) {
            // ...
        }

Create the index

As noted in Define a helper method above, you must pass the embeddings to the hash and query commands as a binary string.

Lettuce lets you choose a codec for each connection to Redis (see Codecs in the Lettuce documentation for more information). RedisCodec.of(StringCodec.UTF8, ByteArrayCodec.INSTANCE) combines two codecs so that keys, field names, and index names are String objects, while values are byte[] arrays. The embeddings are binary strings, and you pass text values as UTF-8 encoded byte[] arrays. However, you will probably find it more convenient to use the default StringCodec for commands that don't require binary strings. It is therefore helpful to have two connections available, one using the composed codec and one using StringCodec.

The code below shows how to declare one connection with the composed codec and another without in the try-with-resources block. You also need two separate instances of RedisAsyncCommands to use the two connections:

Language:

Practical pattern: Create dual connections with a composed String/byte[] codec and StringCodec for vector operations

        RedisClient redisClient = RedisClient.create("redis://localhost:6379");

        try (StatefulRedisConnection<String, String> connection = redisClient.connect();
                StatefulRedisConnection<String, byte[]> binConnection = redisClient
                        .connect(RedisCodec.of(StringCodec.UTF8, ByteArrayCodec.INSTANCE))) {
            RedisAsyncCommands<String, String> asyncCommands = connection.async();
            RedisAsyncCommands<String, byte[]> binAsyncCommands = binConnection.async();
            // ...

        RedisClient redisClient = RedisClient.create("redis://localhost:6379");

        try (StatefulRedisConnection<String, String> connection = redisClient.connect();
                StatefulRedisConnection<String, byte[]> binConnection = redisClient
                        .connect(RedisCodec.of(StringCodec.UTF8, ByteArrayCodec.INSTANCE))) {
            RedisReactiveCommands<String, String> reactiveCommands = connection.reactive();
            RedisReactiveCommands<String, byte[]> binReactiveCommands = binConnection.reactive();
            // ...

Next, create the index. The schema in the example below includes three fields:

The text content to index
A tag field to represent the "genre" of the text
The embedding vector generated from the original text content

The embedding field specifies HNSW indexing, the L2 vector distance metric, Float32 values to represent the vector's components, and 384 dimensions, as required by the all-MiniLM-L6-v2 embedding model.

The CreateArgs object specifies hash objects for storage and a prefix doc: that identifies the hash objects to index.

Language:

Foundational: Create a vector search index with HNSW algorithm, L2 distance metric, and 384-dimensional embeddings

            List<FieldArgs<String>> schema = Arrays.asList(TextFieldArgs.<String> builder().name("content").build(),
                    TagFieldArgs.<String> builder().name("genre").build(),
                    VectorFieldArgs.<String> builder().name("embedding").hnsw().type(VectorFieldArgs.VectorType.FLOAT32)
                            .dimensions(384).distanceMetric(VectorFieldArgs.DistanceMetric.L2).build());

            CreateArgs<String, String> createArgs = CreateArgs.<String, String> builder().on(CreateArgs.TargetType.HASH)
                    .withPrefix("doc:").build();

            CompletableFuture<Void> createIndex = asyncCommands.ftCreate("vector_idx", createArgs, schema)
                    .thenAccept(System.out::println).toCompletableFuture();

            List<FieldArgs<String>> schema = Arrays.asList(TextFieldArgs.<String> builder().name("content").build(),
                    TagFieldArgs.<String> builder().name("genre").build(),
                    VectorFieldArgs.<String> builder().name("embedding").hnsw().type(VectorFieldArgs.VectorType.FLOAT32)
                            .dimensions(384).distanceMetric(VectorFieldArgs.DistanceMetric.L2).build());

            CreateArgs<String, String> createArgs = CreateArgs.<String, String> builder().on(CreateArgs.TargetType.HASH)
                    .withPrefix("doc:").build();

            Mono<Void> createIndex = reactiveCommands.ftCreate("vector_idx", createArgs, schema).doOnNext(result -> {
                System.out.println(result); // >>> OK
            }).then();

Add data

You can now supply the data objects, which will be indexed automatically when you add them with hset(), as long as you use the doc: prefix specified in the index definition.

Use the predict() method of the Predictor object as shown below to create the embedding that represents the content field and use the floatArrayToByteArray() helper method to convert it to a binary string. Use the binary string representation when you are indexing hash objects, but use an array of float for JSON objects (see Differences with JSON objects below).

You must use instances of Map<String, byte[]> to supply the data to hset() when using the binary connection, so you encode the text values with getBytes(). Note that the predict() call is in a try/catch block because it will throw exceptions if it can't download the embedding model (you should add code to handle the exceptions in production code).

Language:

Foundational: Generate embeddings and store hash documents with vector data using a String/byte[] codec

            String sentence1 = "That is a very happy person";

            Map<String, byte[]> doc1 = new HashMap<>();
            doc1.put("content", sentence1.getBytes(StandardCharsets.UTF_8));
            doc1.put("genre", "persons".getBytes(StandardCharsets.UTF_8));

            try {
                doc1.put("embedding", floatArrayToByteArray(predictor.predict(sentence1)));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<Long> addDoc1 = binAsyncCommands.hset("doc:1", doc1)
                    .thenApply(result -> {
                        System.out.println(result); // >>> 3
                        return result;
                    }).toCompletableFuture();

            String sentence2 = "That is a happy dog";

            Map<String, byte[]> doc2 = new HashMap<>();
            doc2.put("content", sentence2.getBytes(StandardCharsets.UTF_8));
            doc2.put("genre", "pets".getBytes(StandardCharsets.UTF_8));

            try {
                doc2.put("embedding", floatArrayToByteArray(predictor.predict(sentence2)));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<Long> addDoc2 = binAsyncCommands.hset("doc:2", doc2)
                    .thenApply(result -> {
                        System.out.println(result); // >>> 3
                        return result;
                    }).toCompletableFuture();

            String sentence3 = "Today is a sunny day";

            Map<String, byte[]> doc3 = new HashMap<>();
            doc3.put("content", sentence3.getBytes(StandardCharsets.UTF_8));
            doc3.put("genre", "weather".getBytes(StandardCharsets.UTF_8));

            try {
                doc3.put("embedding", floatArrayToByteArray(predictor.predict(sentence3)));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<Long> addDoc3 = binAsyncCommands.hset("doc:3", doc3)
                    .thenApply(result -> {
                        System.out.println(result); // >>> 3
                        return result;
                    }).toCompletableFuture();

            String sentence1 = "That is a very happy person";

            Map<String, byte[]> doc1 = new HashMap<>();
            doc1.put("content", sentence1.getBytes(StandardCharsets.UTF_8));
            doc1.put("genre", "persons".getBytes(StandardCharsets.UTF_8));

            try {
                doc1.put("embedding", floatArrayToByteArray(predictor.predict(sentence1)));
            } catch (Exception e) {
                // ...
            }

            Mono<Long> addDoc1 = binReactiveCommands.hset("doc:1", doc1).doOnNext(result -> {
                System.out.println(result); // >>> 3
            });

            String sentence2 = "That is a happy dog";

            Map<String, byte[]> doc2 = new HashMap<>();
            doc2.put("content", sentence2.getBytes(StandardCharsets.UTF_8));
            doc2.put("genre", "pets".getBytes(StandardCharsets.UTF_8));

            try {
                doc2.put("embedding", floatArrayToByteArray(predictor.predict(sentence2)));
            } catch (Exception e) {
                // ...
            }

            Mono<Long> addDoc2 = binReactiveCommands.hset("doc:2", doc2).doOnNext(result -> {
                System.out.println(result); // >>> 3
            });

            String sentence3 = "Today is a sunny day";

            Map<String, byte[]> doc3 = new HashMap<>();
            doc3.put("content", sentence3.getBytes(StandardCharsets.UTF_8));
            doc3.put("genre", "weather".getBytes(StandardCharsets.UTF_8));

            try {
                doc3.put("embedding", floatArrayToByteArray(predictor.predict(sentence3)));
            } catch (Exception e) {
                // ...
            }

            Mono<Long> addDoc3 = binReactiveCommands.hset("doc:3", doc3).doOnNext(result -> {
                System.out.println(result); // >>> 3
            });

Bulk import with HIMPORT

From Redis Open Source 8.10, you can also add the documents with the HIMPORT command family. This creates the same hashes as hset(), but lets Redis store them as compact hashes, which use less memory when many keys share the same field names. Redis Search indexes these hashes in the same way.

Create a HashImport object with the field names, then create each document with himportSet(), passing only its values, in the same order as the field names. Lettuce declares the field names with HIMPORT PREPARE the first time you use the HashImport object on a connection. Because the binary connection uses String keys, HashImport.of() takes the field names as strings. Issue all the imports before you wait for any reply, so that Lettuce pipelines them instead of waiting for each reply in turn. With the reactive API, wait for the imports inside the try block, because a Mono that you subscribe to after the HashImport object closes fails:

Language:

Bulk ingestion: Import hash documents with vector embeddings as compact hashes using a HashImport fieldset and pipelined HIMPORT SET calls

            String[][] docData = {
                    { "doc:1", "That is a very happy person", "persons" },
                    { "doc:2", "That is a happy dog", "pets" },
                    { "doc:3", "Today is a sunny day", "weather" } };

            List<RedisFuture<String>> himportDocs = new ArrayList<>();

            try (HashImport<String> fieldset = HashImport.of("content", "genre", "embedding")) {
                for (String[] doc : docData) {
                    byte[] docEmbedding = null;

                    try {
                        docEmbedding = floatArrayToByteArray(predictor.predict(doc[1]));
                    } catch (Exception e) {
                        // ...
                    }

                    himportDocs.add(binAsyncCommands.himportSet(doc[0], fieldset, doc[1].getBytes(StandardCharsets.UTF_8),
                            doc[2].getBytes(StandardCharsets.UTF_8), docEmbedding));
                }
            }

            CompletableFuture<Void> importDocs = CompletableFuture
                    .allOf(himportDocs.stream().map(RedisFuture::toCompletableFuture).toArray(CompletableFuture[]::new));

            String[][] docData = {
                    { "doc:1", "That is a very happy person", "persons" },
                    { "doc:2", "That is a happy dog", "pets" },
                    { "doc:3", "Today is a sunny day", "weather" } };

            List<Mono<String>> himportDocs = new ArrayList<>();

            try (HashImport<String> fieldset = HashImport.of("content", "genre", "embedding")) {
                for (String[] doc : docData) {
                    byte[] docEmbedding = null;

                    try {
                        docEmbedding = floatArrayToByteArray(predictor.predict(doc[1]));
                    } catch (Exception e) {
                        // ...
                    }

                    himportDocs.add(binReactiveCommands.himportSet(doc[0], fieldset, doc[1].getBytes(StandardCharsets.UTF_8),
                            doc[2].getBytes(StandardCharsets.UTF_8), docEmbedding));
                }

                // Subscribe before the fieldset closes.
                Mono.when(himportDocs).block();
            }

himportSet() replaces any existing key, so running this code after the hset() calls leaves you with the same three documents.

Note:

HIMPORT requires Lettuce v7.7.0 or later, and Lettuce marks its support as experimental, so the API might change. Redis Cloud and Redis Software don't support HIMPORT.

Run a query

After you have created the index and added the data, you are ready to run a query. To do this, you must create another embedding vector from your chosen query text. Redis calculates the vector distance between the query vector and each embedding vector in the index as it runs the query. You can request the results to be sorted to rank them in order of ascending distance.

The code below creates the query embedding using the predict() method, as with the indexing, and passes it as a parameter when the query executes (see Vector search for more information about using query parameters with embeddings). The query is a K nearest neighbors (KNN) search that sorts the results in order of vector distance from the query vector.

Language:

Semantic search: Execute a KNN vector search with semantic similarity ranking on hash documents

            String query = "That is a happy person";
            float[] queryEmbedding = null;

            try {
                queryEmbedding = predictor.predict(query);
            } catch (Exception e) {
                // ...
            }

            SearchArgs<String, byte[]> searchArgs = SearchArgs.<String, byte[]> builder()
                    .param("vec", floatArrayToByteArray(queryEmbedding)).returnField("content").returnField("vector_distance")
                    .sortBy(SortByArgs.<String> builder().attribute("vector_distance").build()).build();

            CompletableFuture<SearchReply<String, byte[]>> hashQuery = binAsyncCommands
                    .ftSearch("vector_idx", "*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes(StandardCharsets.UTF_8),
                            searchArgs)
                    .thenApply(result -> {
                        List<SearchReply.SearchResult<String, byte[]>> results = result.getResults();

                        results.forEach(r -> {
                            String content = new String(r.getFields().get("content"), StandardCharsets.UTF_8);
                            String distance = new String(r.getFields().get("vector_distance"), StandardCharsets.UTF_8);

                            System.out.println("ID: " + r.getId() + ", Content: " + content + ", Distance: " + distance);
                        });
                        // >>> ID: doc:1, Content: That is a very happy person, Distance: 0.114169836044
                        // >>> ID: doc:2, Content: That is a happy dog, Distance: 0.610845506191
                        // >>> ID: doc:3, Content: Today is a sunny day, Distance: 1.48624765873

                        return result;
                    }).toCompletableFuture();

            String query = "That is a happy person";
            float[] queryEmbedding = null;

            try {
                queryEmbedding = predictor.predict(query);
            } catch (Exception e) {
                // ...
            }

            SearchArgs<String, byte[]> searchArgs = SearchArgs.<String, byte[]> builder()
                    .param("vec", floatArrayToByteArray(queryEmbedding)).returnField("content").returnField("vector_distance")
                    .sortBy(SortByArgs.<String> builder().attribute("vector_distance").build()).build();

            Mono<SearchReply<String, byte[]>> hashQuery = binReactiveCommands
                    .ftSearch("vector_idx", "*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes(StandardCharsets.UTF_8),
                            searchArgs)
                    .doOnNext(result -> {
                        List<SearchReply.SearchResult<String, byte[]>> results = result.getResults();

                        results.forEach(r -> {
                            String content = new String(r.getFields().get("content"), StandardCharsets.UTF_8);
                            String distance = new String(r.getFields().get("vector_distance"), StandardCharsets.UTF_8);

                            System.out.println("ID: " + r.getId() + ", Content: " + content + ", Distance: " + distance);
                        });
                        // >>> ID: doc:1, Content: That is a very happy person, Distance: 0.114169836044
                        // >>> ID: doc:2, Content: That is a happy dog, Distance: 0.610845506191
                        // >>> ID: doc:3, Content: Today is a sunny day, Distance: 1.48624765873
                    });

Assuming you have added the code from the steps above to your source file, it is now ready to run, but note that it may take a while to complete when you run it for the first time (which happens because the model must download the all-MiniLM-L6-v2 model data before it can generate the embeddings). When you run the code, it outputs the following result text:

ID: doc:1, Content: That is a very happy person, Distance: 0.114169836044
ID: doc:2, Content: That is a happy dog, Distance: 0.610845506191
ID: doc:3, Content: Today is a sunny day, Distance: 1.48624765873

Note that the results are ordered according to the value of the distance field, with the lowest distance indicating the greatest similarity to the query. As you would expect, the result for doc:1 with the content text "That is a very happy person" is the result that is most similar in meaning to the query text "That is a happy person".

Differences with JSON documents

Indexing JSON documents is similar to hash indexing, but there are some important differences. JSON allows much richer data modeling with nested fields, so you must supply a path in the schema to identify each field you want to index. However, you can declare a short alias for each of these paths (using the as() option) to avoid typing it in full for every query. Also, you must specify CreateArgs.TargetType.JSON when you create the index.

The code below shows these differences, but the index is otherwise very similar to the one created previously for hashes:

Language:

Foundational: Create a vector search index on JSON documents with JSON paths and field aliases

            List<FieldArgs<String>> jsonSchema = Arrays.asList(
                    TextFieldArgs.<String> builder().name("$.content").as("content").build(),
                    TagFieldArgs.<String> builder().name("$.genre").as("genre").build(),
                    VectorFieldArgs.<String> builder().name("$.embedding").as("embedding").hnsw()
                            .type(VectorFieldArgs.VectorType.FLOAT32).dimensions(384)
                            .distanceMetric(VectorFieldArgs.DistanceMetric.L2).build());

            CreateArgs<String, String> jsonCreateArgs = CreateArgs.<String, String> builder().on(CreateArgs.TargetType.JSON)
                    .withPrefix("jdoc:").build();

            CompletableFuture<Void> jsonCreateIndex = asyncCommands.ftCreate("vector_json_idx", jsonCreateArgs, jsonSchema)
                    .thenAccept(System.out::println).toCompletableFuture();

            List<FieldArgs<String>> jsonSchema = Arrays.asList(
                    TextFieldArgs.<String> builder().name("$.content").as("content").build(),
                    TagFieldArgs.<String> builder().name("$.genre").as("genre").build(),
                    VectorFieldArgs.<String> builder().name("$.embedding").as("embedding").hnsw()
                            .type(VectorFieldArgs.VectorType.FLOAT32).dimensions(384)
                            .distanceMetric(VectorFieldArgs.DistanceMetric.L2).build());

            CreateArgs<String, String> jsonCreateArgs = CreateArgs.<String, String> builder().on(CreateArgs.TargetType.JSON)
                    .withPrefix("jdoc:").build();

            Mono<Void> jsonCreateIndex = reactiveCommands.ftCreate("vector_json_idx", jsonCreateArgs, jsonSchema)
                    .doOnNext(result -> {
                        System.out.println(result); // >>> OK
                    }).then();

An important difference with JSON indexing is that the vectors are specified using arrays of float instead of binary strings. This means you don't need to use the binary connection, and you can use Arrays.toString() to convert the float array to a suitable JSON string.

Use jsonSet() to add the data instead of hset(). Use instances of JSONObject to supply the data instead of Map, as you would for hash objects.

Language:

Foundational: Store JSON documents with vector embeddings as float arrays using JSON.SET

            JsonParser parser = asyncCommands.getJsonParser();

            String jSentence1 = "That is a very happy person";

            JsonObject jDoc1 = parser.createJsonObject();
            jDoc1.put("content", parser.createJsonValue("\"" + jSentence1 + "\""));
            jDoc1.put("genre", parser.createJsonValue("\"persons\""));

            try {
                jDoc1.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence1))));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<String> jsonAddDoc1 = asyncCommands.jsonSet("jdoc:1", JsonPath.ROOT_PATH, jDoc1)
                    .thenApply(result -> {
                        System.out.println(result); // >>> OK
                        return result;
                    }).toCompletableFuture();

            String jSentence2 = "That is a happy dog";

            JsonObject jDoc2 = parser.createJsonObject();
            jDoc2.put("content", parser.createJsonValue("\"" + jSentence2 + "\""));
            jDoc2.put("genre", parser.createJsonValue("\"pets\""));

            try {
                jDoc2.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence2))));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<String> jsonAddDoc2 = asyncCommands.jsonSet("jdoc:2", JsonPath.ROOT_PATH, jDoc2)
                    .thenApply(result -> {
                        System.out.println(result); // >>> OK
                        return result;
                    }).toCompletableFuture();

            String jSentence3 = "Today is a sunny day";

            JsonObject jDoc3 = parser.createJsonObject();
            jDoc3.put("content", parser.createJsonValue("\"" + jSentence3 + "\""));
            jDoc3.put("genre", parser.createJsonValue("\"weather\""));

            try {
                jDoc3.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence3))));
            } catch (Exception e) {
                // ...
            }

            CompletableFuture<String> jsonAddDoc3 = asyncCommands.jsonSet("jdoc:3", JsonPath.ROOT_PATH, jDoc3)
                    .thenApply(result -> {
                        System.out.println(result); // >>> OK
                        return result;
                    }).toCompletableFuture();

            JsonParser parser = reactiveCommands.getJsonParser();

            String jSentence1 = "That is a very happy person";
            JsonObject jDoc1 = parser.createJsonObject();
            jDoc1.put("content", parser.createJsonValue("\"" + jSentence1 + "\""));
            jDoc1.put("genre", parser.createJsonValue("\"persons\""));

            try {
                jDoc1.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence1))));
            } catch (Exception e) {
                // ...
            }

            Mono<Void> jsonAddDoc1 = reactiveCommands.jsonSet("jdoc:1", JsonPath.ROOT_PATH, jDoc1).doOnNext(result -> {
                System.out.println(result); // >>> OK
            }).then();

            String jSentence2 = "That is a happy dog";
            JsonObject jDoc2 = parser.createJsonObject();
            jDoc2.put("content", parser.createJsonValue("\"" + jSentence2 + "\""));
            jDoc2.put("genre", parser.createJsonValue("\"pets\""));

            try {
                jDoc2.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence2))));
            } catch (Exception e) {
                // ...
            }

            Mono<Void> jsonAddDoc2 = reactiveCommands.jsonSet("jdoc:2", JsonPath.ROOT_PATH, jDoc2).doOnNext(result -> {
                System.out.println(result); // >>> OK
            }).then();

            String jSentence3 = "Today is a sunny day";
            JsonObject jDoc3 = parser.createJsonObject();
            jDoc3.put("content", parser.createJsonValue("\"" + jSentence3 + "\""));
            jDoc3.put("genre", parser.createJsonValue("\"weather\""));

            try {
                jDoc3.put("embedding", parser.createJsonValue(Arrays.toString(predictor.predict(jSentence3))));
            } catch (Exception e) {
                // ...
            }

            Mono<Void> jsonAddDoc3 = reactiveCommands.jsonSet("jdoc:3", JsonPath.ROOT_PATH, jDoc3).doOnNext(result -> {
                System.out.println(result); // >>> OK
            }).then();

The query is almost identical to the one for the hash documents. This demonstrates how the right choice of aliases for the JSON paths can save you having to write complex queries. An important thing to notice is that the vector parameter for the query is still specified as a binary string, even though the data for the embedding field of the JSON was specified as an array.

Language:

Semantic search: Execute a KNN vector search on JSON documents with semantic similarity ranking

            String jQuery = "That is a happy person";
            float[] jsonQueryEmbedding = null;

            try {
                jsonQueryEmbedding = predictor.predict(jQuery);
            } catch (Exception e) {
                // ...
            }

            SearchArgs<String, byte[]> jsonSearchArgs = SearchArgs.<String, byte[]> builder()
                    .param("vec", floatArrayToByteArray(jsonQueryEmbedding)).returnField("content").returnField("vector_distance")
                    .sortBy(SortByArgs.<String> builder().attribute("vector_distance").build()).build();

            CompletableFuture<SearchReply<String, byte[]>> jsonQuery = binAsyncCommands
                    .ftSearch("vector_json_idx", "*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes(StandardCharsets.UTF_8),
                            jsonSearchArgs)
                    .thenApply(result -> {
                        List<SearchReply.SearchResult<String, byte[]>> results = result.getResults();

                        results.forEach(r -> {
                            String content = new String(r.getFields().get("content"), StandardCharsets.UTF_8);
                            String distance = new String(r.getFields().get("vector_distance"), StandardCharsets.UTF_8);

                            System.out.println("ID: " + r.getId() + ", Content: " + content + ", Distance: " + distance);
                        });
                        // >>> ID: jdoc:1, Content: That is a very happy person, Distance: 0.114169836044
                        // >>> ID: jdoc:2, Content: That is a happy dog, Distance: 0.610845506191
                        // >>> ID: jdoc:3, Content: Today is a sunny day, Distance: 1.48624765873

                        return result;
                    }).toCompletableFuture();

            String jQuery = "That is a happy person";
            float[] jsonQueryEmbedding = null;

            try {
                jsonQueryEmbedding = predictor.predict(jQuery);
            } catch (Exception e) {
                // ...
            }

            SearchArgs<String, byte[]> jsonSearchArgs = SearchArgs.<String, byte[]> builder()
                    .param("vec", floatArrayToByteArray(jsonQueryEmbedding)).returnField("content").returnField("vector_distance")
                    .sortBy(SortByArgs.<String> builder().attribute("vector_distance").build()).build();

            Mono<SearchReply<String, byte[]>> jsonQuery = binReactiveCommands
                    .ftSearch("vector_json_idx", "*=>[KNN 3 @embedding $vec AS vector_distance]".getBytes(StandardCharsets.UTF_8),
                            jsonSearchArgs)
                    .doOnNext(result -> {
                        List<SearchReply.SearchResult<String, byte[]>> results = result.getResults();

                        results.forEach(r -> {
                            String content = new String(r.getFields().get("content"), StandardCharsets.UTF_8);
                            String distance = new String(r.getFields().get("vector_distance"), StandardCharsets.UTF_8);

                            System.out.println("ID: " + r.getId() + ", Content: " + content + ", Distance: " + distance);
                        });
                        // >>> ID: jdoc:1, Content: That is a very happy person, Distance: 0.114169836044
                        // >>> ID: jdoc:2, Content: That is a happy dog, Distance: 0.610845506191
                        // >>> ID: jdoc:3, Content: Today is a sunny day, Distance: 1.48624765873
                    });

The embeddings are generated from the same text as the hash documents, so the query returns the same distances:

ID: jdoc:1, Content: That is a very happy person, Distance: 0.114169836044
ID: jdoc:2, Content: That is a happy dog, Distance: 0.610845506191
ID: jdoc:3, Content: Today is a sunny day, Distance: 1.48624765873

Learn more

See Vector search for more information about the indexing options, distance metrics, and query format for vectors.

	Products	Version
	Develop with Redis Clients, data types, programmability, and Redis for AI. RedisVL is listed separately.
	Libraries and tools Client libraries, observability, and cloud integrations. Redis Data Integration is listed separately.
	RedisVL
	Redis Software
	Redis Cloud
	Redis for Kubernetes
	Redis Open Source
	Redis Iris context engine
	Redis Feature Form
	Redis Insight
	Redis Data Integration
	Glossary
	Command reference

Index and query vectors

Initialize

Import dependencies

Define a helper method

Create an embedding model instance

Create the index

Add data

Bulk import with HIMPORT

Run a query

Differences with JSON documents

Learn more

On this page