Threshold Optimization

Overview

After setting up `SemanticRouter` or `SemanticCache` it's best to tune the `distance_threshold` to get the most performance out of your system. RedisVL provides helper classes to make this light weight optimization easy.

**Note:** Threshold optimization relies on `python > 3.9.`

# CacheThresholdOptimizer

Let's say you setup the following semantic cache with a distance_threshold of `X` and store the entries:

- prompt: `what is the capital of france?` response: `paris`
- prompt: `what is the capital of morocco?` response: `rabat`

[code example]

    /Users/justin.cechmanek/.pyenv/versions/3.13/envs/redisvl-dev/lib/python3.13/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
      from .autonotebook import tqdm as notebook_tqdm

    16:16:11 sentence_transformers.SentenceTransformer INFO   Use pytorch device_name: mps
    16:16:11 sentence_transformers.SentenceTransformer INFO   Load pretrained SentenceTransformer: redis/langcache-embed-v1

    Batches:   0%|          | 0/1 [00:00<?, ?it/s]Compiling the model with `torch.compile` and using a `torch.mps` device is not supported. Falling back to non-compiled mode.
    Batches: 100%|██████████| 1/1 [00:00<00:00,  3.38it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00,  1.02it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 25.04it/s]

This works well but we want to make sure the cache only applies for the appropriate questions. If we test the cache with a question we don't want a response to we see that the current distance_threshold is too high.

[code example]

    Batches: 100%|██████████| 1/1 [00:00<00:00,  1.24it/s]

    [{'entry_id': 'c990cc06e5e77570e5f03360426d2b7f947cbb5a67daa8af8164bfe0b3e24fe3',
      'prompt': 'what is the capital of france?',
      'response': 'paris',
      'vector_distance': 0.335606634617,
      'inserted_at': 1746051375.81,
      'updated_at': 1746051375.81,
      'key': 'sem_cache:c990cc06e5e77570e5f03360426d2b7f947cbb5a67daa8af8164bfe0b3e24fe3'}]

Define test_data and optimize

With the `CacheThresholdOptimizer` you can quickly tune the distance threshold by providing some test data in the form:

[code example]

The threshold optimizer will then efficiently execute and score different threshold against the what is currently populated in your cache and automatically update the threshold of the cache to the best setting

[code example]

    Distance threshold before: 0.5

    Batches: 100%|██████████| 1/1 [00:00<00:00,  1.09it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 23.17it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 24.43it/s]
    /Users/justin.cechmanek/.pyenv/versions/3.13/envs/redisvl-dev/lib/python3.13/site-packages/ranx/metrics/f1.py:36: NumbaTypeSafetyWarning: unsafe cast from uint64 to int64. Precision may be lost.
      scores[i] = _f1(qrels[i], run[i], k, rel_lvl)

    Distance threshold after: 0.10372881355932204

We can also see that we no longer match on the incorrect example:

[code example]

    Batches: 100%|██████████| 1/1 [00:00<00:00, 12.39it/s]

    []

But still match on highly relevant prompts:

[code example]

    Batches: 100%|██████████| 1/1 [00:00<00:00, 25.92it/s]

    [{'entry_id': 'c990cc06e5e77570e5f03360426d2b7f947cbb5a67daa8af8164bfe0b3e24fe3',
      'prompt': 'what is the capital of france?',
      'response': 'paris',
      'vector_distance': 0.043138384819,
      'inserted_at': 1746051375.81,
      'updated_at': 1746051375.81,
      'key': 'sem_cache:c990cc06e5e77570e5f03360426d2b7f947cbb5a67daa8af8164bfe0b3e24fe3'}]

# RouterThresholdOptimizer

Very similar to the caching case, you can optimize your router.

Define the routes

[code example]

Initialize the SemanticRouter

[code example]

    16:16:41 sentence_transformers.SentenceTransformer INFO   Use pytorch device_name: mps
    16:16:41 sentence_transformers.SentenceTransformer INFO   Load pretrained SentenceTransformer: sentence-transformers/all-mpnet-base-v2

    Batches: 100%|██████████| 1/1 [00:00<00:00,  5.90it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00,  4.97it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.29it/s]

Provide test_data

[code example]

Optimize

Note: by default route distance threshold optimization will use a random search to find the best threshold since, unlike caching, there are many thresholds to optimize concurrently.

[code example]

    Route thresholds before: {'greeting': 0.5, 'farewell': 0.5}

    Batches: 100%|██████████| 1/1 [00:00<00:00,  7.52it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.81it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.98it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.45it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.33it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.36it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.65it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.68it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00,  9.45it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.54it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 44.41it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00,  9.21it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00,  9.56it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.66it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.82it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 71.34it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.43it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.47it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.16it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.54it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.00it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.27it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.18it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.69it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.43it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.08it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.44it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.34it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.59it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00,  7.11it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00,  9.09it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.68it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.00it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.14it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.37it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.86it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.03it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.97it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.17it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.85it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.30it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.43it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.73it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.66it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.30it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 71.80it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.50it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 72.13it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 74.82it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.64it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 72.13it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.36it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.13it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.84it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.54it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 72.29it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.65it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.73it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.30it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.15it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.66it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.28it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 75.60it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 73.34it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 72.31it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.26it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.83it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.43it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.30it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 71.86it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 74.77it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.30it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.45it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.17it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.52it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.95it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 72.13it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.44it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.54it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.25it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.19it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.58it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.42it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.09it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.81it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.47it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.34it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 53.98it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.37it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.07it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.17it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.96it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.35it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.44it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.54it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 72.35it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.29it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.32it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.37it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.59it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 71.14it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.93it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.15it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.85it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.55it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 56.62it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.30it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.78it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.14it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.88it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.76it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.95it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.28it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.52it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.75it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.76it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.58it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.73it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.28it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.37it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 56.33it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.02it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.96it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.80it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 52.71it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.92it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.40it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.26it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.94it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.61it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.73it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.75it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.26it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.94it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.38it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.96it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.46it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.29it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.94it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.80it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.36it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.63it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.72it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.42it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.46it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.76it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.73it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.41it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 22.60it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.48it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.27it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.79it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.02it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.24it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.95it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.73it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.25it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 72.12it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 71.30it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 71.67it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.32it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 71.63it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 71.39it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 72.21it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.08it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.69it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.47it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.04it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.11it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.95it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.20it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.88it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.69it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.30it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.32it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.66it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.43it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.60it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.29it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.07it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.30it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.92it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.92it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.32it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.87it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.90it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.28it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.03it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.60it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.77it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.57it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.19it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.22it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.19it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.16it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.36it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.35it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.23it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.73it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.36it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.03it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.25it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.14it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.75it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.75it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.03it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.18it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.82it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.91it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.66it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.59it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.15it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.79it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.12it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.79it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.48it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.55it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 72.56it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.08it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.16it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.51it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 71.72it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.76it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 71.80it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.68it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 72.31it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.08it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.74it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.39it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.05it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.25it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.50it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.96it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 71.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 20.79it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.25it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.29it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.83it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.71it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.44it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.45it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.09it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.22it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.40it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.09it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.61it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.27it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.96it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.76it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.58it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.89it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.49it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.81it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.02it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.82it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.65it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.24it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.17it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.23it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.34it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.43it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.42it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.56it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.31it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.61it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.63it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.13it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.65it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 52.12it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.91it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.39it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.87it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.06it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.97it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.52it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.49it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.02it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.21it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.51it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.87it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.91it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 55.59it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.41it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.65it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.59it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.19it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.39it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.55it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.67it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.40it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.71it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.84it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.98it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.65it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.80it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.57it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.91it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.28it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.62it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.57it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.55it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.24it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.14it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 22.93it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.63it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.72it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.48it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.79it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.61it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.58it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.24it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.54it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.50it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.77it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.97it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.96it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.60it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.20it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.35it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.31it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.98it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.37it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.65it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.80it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.30it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.02it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.34it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.69it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.54it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.99it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.61it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.35it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.40it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.21it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.53it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.69it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.16it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.65it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.35it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.74it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.61it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.60it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.25it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.11it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.75it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.51it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.96it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.47it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.14it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 19.47it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.93it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.31it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.57it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.38it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.51it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.12it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.60it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.27it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.73it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.85it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.40it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.86it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.41it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.54it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.63it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.77it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.27it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.09it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.59it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.32it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.25it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.37it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.83it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 56.11it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.48it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.84it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.81it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.64it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.02it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.15it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 48.02it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.87it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.65it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.22it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.46it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 56.88it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.85it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.55it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.53it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.77it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.28it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.75it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.11it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.83it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.53it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.63it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.21it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.50it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.85it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 24.78it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 51.26it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.08it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.52it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.93it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.66it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.92it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.96it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.37it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.64it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.04it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.04it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.39it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.26it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.52it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.94it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.31it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.79it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.49it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.78it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.65it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.66it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.08it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.51it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.46it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.94it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.96it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.03it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.31it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.17it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.56it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.63it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 56.72it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.09it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.11it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.14it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.57it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.98it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.72it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.61it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.24it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 26.60it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 48.69it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.41it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.40it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.24it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.19it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.19it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.58it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.63it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.43it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.19it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.40it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.42it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.22it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.80it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.48it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.80it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.15it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.55it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.26it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.54it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.91it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.54it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 56.63it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.97it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.31it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.31it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.56it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 50.77it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.48it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.06it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.79it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.45it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.24it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.24it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.72it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.95it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.16it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.38it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.71it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.26it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 28.07it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.12it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.97it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.73it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.33it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.02it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.12it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.95it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.59it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.20it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.98it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.04it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 52.98it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.17it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 56.58it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.79it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.12it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.98it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.35it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.53it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.40it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.97it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.33it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.92it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.80it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.31it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.50it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.48it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.67it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.83it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.62it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.44it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.42it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 31.04it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.48it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.55it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 54.57it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.14it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.03it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.69it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.59it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.47it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.09it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.45it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 54.12it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 53.34it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 55.71it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.44it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.53it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.57it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 56.97it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.61it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.55it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.79it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.33it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.89it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.57it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.39it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.12it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.34it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.32it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.32it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.89it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.65it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.53it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 56.02it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.35it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.51it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.22it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 55.57it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 38.77it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 54.75it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.26it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.40it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 52.05it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 53.57it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.93it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.06it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.14it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.50it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.33it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.88it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.14it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.79it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.01it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.35it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.24it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.24it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.05it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.70it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.73it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 69.52it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 56.09it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 52.72it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 54.25it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 55.34it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.47it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.92it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 28.70it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.89it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.14it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.98it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.57it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.40it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.81it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.61it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.07it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.85it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.54it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.23it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.84it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.08it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.40it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.31it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.77it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.58it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.87it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.30it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.77it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.50it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.51it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.72it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 57.14it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.08it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.53it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.98it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.40it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.41it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 67.63it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.69it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 17.08it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.97it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.83it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.86it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.74it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 56.91it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.53it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.71it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.64it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 56.45it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.61it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.10it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.71it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.22it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 59.16it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.86it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 58.44it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.69it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.12it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.96it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.95it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.36it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.27it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 68.32it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.83it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.57it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 41.90it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.38it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 20.01it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.68it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 61.99it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.13it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.30it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.66it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.39it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.27it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 63.94it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.42it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 60.40it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.71it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.32it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 62.48it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 64.38it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.65it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 65.33it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 70.37it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 55.00it/s]
    Batches: 100%|██████████| 1/1 [00:00<00:00, 66.63it/s]

    Eval metric F1: start 0.438, end 0.812
    Ending thresholds: {'greeting': 0.5828282828282831, 'farewell': 0.7545454545454545}

Test it out

[code example]

    Batches: 100%|██████████| 1/1 [00:00<00:00, 55.72it/s]

    RouteMatch(name='greeting', distance=0.295984089375)

Cleanup

[code example]

from redisvl.extensions.cache.llm import SemanticCache
from redisvl.utils.vectorize import HFTextVectorizer

sem_cache = SemanticCache(
    name="sem_cache",                                       # underlying search index name
    redis_url="redis://localhost:6379",                     # redis connection url string
    distance_threshold=0.5,                                 # semantic cache distance threshold
    vectorizer=HFTextVectorizer("redis/langcache-embed-v1") # embedding model
)

paris_key = sem_cache.store(prompt="what is the capital of france?", response="paris")
rabat_key = sem_cache.store(prompt="what is the capital of morocco?", response="rabat")

sem_cache.check("what's the capital of britain?")

from redisvl.utils.optimize import CacheThresholdOptimizer

test_data = [
    {
        "query": "What's the capital of Britain?",
        "query_match": ""
    },
    {
        "query": "What's the capital of France??",
        "query_match": paris_key
    },
    {
        "query": "What's the capital city of Morocco?",
        "query_match": rabat_key
    },
]

print(f"Distance threshold before: {sem_cache.distance_threshold} \n")
optimizer = CacheThresholdOptimizer(sem_cache, test_data)
optimizer.optimize()
print(f"Distance threshold after: {sem_cache.distance_threshold} \n")

sem_cache.check("what's the capital city of france?")

from redisvl.extensions.router import Route

routes = [
        Route(
            name="greeting",
            references=["hello", "hi"],
            metadata={"type": "greeting"},
            distance_threshold=0.5,
        ),
        Route(
            name="farewell",
            references=["bye", "goodbye"],
            metadata={"type": "farewell"},
            distance_threshold=0.5,
        ),
    ]

import os
from redisvl.extensions.router import SemanticRouter
from redisvl.utils.vectorize import HFTextVectorizer

os.environ["TOKENIZERS_PARALLELISM"] = "false"

# Initialize the SemanticRouter
router = SemanticRouter(
    name="greeting-router",
    vectorizer=HFTextVectorizer(),
    routes=routes,
    redis_url="redis://localhost:6379",
    overwrite=True # Blow away any other routing index with this name
)

test_data = [
    # Greetings
    {"query": "hello", "query_match": "greeting"},
    {"query": "hi", "query_match": "greeting"},
    {"query": "hey", "query_match": "greeting"},
    {"query": "greetings", "query_match": "greeting"},
    {"query": "good morning", "query_match": "greeting"},
    {"query": "good afternoon", "query_match": "greeting"},
    {"query": "good evening", "query_match": "greeting"},
    {"query": "howdy", "query_match": "greeting"},
    {"query": "what's up", "query_match": "greeting"},
    {"query": "yo", "query_match": "greeting"},
    {"query": "hiya", "query_match": "greeting"},
    {"query": "salutations", "query_match": "greeting"},
    {"query": "how's it going", "query_match": "greeting"},
    {"query": "how are you", "query_match": "greeting"},
    {"query": "nice to meet you", "query_match": "greeting"},
    # Farewells
    {"query": "goodbye", "query_match": "farewell"},
    {"query": "bye", "query_match": "farewell"},
    {"query": "see you later", "query_match": "farewell"},
    {"query": "take care", "query_match": "farewell"},
    {"query": "farewell", "query_match": "farewell"},
    {"query": "have a good day", "query_match": "farewell"},
    {"query": "see you soon", "query_match": "farewell"},
    {"query": "catch you later", "query_match": "farewell"},
    {"query": "so long", "query_match": "farewell"},
    {"query": "peace out", "query_match": "farewell"},
    {"query": "later", "query_match": "farewell"},
    {"query": "all the best", "query_match": "farewell"},
    {"query": "take it easy", "query_match": "farewell"},
    {"query": "have a good one", "query_match": "farewell"},
    {"query": "cheerio", "query_match": "farewell"},
    # Null matches
    {"query": "what's the capital of britain?", "query_match": ""},
    {"query": "what does laffy taffy taste like?", "query_match": ""},
]

from redisvl.utils.optimize import RouterThresholdOptimizer

print(f"Route thresholds before: {router.route_thresholds} \n")
optimizer = RouterThresholdOptimizer(router, test_data)
optimizer.optimize()

# Query the router with a statement
route_match = router("hi there")
route_match


Vectorizers

Overview

In this notebook, we will show how to use RedisVL to create embeddings using the built-in text embedding vectorizers. Today RedisVL supports:
1. OpenAI
2. HuggingFace
3. Vertex AI
4. Cohere
5. Mistral AI
6. Amazon Bedrock
7. Bringing your own vectorizer
8. VoyageAI

Before running this notebook, be sure to
1. Have installed ``redisvl`` and have that environment active for this notebook.
2. Have a running Redis Stack instance with RediSearch > 2.4 active.

For example, you can run Redis Stack locally with Docker:

[code example]

This will run Redis on port 6379 and RedisInsight at http://localhost:8001.

[code example]

Creating Text Embeddings

This example will show how to create an embedding from 3 simple sentences with a number of different text vectorizers in RedisVL.

- "That is a happy dog"
- "That is a happy person"
- "Today is a nice day"

OpenAI

The ``OpenAITextVectorizer`` makes it simple to use RedisVL with the embeddings models at OpenAI. For this you will need to install ``openai``.

[code example]

[code example]

[code example]

    Vector dimensions:  1536

    [-0.0011391325388103724,
     -0.003206387162208557,
     0.002380132209509611,
     -0.004501554183661938,
     -0.010328996926546097,
     0.012922565452754498,
     -0.005491119809448719,
     -0.0029864837415516376,
     -0.007327961269766092,
     -0.03365817293524742]

[code example]

    [-0.017466850578784943,
     1.8471690054866485e-05,
     0.00129731057677418,
     -0.02555876597762108,
     -0.019842341542243958,
     0.01603139191865921,
     -0.0037347301840782166,
     0.0009670283179730177,
     0.006618348415941,
     -0.02497442066669464]

[code example]

    Number of Embeddings: 3

Azure OpenAI

The ``AzureOpenAITextVectorizer`` is a variation of the OpenAI vectorizer that calls OpenAI models within Azure. If you've already installed ``openai``, then you're ready to use Azure OpenAI.

The only practical difference between OpenAI and Azure OpenAI is the variables required to call the API.

[code example]

[code example]

    ---------------------------------------------------------------------------

    ValueError                                Traceback (most recent call last)

    Cell In[7], line 4
          1 from redisvl.utils.vectorize import AzureOpenAITextVectorizer
          3 # create a vectorizer
    ----> 4 az_oai = AzureOpenAITextVectorizer(
          5     model=deployment_name, # Must be your CUSTOM deployment name
          6     api_config={
          7         "api_key": api_key,
          8         "api_version": api_version,
          9         "azure_endpoint": azure_endpoint
         10     },
         11 )
         13 test = az_oai.embed("This is a test sentence.")
         14 print("Vector dimensions: ", len(test))

    File ~/src/redis-vl-python/redisvl/utils/vectorize/text/azureopenai.py:78, in AzureOpenAITextVectorizer.__init__(self, model, api_config, dtype)
         54 def __init__(
         55     self,
         56     model: str = "text-embedding-ada-002",
         57     api_config: Optional[Dict] = None,
         58     dtype: str = "float32",
         59 ):
         60     """Initialize the AzureOpenAI vectorizer.
         61
         62     Args:
       (...)
         76         ValueError: If an invalid dtype is provided.
         77     """
    ---> 78     self._initialize_clients(api_config)
         79     super().__init__(model=model, dims=self._set_model_dims(model), dtype=dtype)

    File ~/src/redis-vl-python/redisvl/utils/vectorize/text/azureopenai.py:106, in AzureOpenAITextVectorizer._initialize_clients(self, api_config)
         99 azure_endpoint = (
        100     api_config.pop("azure_endpoint")
        101     if api_config
        102     else os.getenv("AZURE_OPENAI_ENDPOINT")
        103 )
        105 if not azure_endpoint:
    --> 106     raise ValueError(
        107         "AzureOpenAI API endpoint is required. "
        108         "Provide it in api_config or set the AZURE_OPENAI_ENDPOINT\
        109             environment variable."
        110     )
        112 api_version = (
        113     api_config.pop("api_version")
        114     if api_config
        115     else os.getenv("OPENAI_API_VERSION")
        116 )
        118 if not api_version:

    ValueError: AzureOpenAI API endpoint is required. Provide it in api_config or set the AZURE_OPENAI_ENDPOINT                    environment variable.

[code example]

Huggingface

[Huggingface](https://huggingface.co/models) is a popular NLP platform that has a number of pre-trained models you can use off the shelf. RedisVL supports using Huggingface "Sentence Transformers" to create embeddings from text. To use Huggingface, you will need to install the ``sentence-transformers`` library.

[code example]

[code example]

[code example]

VertexAI

[VertexAI](https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings) is GCP's fully-featured AI platform including a number of pretrained LLMs. RedisVL supports using VertexAI to create embeddings from these models. To use VertexAI, you will first need to install the ``google-cloud-aiplatform`` library.

[code example]

1. Then you need to gain access to a [Google Cloud Project](https://cloud.google.com/gcp?hl=en) and provide [access to credentials](https://cloud.google.com/docs/authentication/application-default-credentials). This is accomplished by setting the `GOOGLE_APPLICATION_CREDENTIALS` environment variable pointing to the path of a JSON key file downloaded from your service account on GCP.
2. Lastly, you need to find your [project ID](https://support.google.com/googleapi/answer/7014113?hl=en) and [geographic region for VertexAI](https://cloud.google.com/vertex-ai/docs/general/locations).

**Make sure the following env vars are set:**

[code example]

[code example]

Cohere

[Cohere](https://dashboard.cohere.ai/) allows you to implement language AI into your product. The `CohereTextVectorizer` makes it simple to use RedisVL with the embeddings models at Cohere. For this you will need to install `cohere`.

[code example]

[code example]

Special attention needs to be paid to the `input_type` parameter for each `embed` call. For example, for embedding
queries, you should set `input_type='search_query'`; for embedding documents, set `input_type='search_document'`. See
more information [here](https://docs.cohere.com/reference/embed)

[code example]

Learn more about using RedisVL and Cohere together through [this dedicated user guide](https://docs.cohere.com/docs/redis-and-cohere).

VoyageAI

[VoyageAI](https://dash.voyageai.com/) allows you to implement language AI into your product. The `VoyageAITextVectorizer` makes it simple to use RedisVL with the embeddings models at VoyageAI. For this you will need to install `voyageai`.

[code example]

[code example]

Special attention needs to be paid to the `input_type` parameter for each `embed` call. For example, for embedding
queries, you should set `input_type='query'`; for embedding documents, set `input_type='document'`. See
more information [here](https://docs.voyageai.com/docs/embeddings)

[code example]

Mistral AI

[Mistral](https://console.mistral.ai/) offers LLM and embedding APIs for you to implement into your product. The `MistralAITextVectorizer` makes it simple to use RedisVL with their embeddings model.
You will need to install `mistralai`.

[code example]

[code example]

Amazon Bedrock

Amazon Bedrock provides fully managed foundation models for text embeddings. Install the required dependencies:

[code example]

#### Configure AWS credentials:

[code example]

#### Create embeddings:

[code example]

Custom Vectorizers

RedisVL supports the use of other vectorizers and provides a class to enable compatibility with any function that generates a vector or vectors from string data

[code example]

This enables the use of custom vectorizers with other RedisVL components

[code example]

Search with Provider Embeddings

Now that we've created our embeddings, we can use them to search for similar sentences. We will use the same 3 sentences from above and search for similar sentences.

First, we need to create the schema for our index.

Here's what the schema for the example looks like in yaml for the HuggingFace vectorizer:

[code example]

[code example]

[code example]

Loading data to RedisVL is easy. It expects a list of dictionaries. The vector is stored as bytes.

[code example]

[code example]

Selecting your float data type

When embedding text as byte arrays RedisVL supports 4 different floating point data types, `float16`, `float32`, `float64` and `bfloat16`, and 2 integer types, `int8` and `uint8`.
Your dtype set for your vectorizer must match what is defined in your search index. If one is not explicitly set the default is `float32`.

[code example]

[code example]

docker run -d -p 6379:6379 -p 8001:8001 redis/redis-stack:latest

import getpass

# setup the API Key
api_key = os.environ.get("OPENAI_API_KEY") or getpass.getpass("Enter your OpenAI API key: ")

from redisvl.utils.vectorize import OpenAITextVectorizer

# create a vectorizer
oai = OpenAITextVectorizer(
    model="text-embedding-ada-002",
    api_config={"api_key": api_key},
)

test = oai.embed("This is a test sentence.")
print("Vector dimensions: ", len(test))
test[:10]

# Create many embeddings at once
sentences = [
    "That is a happy dog",
    "That is a happy person",
    "Today is a sunny day"
]

embeddings = oai.embed_many(sentences)
embeddings[0][:10]

# openai also supports asynchronous requests, which we can use to speed up the vectorization process.
embeddings = await oai.aembed_many(sentences)
print("Number of Embeddings:", len(embeddings))

# additionally to the API Key, setup the API endpoint and version
api_key = os.environ.get("AZURE_OPENAI_API_KEY") or getpass.getpass("Enter your AzureOpenAI API key: ")
api_version = os.environ.get("OPENAI_API_VERSION") or getpass.getpass("Enter your AzureOpenAI API version: ")
azure_endpoint = os.environ.get("AZURE_OPENAI_ENDPOINT") or getpass.getpass("Enter your AzureOpenAI API endpoint: ")
deployment_name = os.environ.get("AZURE_OPENAI_DEPLOYMENT_NAME", "text-embedding-ada-002")

from redisvl.utils.vectorize import AzureOpenAITextVectorizer

# create a vectorizer
az_oai = AzureOpenAITextVectorizer(
    model=deployment_name, # Must be your CUSTOM deployment name
    api_config={
        "api_key": api_key,
        "api_version": api_version,
        "azure_endpoint": azure_endpoint
    },
)

test = az_oai.embed("This is a test sentence.")
print("Vector dimensions: ", len(test))
test[:10]

# Just like OpenAI, AzureOpenAI supports batching embeddings and asynchronous requests.
sentences = [
    "That is a happy dog",
    "That is a happy person",
    "Today is a sunny day"
]

embeddings = await az_oai.aembed_many(sentences)
embeddings[0][:10]

os.environ["TOKENIZERS_PARALLELISM"] = "false"
from redisvl.utils.vectorize import HFTextVectorizer

# create a vectorizer
# choose your model from the huggingface website
hf = HFTextVectorizer(model="sentence-transformers/all-mpnet-base-v2")

# embed a sentence
test = hf.embed("This is a test sentence.")
test[:10]

# You can also create many embeddings at once
embeddings = hf.embed_many(sentences, as_buffer=True)

GOOGLE_APPLICATION_CREDENTIALS=<path to your gcp JSON creds>
GCP_PROJECT_ID=<your gcp project id>
GCP_LOCATION=<your gcp geo region for vertex ai>

from redisvl.utils.vectorize import VertexAITextVectorizer

# create a vectorizer
vtx = VertexAITextVectorizer(api_config={
    "project_id": os.environ.get("GCP_PROJECT_ID") or getpass.getpass("Enter your GCP Project ID: "),
    "location": os.environ.get("GCP_LOCATION") or getpass.getpass("Enter your GCP Location: "),
    "google_application_credentials": os.environ.get("GOOGLE_APPLICATION_CREDENTIALS") or getpass.getpass("Enter your Google App Credentials path: ")
})

# embed a sentence
test = vtx.embed("This is a test sentence.")
test[:10]

import getpass
# setup the API Key
api_key = os.environ.get("COHERE_API_KEY") or getpass.getpass("Enter your Cohere API key: ")

from redisvl.utils.vectorize import CohereTextVectorizer

# create a vectorizer
co = CohereTextVectorizer(
    model="embed-english-v3.0",
    api_config={"api_key": api_key},
)

# embed a search query
test = co.embed("This is a test sentence.", input_type='search_query')
print("Vector dimensions: ", len(test))
print(test[:10])

# embed a document
test = co.embed("This is a test sentence.", input_type='search_document')
print("Vector dimensions: ", len(test))
print(test[:10])

import getpass
# setup the API Key
api_key = os.environ.get("VOYAGE_API_KEY") or getpass.getpass("Enter your VoyageAI API key: ")

from redisvl.utils.vectorize import VoyageAITextVectorizer

# create a vectorizer
vo = VoyageAITextVectorizer(
    model="voyage-law-2",  # Please check the available models at https://docs.voyageai.com/docs/embeddings
    api_config={"api_key": api_key},
)

# embed a search query
test = vo.embed("This is a test sentence.", input_type='query')
print("Vector dimensions: ", len(test))
print(test[:10])

# embed a document
test = vo.embed("This is a test sentence.", input_type='document')
print("Vector dimensions: ", len(test))
print(test[:10])

from redisvl.utils.vectorize import MistralAITextVectorizer

mistral = MistralAITextVectorizer()

# embed a sentence using their asynchronous method
test = await mistral.aembed("This is a test sentence.")
print("Vector dimensions: ", len(test))
print(test[:10])

pip install 'redisvl[bedrock]'  # Installs boto3

import os
import getpass

if "AWS_ACCESS_KEY_ID" not in os.environ:
    os.environ["AWS_ACCESS_KEY_ID"] = getpass.getpass("Enter AWS Access Key ID: ")
if "AWS_SECRET_ACCESS_KEY" not in os.environ:
    os.environ["AWS_SECRET_ACCESS_KEY"] = getpass.getpass("Enter AWS Secret Key: ")

os.environ["AWS_REGION"] = "us-east-1"  # Change as needed

from redisvl.utils.vectorize import BedrockTextVectorizer

bedrock = BedrockTextVectorizer(
    model="amazon.titan-embed-text-v2:0"
)

# Single embedding
text = "This is a test sentence."
embedding = bedrock.embed(text)
print(f"Vector dimensions: {len(embedding)}")

# Multiple embeddings
sentences = [
    "That is a happy dog",
    "That is a happy person",
    "Today is a sunny day"
]
embeddings = bedrock.embed_many(sentences)

from redisvl.utils.vectorize import CustomTextVectorizer

def generate_embeddings(text_input, **kwargs):
    return [0.101] * 768

custom_vectorizer = CustomTextVectorizer(generate_embeddings)

custom_vectorizer.embed("This is a test sentence.")[:10]

from redisvl.extensions.cache.llm import SemanticCache

cache = SemanticCache(name="custom_cache", vectorizer=custom_vectorizer)

cache.store("this is a test prompt", "this is a test response")
cache.check("this is also a test prompt")

version: '0.1.0'

index:
    name: vectorizers
    prefix: doc
    storage_type: hash

fields:
    - name: sentence
      type: text
    - name: embedding
      type: vector
      attrs:
        dims: 768
        algorithm: flat
        distance_metric: cosine

from redisvl.index import SearchIndex

# construct a search index from the schema
index = SearchIndex.from_yaml("./schema.yaml", redis_url="redis://localhost:6379")

# create the index (no data yet)
index.create(overwrite=True)

# use the CLI to see the created index
!rvl index listall

from redisvl.redis.utils import array_to_buffer

embeddings = hf.embed_many(sentences)

data = [{"text": t,
         "embedding": array_to_buffer(v, dtype="float32")}
        for t, v in zip(sentences, embeddings)]

index.load(data)

from redisvl.query import VectorQuery

# use the HuggingFace vectorizer again to create a query embedding
query_embedding = hf.embed("That is a happy cat")

query = VectorQuery(
    vector=query_embedding,
    vector_field_name="embedding",
    return_fields=["text"],
    num_results=3
)

results = index.query(query)
for doc in results:
    print(doc["text"], doc["vector_distance"])

vectorizer = HFTextVectorizer(dtype="float16")

# subsequent calls to embed('', as_buffer=True) and embed_many('', as_buffer=True) will now encode as float16
float16_bytes = vectorizer.embed('test sentence', as_buffer=True)

# to generate embeddings with different dtype instantiate a new vectorizer
vectorizer_64 = HFTextVectorizer(dtype='float64')
float64_bytes = vectorizer_64.embed('test sentence', as_buffer=True)

float16_bytes != float64_bytes


Configuration parameters

Summary: Redis Search can be tuned through multiple configuration parameters. Some of these parameters can only be set at load-time, while other parameters can be set either at load-time or at run-time.

Overview

As of Redis 8 in Redis Open Source (Redis 8), configuration parameters for Redis Search are now set in the following ways:
* At load time via your `redis.conf` file.
* At run time (where applicable) using the [`CONFIG SET`]() command.

Also, Redis 8 persists Redis Search configuration parameters just like any other configuration parameters (e.g., using the [`CONFIG REWRITE`]() command).

Redis Search configuration parameters

The following table summarizes which configuration parameters can be set at run-time, and compatibility with Redis Software and Redis Cloud.

| Parameter name
(version < 8.0) | Parameter name
(version ≥ 8.0) | Run-time | Redis
Software | Redis
Cloud |
| :------- | :------- | :------- | :------- | :------- |
| BG_INDEX_SLEEP_GAP                | [search-bg-index-sleep-gap](#search-bg-index-sleep-gap)         | :white_large_square: | ✅ Supported

 | <nobr>❌ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| BG_INDEX_SLEEP_DURATION_US        | [search-bg-index-sleep-duration-us](#search-bg-index-sleep-duration-us) | :white_large_square: |||
| CONCURRENT_WRITE_MODE             | [search-concurrent-write-mode](#search-concurrent-write-mode)   | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| CONN_PER_SHARD                    | [search-conn-per-shard](#search-conn-per-shard)                 | :white_check_mark:   |||
| CURSOR_MAX_IDLE                   | [search-cursor-max-idle](#search-cursor-max-idle)               | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| CURSOR_READ_SIZE                  | [search-cursor-read-size](#search-cursor-read-size)             | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| CURSOR_REPLY_THRESHOLD            | [search-cursor-reply-threshold](#search-cursor-reply-threshold) | :white_check_mark:   |||
| DEFAULT_DIALECT                   | [search-default-dialect](#search-default-dialect)               | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| EXTLOAD                           | [search-ext-load](#search-ext-load)                             | :white_large_square: | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| FORK_GC_CLEAN_THRESHOLD           | [search-fork-gc-clean-threshold](#search-fork-gc-clean-threshold) | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| FORK_GC_RETRY_INTERVAL            | [search-fork-gc-retry-interval](#search-fork-gc-retry-interval) | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| FORK_GC_RUN_INTERVAL              | [search-fork-gc-run-interval](#search-fork-gc-run-interval)     | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| FORKGC_SLEEP_BEFORE_EXIT          | [search-fork-gc-sleep-before-exit](#search-fork-gc-sleep-before-exit) | :white_check_mark: |||
| FRISOINI                          | [search-friso-ini](#search-friso-ini)                             | :white_large_square: | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| [GC_POLICY](#gc_policy)           | There is no matching `CONFIG` parameter.                        | :white_large_square: | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| GCSCANSIZE                        | [search-gc-scan-size](#search-gc-scan-size)                       | :white_large_square: | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| INDEX_CURSOR_LIMIT                | [search-index-cursor-limit](#search-index-cursor-limit)         | :white_large_square: |||
| INDEX_THREADS                     | search-index-threads                                            | :white_large_square: |||
| MAXAGGREGATERESULTS               | [search-max-aggregate-results](#search-max-aggregate-results)   | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| MAXDOCTABLESIZE                   | [search-max-doctablesize](#search-max-doctablesize)             | :white_large_square: | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| MAXEXPANSIONS                     | [search-max-expansions](#search-max-expansions)                 | :white_check_mark:   |||
| MAXPREFIXEXPANSIONS               | [search-max-prefix-expansions](#search-max-prefix-expansions)   | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| MAXSEARCHRESULTS                  | [search-max-search-results](#search-max-search-results)           | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| MIN_OPERATION_WORKERS             | [search-min-operation-workers](#search-min-operation-workers)   | :white_check_mark:   |||
| MIN_PHONETIC_TERM_LEN             | [search-min-phonetic-term-len](#search-min-phonetic-term-len)   | :white_check_mark:   |||
| MINPREFIX                         | [search-min-prefix](#search-min-prefix)                         | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| MINSTEMLEN                        | [search-min-stem-len](#search-min-stem-len)                     | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| MULTI_TEXT_SLOP                   | [search-multi-text-slop](#search-multi-text-slop)               | :white_large_square: |||
| NO_MEM_POOLS                      | [search-no-mem-pools](#search-no-mem-pools)                     | :white_large_square: |||
| NOGC                              | [search-no-gc](#search-no-gc)                                     | :white_large_square: | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| ON_TIMEOUT                        | [search-on-timeout](#search-on-timeout)                         | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| PARTIAL_INDEXED_DOCS              | [search-partial-indexed-docs](#search-partial-indexed-docs)     | :white_large_square: | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| RAW_DOCID_ENCODING                | [search-raw-docid-encoding](#search-raw-docid-encoding)         | :white_large_square: |||
| SEARCH_THREADS                    | [search-threads](#search-threads)                               | :white_large_square: |||
| SEARCH_IO_THREADS                 | [search-io-threads](#search-io-threads)                         | :white_large_square: |||
| TIERED_HNSW_BUFFER_LIMIT          | [search-tiered-hnsw-buffer-limit](#search-tiered-hnsw-buffer-limit) | :white_large_square: |||
| TIMEOUT                           | [search-timeout](#search-timeout)                               | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| TOPOLOGY_VALIDATION_TIMEOUT       | [search-topology-validation-timeout](#search-topology-validation-timeout) | :white_check_mark: |||
| UNION_ITERATOR_HEAP               | [search-union-iterator-heap](#search-union-iterator-heap)       | :white_check_mark:   | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| UPGRADE_INDEX                     | [search-upgrade-index](#search-upgrade-index)                   | :white_large_square: | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| VSS_MAX_RESIZE                    | [search-vss-max-resize](#search-vss-max-resize)                 | :white_check_mark:   |||
| WORKERS_PRIORITY_BIAS_THRESHOLD   | [search-workers-priority-bias-threshold](#search-workers-priority-bias-threshold) | :white_large_square: |||
| WORKERS                           | [search-workers](#search-workers)                               | :white_check_mark: |||
| OSS_GLOBAL_PASSWORD               | Deprecated in v8.0.0. Replace with the `masterauth` password.   | :white_large_square: | ✅ Supported

 | <nobr>❌ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> |
| MT_MODE                           | Deprecated in v8.0.0. Use search-workers.                       | :white_large_square: |||
| PRIVILEGED_THREADS_NUM            | Deprecated in v8.0.0. Use search-workers-priority-bias-threshold.| :white_large_square: |||
| WORKER_THREADS                    | Deprecated in v8.0.0. Use search-min-operation-workers.         | :white_large_square: |||
| SAFEMODE                          | Deprecated in v1.6.0. This is now the default setting.          | :white_large_square: |||
| FORK_GC_CLEAN_NUMERIC_EMPTY_NODES | Deprecated in v8.0.0.                                           | :white_large_square: |||

Parameter names for Redis Open Source versions < 8.0, while deprecated, will still be supported in Redis 8.

---

search-bg-index-sleep-gap

The number of iterations to run while performing background indexing before `usleep(1)` (sleep for 1 microsecond) is called, ensuring that Redis can process other commands.

Type: integer

Valid range: `[1 .. 4294967295]`

Default: `100`

search-bg-index-sleep-duration-us

Added in v8.2.

The sleep duration (in microseconds) used during background indexing. During background indexing (triggered by `FT.CREATE` on existing keys), Redis periodically sleeps to allow the main thread to process commands. This parameter controls how long each sleep lasts. Works in conjunction with [`search-bg-index-sleep-gap`](#search-bg-index-sleep-gap), which controls how many iterations occur between sleeps.

Type: integer

Valid range: `[1 .. 999999]`

Default: `1`

search-concurrent-write-mode

If enabled, the tokenization of write queries will be performed concurrently.

Type: boolean

Default: `FALSE`

search-conn-per-shard

The number of connections to each shard in a cluster.
If `0`, the number of connections is set to `search-workers` + 1.

Type: integer

Valid range: `[0 .. 9,223,372,036,854,775,807]`

Default: `0`

search-cursor-max-idle

The maximum idle time (in ms) that can be set to the [cursor api]().

Type: integer

Valid range: `[0 .. 9,223,372,036,854,775,807]`

Default: `300000`

search-cursor-read-size

Type: integer

Default: `1000`

search-cursor-reply-threshold

The maximum number of replies to accumulate before triggering `_FT.CURSOR READ` on the shards.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Default: `1`

search-default-dialect

The default
[DIALECT]()
to be used by [`FT.CREATE`](), [`FT.AGGREGATE`](), [`FT.EXPLAIN`](), [`FT.EXPLAINCLI`](), and [`FT.SPELLCHECK`]().
See [Query dialects]()
for more information.

Default: `1`

search-ext-load

If present, Redis will try to load an extension dynamic library from the specified file path.
See [Extensions]() for details.

Type: string

Default: not set

search-fork-gc-clean-numeric-empty-nodes

Clean empty nodes from numeric tree.

Type: boolean

Default: `TRUE`

search-fork-gc-clean-threshold

The fork GC will only start to clean when the number of uncleaned documents exceeds this threshold, otherwise it will skip this run.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Default: `100`

search-fork-gc-retry-interval

Interval (in seconds) in which Redis will retry to run fork GC in case of a failure.
This setting can only be combined with the [`search-gc-policy`](#search-gc-policy) `FORK` setting.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Default: `5`

search-fork-gc-run-interval

Interval (in seconds) between two consecutive fork GC runs.
This setting can only be combined with the [`search-gc-policy`](#search-gc-policy) `FORK` setting.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Default: `30`

search-fork-gc-sleep-before-exit

The number of seconds for the fork GC to sleep before exit. This value should always be set to 0 except when testing.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Default: `0`

search-friso-ini

If present, load the custom Chinese dictionary from the specified path. See [Using custom dictionaries]() for more details.

Type: string

Default: not set

GC_POLICY

The garbage collection policy. The two supported policies are:
* FORK: uses a forked thread for garbage collection (v1.4.1 and above). This is the default GC policy since v1.6.1 and is ideal for general purpose workloads.
* LEGACY: uses a synchronous, in-process fork. This is ideal for read-heavy and append-heavy workloads with very few updates/deletes. Deprecated in v2.6.0.

Note: When `GC_POLICY` is set to `FORK`, it can be combined with the `search-fork-gc-run-interval` and `search-fork-gc-retry-interval` settings.

Type: string

Valid values: `FORK` or `DEFAULT`

Default: `FORK`

search-gc-scan-size

The bulk size of the internal GC used for cleaning up indexes.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Redis Open Source default: `100`

Redis Software default: `-1` (unlimited)

Redis Cloud defaults:
- Flexible & Annual: `-1` (unlimited)
- Free & Fixed: `10000`

search-index-cursor-limit

Added in v2.10.8.

The maximum number of cursors that can be opened, per shard, at any given time. Cursors can be opened by the user via [`FT.AGGREGATE WITHCURSOR`](). Cursors are also opened internally by  Redis Search for long-running queries. Once `INDEX_CURSOR_LIMIT` is reached, any further attempts to open a cursor will result in an error.

* Caution should be used in modifying this parameter.  Every open cursor results in additional memory usage.
* Cursor usage should be regulated first by use of [`FT.CURSOR DEL`]() and/or [`MAXIDLE`]() prior to modifying `INDEX_CURSOR_LIMIT`
* See [Cursor API]() for more details.

Type: integer

Default: `128`

search-max-aggregate-results

The maximum number of results to be returned by the `FT.AGGREGATE` command if `LIMIT` is used.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Redis Open Source default: `-1` (unlimited)

Redis Software default: `-1` (unlimited)

Redis Cloud defaults:
- Flexible & Annual: `-1` (unlimited)
- Free & Fixed: `10000`

search-max-doctablesize

The maximum size of the internal hash table used for storing documents.
Note: this configuration option doesn't limit the number of documents that can be stored. It only affects the hash table internal array maximum size.
Decreasing this property can decrease the memory overhead in cases where the index holds a small number of documents that are constantly updated.

Type: integer

Valid range: `[1 .. 18,446,744,073,709,551,615]`

Default: `1000000`

search-max-expansions

This parameter is an alias for [search-max-prefix-expansions](#search-max-prefix-expansions).

search-max-prefix-expansions

The maximum number of expansions allowed for query prefixes.
The maximum number of expansions allowed for query prefixes. Setting it too high can cause performance issues. If `search-max-prefix-expansions` is reached, the query will continue with the first acquired results. The configuration is applicable for all affix queries including prefix, suffix, and infix (contains) queries.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Default: `200`

search-max-search-results

The maximum number of results to be returned by the `FT.SEARCH` command if `LIMIT` is used. Set it to `-1` to remove the limit.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Redis Open Source default: `1000000`

Redis Software default: `1000000`

Redis Cloud defaults:
- Flexible & Annual: `1000000`
- Free & Fixed: `10000`

search-min-operation-workers

The number of worker threads to use for background tasks when the server is in an operation event.

Type: integer

Valid range: `[0 .. 8192]`

Default: `4`

search-min-phonetic-term-len

The minimum length of term to be considered for phonetic matching.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Default: `3`

search-min-prefix

The minimum number of characters allowed for prefix queries (for example, hel*). Setting it to `1` can reduce performance.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Default: `2`

search-min-stem-len

The minimum word length to stem. Setting it lower than `4` can reduce performance.

Type: integer

Valid range: `[2 .. 4,294,967,295]`

Redis Open Source default: `4`

Redis Software and Redis Cloud default: `2`

search-multi-text-slop

Set the delta that is used to increase positional offsets between array slots for multi text values.
This will allow you to control the level of separation between phrases in different array slots; related to the `SLOP` parameter of `FT.SEARCH` command.

Type: integer

Valid range: `[0 .. 4,294,967,295]`

Default: `100`

search-no-mem-pools

Set Redis Search to run without memory pools.

Type: boolean

Default: `FALSE`

search-no-gc

If set to `TRUE`, garbage collection is disabled for all indexes.

Type: boolean

Default: `FALSE`

search-on-timeout

The response policy for queries that exceed the [`search-timeout`](#search-timeout) setting can be one of the following:

* `RETURN`: this policy will return the top results accumulated by the query until it timed out.
* `FAIL`: will return an error when the query exceeds the timeout value.

Type: string

Valid values: `RETURN`, `FAIL`

Default: `RETURN`

search-partial-indexed-docs

Added in v2.0.0.

Enable/disable the Redis command filter. The filter optimizes partial updates of hashes
and may avoid re-indexing the hash if changed fields are not part of the schema.

The Redis command filter will be executed upon each Redis command.  Though the filter is
optimized, this will introduce a small increase in latency on all commands.
This configuration is best used with partially indexed documents where the non-indexed fields are updated frequently.

Type: integer

Valid values: `0` (false), `1` (true)

Default: `0`

search-raw-docid-encoding

Disable compression for DocID inverted indexes to boost CPU performance.

Type: boolean

Default: `FALSE`

search-threads

Sets the number of search threads in the coordinator thread pool.

Type: integer

search-io-threads

Sets the number of threads used in the coordinator to run I/O tasks with other shards.

Type: integer

Valid range: `[1 .. 256]`

Default: 1

search-tiered-hnsw-buffer-limit

Used for setting the buffer limit threshold for vector tiered HNSW indexes. If Redis is using `WORKERS` for indexing, and the number of vectors waiting in the buffer to be indexed exceeds this limit, new vectors are inserted directly into HNSW.

Type: integer

Valid range: `[0 .. 9,223,372,036,854,775,807]`

Default: `1024`

search-timeout

The maximum amount of time in milliseconds that a search query is allowed to run. If this time is exceeded, Redis returns the top results accumulated so far, or an error depending on the policy set with [`search-on-timeout`](#search-on-timeout). The timeout can be disabled by setting it to `0`.

* `search-timeout` refers to query time only.
* Parsing the query is not counted towards `search-timeout`.
* If `search-timeout` was not reached during the search, finalizing operations such as loading document content or reducers continue.

Type: integer

Value range: `[1 .. 9,223,372,036,854,775,807]`

Redis Open Source default: `500`

Redis Software default: `500`

Redis Cloud defaults:
- Flexible & Annual: `500`
- Free & Fixed: `100`

search-topology-validation-timeout

Sets the timeout in milliseconds for topology validation. After this timeout, any pending requests will be processed, even if the topology is not fully connected. A value of `0` means no timeout.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Default: `30000`

search-union-iterator-heap

The minimum number of iterators in a union at which the iterator will switch to a heap based implementation.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Default: `20`

search-upgrade-index

Relevant only when loading an v1.x RDB file. Specify the argument for upgrading the index.
This configuration setting is a special configuration option introduced to upgrade indexes from v1.x Redis Search versions, otherwise known as legacy indexes. This configuration option needs to be given for each legacy index, followed by the index name and all valid options for the index description (also referred to as the `ON` arguments for following hashes) as described on [FT.CREATE]() command page.

Type: string

Default: there is no default for index name, and the other arguments have the same defaults as with the [`FT.CREATE`]() command.

**Example**

[code example]

* If the RDB file does not contain a legacy index that's specified in the configuration, a warning message will be added to the log file, and loading will continue.
* If the RDB file contains a legacy index that wasn't specified in the configuration, loading will fail and the server won't start.

search-vss-max-resize

Added in v2.4.8.

The maximum memory resize (in bytes) for vector indexes.
The maximum memory resize (in bytes) for vector indexes. This value will override default memory limits if you need to allow for a large [`BLOCK_SIZE`]().

Type: integer

Valid range: `[0 .. 4,294,967,295]`

Default: `0`

search-workers-priority-bias-threshold

The number of high priority tasks to be executed at any given time by the worker thread pool before executing low priority tasks. After this number of high priority tasks are being executed, the worker thread pool will execute high and low priority tasks alternately.

Type: integer

Valid range: `[1 .. 9,223,372,036,854,775,807]`

Default: `1`

search-workers

The number of worker threads to use for query processing and background tasks.

Type: integer

Valid range: `[0 .. 8192]`

Default: `0`

search-on-oom

Specifies the response policy for queries when the server's current memory usage exceeds the configured [maxmemory](https://redis.io/docs/latest/develop/reference/eviction/#maxmem) limit.

* `IGNORE`: Execute the query regardless of current memory usage.
* `RETURN`: In cluster mode, the query returns partial results from shards that did not exceed the memory limit. Shards that exceed the limit will not contribute results.
* `FAIL`: Will return an error if memory usage exceeds the memory limit.

Type: string

Valid values: `IGNORE`, `RETURN`, `FAIL`

Default: `IGNORE`

To prevent potential out-of-memory conditions, it is recommended that you set this parameter to FAIL or RETURN rather than IGNORE.

Set configuration parameters at module load-time (deprecated)

These methods are deprecated beginning with Redis 8.

Setting configuration parameters at load-time is done by appending arguments after the `--loadmodule` argument when starting a server from the command line, or after the `loadmodule` directive in a Redis config file. For example:

In [redis.conf]():

[code example]

From the [Redis CLI](), using the [MODULE LOAD]() command:

[code example]

From the command line:

[code example]

Set configuration parameters at run-time (for supported parameters, deprecated)

These methods are deprecated beginning with Redis 8.

Redis Search exposes the `FT.CONFIG` endpoint to allow for the setting and retrieval of configuration parameters at run-time.

To set the value of a configuration parameter at run-time (for supported parameters), simply run:

[code example]

Similarly, you can retrieve current configuration parameter values using:

[code example]

Values set using [`FT.CONFIG SET`]() are not persisted after server restart.

search-upgrade-index idx PREFIX 1 tt LANGUAGE french LANGUAGE_FIELD MyLang SCORE 0.5 SCORE_FIELD MyScore
    PAYLOAD_FIELD MyPayload UPGRADE_INDEX idx1

127.0.0.6379> MODULE LOAD redisearch.so [OPT VAL]...

$ redis-server --loadmodule ./redisearch.so [OPT VAL]...


Internal design

Summary: Details about design choices and implementations

Overview

Redis Open Source implements inverted indexes on top of Redis, but unlike previous implementations of Redis inverted indexes, it uses a custom data encoding that allows more memory and CPU efficient searches, and more advanced search features.

This document details some of the design choices and how these features are implemented.

Intro: Redis String DMA

The main feature that this module takes advantage of is Redis Modules Strings Direct Memory Access (DMA).

This feature is simple, yet very powerful. It allows modules to allocate data on Redis string keys, and then get direct pointers to the data allocated by the keys without copying or serializing.

This allows very fast access to huge amounts of memory. From the module's perspective, the string value is exposed simply as `char *`, meaning it can be cast to any data structure.

You simply call `RedisModule_StringTruncate` to resize a memory chunk to the size needed. Then you call `RedisModule_StringDMA` to get direct access to the memory in that key. See [https://github.com/RedisLabs/RedisModulesSDK/blob/master/FUNCTIONS.md#redismodule_stringdma](https://github.com/RedisLabs/RedisModulesSDK/blob/master/FUNCTIONS.md#redismodule_stringdma)

This API is used in the module mainly to encode inverted indexes, and also for other auxiliary data structures.

A generic "Buffer" implementation using DMA strings can be found in [buffer.c](https://github.com/RediSearch/RediSearch/blob/master/src/buffer.c). It automatically resizes the Redis string it uses as raw memory when the capacity needs to grow.

Inverted index encoding

An [inverted index](https://en.wikipedia.org/wiki/Inverted_index) is the data structure at the heart of all search engines. The idea is simple. For each word or search term, a list of all the documents it appears in is kept. Other data is kept as well, such as term frequency, and the offsets where a term appeared in the document. Offsets are used for exact match type searches, or for ranking of results.

When a search is performed, either a single index is traversed, or the intersection or union of two or more indexes is traversed. Classic Redis implementations of search engines use sorted sets as inverted indexes. This works but has significant memory overhead, and it also does not allow for encoding of offsets, as explained above.

Redis Open Source uses string DMA (see above) to efficiently encode inverted indexes. It combines [delta encoding](https://en.wikipedia.org/wiki/Delta_encoding) and [varint encoding](https://developers.google.com/protocol-buffers/docs/encoding#varints) to encode entries, minimizing space used for indexes, while keeping decompression and traversal efficient.

For each hit (document/word entry), the following items are encoded:

* The document ID as a delta from the previous document.
* The term frequency, factored by the document's rank (see below).
* Flags, that can be used to filter only specific fields or other user-defined properties.
* An offset vector of all the document offsets of the word.

Document IDs as entered by the user are converted to internal incremental document IDs, that allow delta encoding to be efficient and let the inverted indexes be sorted by document ID.

This allows for a single index hit entry to be encoded in as little as 6 bytes. Note: this is the best case. Depending on the number of occurrences of the word in the document, this can get much higher.

To optimize searches, two additional auxiliary data structures are kept in different DMA string keys:

1. **Skip index**: a table of the index offset of 1/50th of the index entries. This allows faster lookup when intersecting inverted indexes, as the entire list doesn't need to be traversed.
2. **Score index**: In simple single-word searches, there is no real need to traverse all the results, just the top N results the user is interested in. So an auxiliary index of the top 20 or so entries is stored for each term, which are used when applicable.

Document and result ranking

Each document entered to the engine has a [TF-IDF](https://en.wikipedia.org/wiki/Tf%E2%80%93idf) scoring of each word to rank the results.

As an optimization, each inverted index hit is encoded with `TF * Document_rank` as its score, and only IDF is applied during searches. This may change in the future.

On top of that, in the case of intersection queries, the minimal distance between the terms in the query is factored into the ranking. The closest the terms are to each other, the better the result.

When searching, priority queue of the top N results requested is maintained, which are eventually returned, sorted by rank.

Index ppecs and field weights

When creating an "index" using [`FT.CREATE`](), the user specifies the fields to be indexed and their respective weights. This can be used to give some document fields, like a title, more weight in ranking results.

For example:

[code example]

will create an index on fields named title, body, and url, with scores of 10, 1, and 2 respectively.

When documents are indexed, the weights are taken from the saved *index Spec* that is stored in a special Redis key, and only fields that appear in this spec are indexed.

Query execution engine

A chained-iterator based approach is used as part of query execution, which is similar to [Python generators](https://wiki.python.org/moin/Generators) in concept.

Iterators that yield index hits are chained together. Those can be:

1. **Read Iterators**, reading hits one by one from an inverted index. For example, `hello`.
2. **Intersect Iterators**, aggregating two or more iterators, yielding only their intersection points. For example, `hello AND world`.
3. **Exact Intersect Iterators** - same as above, but yielding results only if the intersection is an exact phrase. For example,  `hello NEAR world`.
4. **Union Iterators** - combining two or more iterators, and yielding a union of their hits. For example,  `hello OR world`.

These are combined based on the query as an execution plan that is evaluated lazily. For example:

[code example]

All these iterators are lazily evaluated, entry by entry, with constant memory overhead.

The root iterator is read by the query execution engine and filtered for the top N results contained in it.

Numeric filters

It's possible to define a field in the index schema as `NUMERIC`, meaning you will be able to limit search results only to those where the given value falls within a specific range. Filtering is done by adding `FILTER` predicates (more than one is supported) to your query. For example:

[code example]

The filter syntax follows the ZRANGEBYSCORE semantics of Redis, meaning `-inf` and `+inf` are supported, and prepending `(` to a number means an exclusive range.

As of release 0.6, the implementation uses a multi-level range tree, saving ranges at multiple resolutions to allow efficient range scanning. Adding numeric filters can accelerate slow queries if the numeric range is small relative to the entire span of the filtered field. For example, a filter on dates focusing on a few days out of years of data can speed a heavy query by an order of magnitude.

Auto-complete and fuzzy suggestions

Another important feature for searching and querying is auto-completion or suggestion. It allows you to create dictionaries of weighted terms, and then query them for completion suggestions to a given user prefix.  For example, if you put the term "lcd tv" into a dictionary, sending the prefix "lc" will return it as a result. The dictionary is modeled as a compressed trie (prefix tree) with weights, that is traversed to find the top suffixes of a prefix.

Redis Open Source also allows for fuzzy suggestions, meaning you can get suggestions to user prefixes even if the user has a typo in the prefix. This is enabled using a Levenshtein automaton, allowing efficient searching of a dictionary for all terms within a maximal Levenshtein distance of a term or prefix. Suggestions are weighted based on both their original score and their distance from a prefix typed by the user. Only suggestions where the prefix is up to one Levenshtein distance away from the typed prefix are supported for performance reasons.

However, since searching for fuzzy prefixes, especially very short ones, will traverse an enormous amount of suggestions (in fact, fuzzy suggestions for any single letter will traverse the entire dictionary!), it is recommended that you use this feature carefully, and only when considering the performance penalty it incurs. Since Redis is single threaded, blocking it for any amount of time means no other queries can be processed at that time.

To support unicode fuzzy matching, 16-bit runes are used inside the trie and not bytes. This increases memory consumption if the text is purely ASCII, but allows completion with the same level of support to all modern languages. This is done in the following manner:

1. Assume all input to `FT.SUG*` commands is valid UTF-8.
2. Input strings are converted to 32-bit Unicode, optionally normalizing, case-folding, and removing accents on the way. If the conversion fails it's because the input is not valid UTF-8.
3. The 32-bit runes are trimmed to 16-bit runes using the lower 16 bits. These can be used for insertion, deletion, and search.
4. The output of searches is converted back to UTF-8.

FT.CREATE my_index title 10.0 body 1.0 url 2.0

hello ==> read("hello")

hello world ==> intersect( read("hello"), read("world") )

"hello world" ==> exact_intersect( read("hello"), read("world") )

"hello world" foo ==> intersect(
                            exact_intersect(
                                read("hello"),
                                read("world")
                            ),
                            read("foo")
                      )

FT.SEARCH products "hd tv" FILTER price 100 (300