Configure the query performance factor for Redis Query Engine in Redis Enterprise

Configure the query performance factor for Redis Query Engine in Redis Enterprise to increase the performance of queries.

Query performance factors are intended to increase the performance of queries, including vector search. When enabled, it allows you to increase a database's compute capacity and throughput by allocating more virtual CPUs per shard in addition to horizontal scaling with more shards. This document describes how to configure the query performance factor.

Prerequisites

Redis Query Engine requires a cluster running Redis Enterprise Software version 7.4.2-54 or later.

If you do not have a cluster that supports Redis Query Engine, install Redis Enterprise Software version 7.4.2-54 or later on a new cluster, or upgrade an existing cluster.

Sizing

1. Calculate the hardware requirements for your Redis database:

   1. Use the hardware requirements documentation to derive the overall cluster architecture.

   2. Calculate the RAM requirements using the Index Size Calculator. The total RAM required is the sum of the dataset and index sizes.

2. Determine the query performance factor you want and the required number of CPUs. Unused CPUs, above the 20% necessary for Redis, can be used for the scalable Redis Query Engine.

3. Create a new Redis database with the number of CPUs configured for the Query performance factor.

Calculate performance factor

CPUs for query performance factor

Vertical scaling of the Redis Query Engine is achieved by provisioning additional CPUs for the search module. At least 20% of the available CPUs must be reserved for Redis internal processing. Use the following formula to define the maximum number of CPUs that can be allocated to search.

Query performance factor versus CPUs

The following table shows the number of CPUs required for each performance factor. This calculation is sensitive to how the search index and queries are defined. Certain scenarios might yield less throughput than the ratios in the following table.

Example performance factor calculation

Configure query performance factor

To configure the query performance factor in Redis Enterprise, use the REST API to create a new database or update an existing database.

Create new database

To create a database and configure the query performance factor, use the create database REST API endpoint with a BDB object that includes the following parameters:

{
    "sched_policy": "mnp",
    "conns": 32,
    "module_list": [{
        "module_name": "search",
        "module_args": "MT_MODE MT_MODE_FULL WORKER_THREADS <NUMBER_OF_CPUS>"
    }]
}

See Calculate performance factor to determine the value to use for <NUMBER_OF_CPUS>.

Example REST API request for a new database

The following JSON is an example request body used to create a new database with a 4x query performance factor configured:

{
    "name": "scalable-search-db",
    "type": "redis",
    "memory_size": 10000000,
    "port": 13000,
    "authentication_redis_pass": "<your default db pwd>",
    "proxy_policy": "all-master-shards",
    "sched_policy": "mnp",
    "conns": 32,
    "sharding": true,
    "shards_count": 3,
    "shards_placement": "sparse",
    "shard_key_regex": [{"regex": ".*\\{(?<tag>.*)\\}.*"}, {"regex": "(?<tag>.*)"}],
    "replication": false,
    "module_list": [{
        "module_name": "search",
        "module_args": "MT_MODE MT_MODE_FULL WORKER_THREADS 6"
    }]
}

The following cURL request creates a new database from the JSON example:

curl -k -u "<user>:<password>" https://<host>:9443/v1/bdbs -H "Content-Type:application/json" -d @scalable-search-db.json

Update existing database

To configure the query performance factor for an existing database, use the following REST API requests:

• Update database configuration to modify the DMC proxy.

• Upgrade module to set the search module’s query performance factor.

The following example script uses both endpoints to configure a 4x query performance factor:

#!/bin/bash
export DB_ID=1
export CPU=6
export MODULE_ID=`curl -s -k -u "<user>:<password>" https://<host>:9443/v1/bdbs/$DB_ID | jq '.module_list[] | select(.module_name=="search").module_id' | tr -d '"'`

curl -o /dev/null -s -k -u "<user>:<password>" -X PUT https://<host>:9443/v1/bdbs/$DB_ID -H "Content-Type:application/json" -d '{
    "sched_policy": "mnp",
    "conns": 32
}'

sleep 1

curl -o /dev/null -s -k -u "<user>:<password>" https://<host>:9443/v1/bdbs/$DB_ID/modules/upgrade -H "Content-Type:application/json" -d '{
    "modules": [
      {
        "module_name": "search",
        "new_module_args": "MT_MODE MT_MODE_FULL WORKER_THREADS '$CPU'",
        "current_module": "'$MODULE_ID'",
        "new_module": "'$MODULE_ID'"
      }
    ]
}'

Monitoring Redis Query Engine

To monitor a database with a query performance factor configured:

1. Integrate your Redis Enterprise deployment with Prometheus. See Prometheus and Grafana with Redis Enterprise for instructions.

2. Monitor the redis_process_cpu_usage_percent shard metric.

   The following Prometheus UI screenshot shows redis_process_cpu_usage_percent spikes for a database with two shards:

   • 1st 100% spike: memtier_benchmark search test at the default (no additional CPUs for search).

   • 2nd 100% spike: reconfiguration and shard restart for a 4x query performance factor.

   • 3rd 600% spike: memtier_benchmark search test with threading at a 4x query performance factor (6 CPUs per shard).