Best practices for scalable Redis Query Engine

Best practices for scalable Redis Query Engine in Redis Software and Redis Cloud.

Vertical scaling of Redis Query Engine requires configuring query performance factors. With careful crafting of search indexes and queries, query performance factors allow throughput scaling up to 16X. The following recommendations can help optimize your indexes and queries to maximize the performance benefits from additional CPUs allocated by query performance factors.

Best candidates for query performance factor improvements

• Query types:

  • Full-text

  • Tag

  • Vector

  • Numeric

  • Geo

• Result set types:

  • Small result sets

  • Document subsets that are indexed in their non-normalized form

Best practices

If query performance factors have not boosted the performance of your queries as much as expected:

1. Verify your index includes all queried and returned fields.

2. Identify and avoid query anti-patterns that limit scalability.

3. Follow best practices to improve indexing.

4. Follow best practices to improve queries.

Improve indexing

Follow these best practices for indexing:

• Include fields in the index definition that are used in the query or the required result sets (projections).

• Use SORTABLE for all fields returned in result sets.

• Use the UNF option for TAG and GEO fields.

• Use the NOSTEM option for TEXT fields.

Improve queries

Follow these best practices to optimize queries:

• Specify the result set fields in the RETURN or LOAD clauses and include them in the index definition. Don’t just return the default result set from FT.SEARCH or LOAD * from FT.AGGREGATE.

• Use LIMIT to reduce the result set size.

• Use DIALECT 3 or higher for any queries against JSON.

Index and query examples

The following examples depict an anti-pattern index schema and query, followed by a corrected schema and query, which allows for scalability with the Redis Query Engine.

Anti-pattern index schema

The following index schema is not optimized for vertical scaling:

FT.CREATE jsonidx:profiles ON JSON PREFIX 1 profiles: 
          SCHEMA $.tags.* as t NUMERIC SORTABLE 
                 $.firstName as name TEXT 
                 $.location as loc GEO

Anti-pattern query

The following query is not optimized for vertical scaling:

FT.AGGREGATE jsonidx:profiles '@t:[1299 1299]' LOAD * LIMIT 0 10

Improved index schema

Here's an improved index schema that follows best practices for vertical scaling:

FT.CREATE jsonidx:profiles ON JSON PREFIX 1 profiles: 
          SCHEMA $.tags.* as t NUMERIC SORTABLE 
                 $.firstName as name TEXT NOSTEM SORTABLE 
                 $.lastName as lastname TEXT NOSTEM SORTABLE 
                 $.location as loc GEO SORTABLE 
                 $.id as id TAG SORTABLE UNF 
                 $.ver as ver TAG SORTABLE UNF

Improved query

Here's an improved query that follows best practices for vertical scaling:

FT.AGGREGATE jsonidx:profiles '@t:[1299 1299]' 
                LOAD 6 id t name lastname loc ver
                LIMIT 0 10
                DIALECT 3

Performance results

The following benchmarks show the performance improvements for different query types achieved with query performance factors. Vector, tag, and text queries strongly benefit, while numeric and geographic queries show more limited improvements.

Vector schema type

Vector ingest

Vector query

Tag schema type

Text schema type

Two-word union queries

Two-word intersection queries

Simple one-word match

Numeric schema type

Geo schema type

Geo queries without UNF

Geo queries with UNF