Requirements to Store Huge Keys using Redis on Flash

Redis Enterprise Software
Redis Cloud
redisonflash

Last updated 18, Apr 2024

Goal

Understand the constraints on the size of values when configuring Redis on Flash. Does the RAM percentage need to be greater than the biggest single-value size? As an example, in the following scenario:

  • memory limit = 10 GB
  • RoF = 20% in RAM (2GB)
  • Key value size > 2GB (e.g. a stream)

How can Redis fetch the value from the disk and transfer it to RAM?

Solution

The requirements to make Auto tiering a viable solution for what concerns the size of the values are:

  • All the keys need to fit in the RAM
  • Whatever key is retrieved from the Flash storage needs to fit in the RAM

This means that the configured RAM needs to be bigger than the larger value of any key in the dataset. Furthermore, RAM should also have enough room to store all the keys with their overhead.

Storing large key-value pairs (or using huge Redis Streams) with Redis on Flash enabled is generally an anti-pattern.  If you have to pull a whole 1MB HASH from Flash just to set one pair inside the hash, there is an unnecessary overhead.

References

Refer to the Auto tiering feature documentation.