How to use Redis for Query Caching

Have you ever been in a situation where your database queries are slowing down? Query caching is the technique you need to speed database queries by using different caching methods while keeping costs down! Imagine that you built an e-commerce application. It started small but is growing fast. By now, you have an extensive product catalog and millions of customers.

That's good for business, but a hardship for technology. Your queries to primary database (MongoDB/ Postgressql) are beginning to slow down, even though you already attempted to optimize them. Even though you can squeak out a little extra performance, it isn't enough to satisfy your customers.

Redis is an in-memory datastore, best known for caching. Redis allows you to reduce the load on a primary database while speeding up database reads.

With any e-commerce application, there is one specific type of query that is most often requested. If you guessed that it’s the product search query, you’re correct!

To improve product search in an e-commerce application, you can implement one of following caching patterns:

• •Cache prefetching: An entire product catalog can be pre-cached in Redis, and the application can perform any product query on Redis similar to the primary database.
• •Cache-aside pattern: Redis is filled on demand, based on whatever search parameters are requested by the frontend.

This tutorial focuses on the cache-aside pattern. The goal of this design pattern is to set up optimal caching (load-as-you-go) for better read operations. With caching, you might be familiar with a "cache miss," where you do not find data in the cache, and a "cache hit," where you can find data in the cache. Let's look at how the cache-aside pattern works with Redis for both a "cache miss" and a "cache hit."

This diagram illustrates the steps taken in the cache-aside pattern when there is a "cache miss." To understand how this works, consider the following process:

1. 1.An application requests data from the backend.
2. 2.The backend checks to find out if the data is available in Redis.
3. 3.Data is not found (a cache miss), so the data is fetched from the database.
4. 4.The data returned from the database is subsequently stored in Redis.
5. 5.The data is then returned to the application.

Now that you have seen what a "cache miss" looks like, let's cover a "cache hit." Here is the same diagram, but with the "cache hit" steps highlighted in green.

1. 1.An application requests data from the backend.
2. 2.The backend checks to find out if the data is available in Redis.
3. 3.The data is then returned to the application.

The cache-aside pattern is useful when you need to:

1. 1.Query data frequently: When you have a large volume of reads (as is the case in an e-commerce application), the cache-aside pattern gives you an immediate performance gain for subsequent data requests.
2. 2.Fill the cache on demand: The cache-aside pattern fills the cache as data is requested rather than pre-caching, thus saving on space and cost. This is useful when it isn't clear what data will need to be cached.
3. 3.Be cost-conscious: Since cache size is directly related to the cost of cache storage in the cloud, the smaller the size, the less you pay.

The e-commerce microservices application discussed in the rest of this tutorial uses the following architecture:

1. 1.products service: handles querying products from the database and returning them to the frontend
2. 2.orders service: handles validating and creating orders
3. 3.order history service: handles querying a customer's order history
4. 4.payments service: handles processing orders for payment
5. 5.digital identity service: handles storing digital identity and calculating identity score
6. 6.api gateway: unifies services under a single endpoint
7. 7.mongodb/ postgresql: serves as the primary database, storing orders, order history, products, etc.
8. 8.redis: serves as the stream processor and caching database

The e-commerce microservices application consists of a frontend, built using Next.js with TailwindCSS. The application backend uses Node.js. The data is stored in Redis and MongoDB/ Postgressql using Prisma. Below you will find screenshots of the frontend of the e-commerce app:

• •Dashboard: Shows the list of products with search functionality

Shopping Cart: Add products to the cart, then check out using the "Buy Now" button

In our sample application, the products service publishes an API for filtering products. Here's what a call to the API looks like:

The following code shows the function used to search for products in primary database:

You simply make a call to primary database (MongoDB/ Postgressql) to find products based on a filter on the product's displayName property. You can set up multiple columns for better fuzzy searching, but we simplified it for the purposes of this tutorial.

Using primary database directly without Redis works for a while, but eventually it slows down. That's why you might use Redis, to speed things up. The cache-aside pattern helps you balance performance with cost.

The basic decision tree for cache-aside is as follows.

When the frontend requests products:

1. 1.Form a hash with the contents of the request (i.e., the search parameters).
2. 2.Check Redis to see if a value exists for the hash.
3. 3.Is there a cache hit? If data is found for the hash, it is returned; the process stops here.
4. 4.Is there a cache miss? When data is not found, it is read out of primary database and subsequently stored in Redis prior to being returned.

Here’s the code used to implement the decision tree:

You now know how to use Redis for caching with one of the most common caching patterns (cache-aside). It's possible to incrementally adopt Redis wherever needed with different strategies/patterns. For more resources on the topic of microservices, check out the links below:

Microservices with Redis

• • CQRS
• • Interservice communication
• • API gateway caching

General

• •Redis YouTube channel
• •Clients like Node Redis and Redis om Node help you to use Redis in Node.js applications.
• •RedisInsight : To view your Redis data or to play with raw Redis commands in the workbench
• •Try Redis Cloud for free