How to Embed Redis into Your Continuous Integration and Continuous Deployment (CI/CD) Process

Earlier this week, I wrote about how Redis can benefit distributed development teams by helping them release new features safely and roll them back with minimal impact when required. Today, I’ll dive into specific details about how feature toggles, feature context and error logs can enhance your continuous integration and continuous deployment (CI/CD) process.

Feature Toggling with Redis

“Feature toggling” is a set of patterns that help you deliver new functionality to application users rapidly but safely. Feature toggles are also referred to as feature flags, feature bits or feature flippers.

In Redis Enterprise, it’s very easy to structure a toggle strategy using the native Redis HASH data structure.

With Redis-cli or any Redis client for the language of your choice, you can create HASH keys (such as “useNewAlgorithm”) with values that set a toggle flag, release number, developer names, an override flag, etc.

Using this type of data structure, you can set up as many feature toggles in Redis as you need. Your application can then look these up at runtime to check the real-time flag status and metadata.

Using this type of data structure, you can set up as many feature toggles in Redis as you need. Your application can then look these up at runtime to check the real-time flag status and metadata.

HSET useNewAlgorithm toggle "True" releaseNum "2.5" fingerprint "John S" override "fb-user"

HGETALL useNewAlgorithm

Response:

 1) "name"
 2) "John Smith"
 3) "email"
 4) "john.smith@jsmith.com"
 5) "phone"
 6) "718-111-2222"
 7) "authType"
 8) "fb-user"
 9) "creationTime"
10) "Fri Apr 5 09:51:05 GMT-5"

Again, using the native Redis HASH data structure and Redis-cli (or any Redis client), you can store session data in keys and look it up at runtime to read real-time values.

For example, in Figure 6, the ‘sessionAuthorized’ method checks if a user has logged in with Facebook using the ‘authType” value stored in the session key.

HSET session:4156e021:1234 name "John Smith" email "john.smith@jsmith.com" phone "718-111-2222" authType "fb-user" creationTime "Fri Apr 5 09:51:05 GMT-5"

HGETALL session:4156e021:1234

Response:

 1) "name"
 2) "John Smith"
 3) "email"
 4) "john.smith@jsmith.com"
 5) "phone"
 6) "718-111-2222"
 7) "authType"
 8) "fb-user"
 9) "creationTime"
10) "Fri Apr 5 09:51:05 GMT-5"

Feel free to read up more on session stores here.

Using Redis for a Fast, Searchable Error Database

An “error database” is a centralized data store for errors reported during runtime. Errors can be stored in RediSearch, a powerful text search and secondary indexing engine, which provides a fast search for looking them up. The version of RediSearch that comes with Redis Enterprise supports scaling across many servers, allowing it to easily grow to billions of documents on hundreds of servers.

You can push any errors reported during runtime into RediSearch, along with other useful information from the feature toggle and feature context, which will help with the triage process.

Create search indexes with Redis-cli or any Redis client.

For example, let’s create a new index called “toggle_errors_db” to store all errors reported whenever a feature toggle is used.

FT.CREATE toggle_errors_db SCHEMA toggleName TEXT toggleValue TEXT fingerPrint TEXT exceptionBody TEXT

Let’s add some data to this index with a new key using the format:useNewAlgorithm:03-12-19-10-32-05

FT.ADD toggle_errors_db useNewAlgorithm:03-12-19-10-32-05 1.0 FIELDS toggleName 'useNewAlgorithm' toggleValue 'True' fingerPrint 'John S,Stephanie B' exceptionBody 'Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 5 at Exceptions.Unchecked_Demo.main(Unchecked_Demo.java:8)'

Try a search on this index for any object with keywords, e.g. search for developer name: John

FT.SEARCH toggle_errors_db John

Response:

1) (integer) 1
2) "useNewAlgorithm:03-12-19-10-32-05"
3) 1) "fingerPrint"
   2) "John S,Stephanie B"
   3) "exceptionBody"
   4) "Exception in thread \"main\" java.lang.ArrayIndexOutOfBoundsException: 5 at Exceptions.Unchecked_Demo.main(Unchecked_Demo.java:8)"
   5) "toggleValue"
   6) "True"
   7) "toggleName"
   8) "useNewAlgorithm"

Search for feature toggle name: useNewAlgorithm

FT.SEARCH toggle_errors_db useNewAlgorithm

Response:

1) (integer) 1
2) "useNewAlgorithm:03-12-19-10-32-05"
3) 1) "fingerPrint"
   2) "John S,Stephanie B"
   3) "exceptionBody"
   4) "Exception in thread \"main\" java.lang.ArrayIndexOutOfBoundsException: 5 at Exceptions.Unchecked_Demo.main(Unchecked_Demo.java:8)"
   5) "toggleValue"
   6) "True"
   7) "toggleName"
   8) "useNewAlgorithm"

Logs Database

A “logs database” is a centralized data store that tracks log messages. You can use Redis to store a recent list of log messages, which will give you a snapshot view of your logs at any time.

To keep a recent list of logs, you can LPUSH log messages to a LIST and then trim that LIST to a fixed size.

Later, if you want to read those log messages, just perform a simple LRANGE to fetch them.

SEVERITY = {
logging.DEBUG: 'debug',
logging.INFO: 'info',
logging.WARNING: 'warning',
logging.ERROR: 'error',
logging.CRITICAL: 'critical',
}
SEVERITY.update((name, name) for name in SEVERITY.values())
//Set up a mapping that should help turn most logging severity levels into something consistent
def log_recent(conn, name, message, severity=logging.INFO, pipe=None):
severity = str(SEVERITY.get(severity, severity)).lower()
//Actually try to turn a logging level into a simple string.
destination = 'recent:%s:%s'%(name, severity)
//Create the key that messages will be written to.
message = time.asctime() + ' ' + message
//Add the current time so that we know when the message was sent.
pipe = pipe or conn.pipeline()
pipe.lpush(destination, message)
//Add the message to the beginning of the log list.
pipe.ltrim(destination, 0, 99)
//Trim the log list to only include the most recent 100 messages.
pipe.execute()

Get Started with Redis Enterprise

Each of these techniques brings an opportunity to take your continuous updates to the next level and minimize the time and headaches inherent in managing development for large-scale apps with frequent releases. Of course, Redis Enterprise is a great way to bring more power and flexibility to the whole CI/CD process. Thankfully, creating a Redis cluster with Redis Enterprise Pro (which includes the RediSearch module and features data persistence) is easy and free.

Get Started with Redis Enterprise today.