Redis modules and blocking commands

Summary: How to implement blocking commands in a Redis module

Content

Redis has a few blocking commands among the built-in set of commands.
One of the most used is [`BLPOP`]() (or the symmetric [`BRPOP`]()) which blocks
waiting for elements arriving in a list.

The interesting fact about blocking commands is that they do not block
the whole server, but just the client calling them. Usually the reason to
block is that we expect some external event to happen: this can be
some change in the Redis data structures like in the [`BLPOP`]() case, a
long computation happening in a thread, to receive some data from the
network, and so forth.

Redis modules have the ability to implement blocking commands as well,
this documentation shows how the API works and describes a few patterns
that can be used in order to model blocking commands.

How blocking and resuming works.
---

_Note: You may want to check the `helloblock.c` example in the Redis source tree
inside the `src/modules` directory, for a simple to understand example
on how the blocking API is applied._

In Redis modules, commands are implemented by callback functions that
are invoked by the Redis core when the specific command is called
by the user. Normally the callback terminates its execution sending
some reply to the client. Using the following function instead, the
function implementing the module command may request that the client
is put into the blocked state:

    RedisModuleBlockedClient *RedisModule_BlockClient(RedisModuleCtx *ctx, RedisModuleCmdFunc reply_callback, RedisModuleCmdFunc timeout_callback, void (*free_privdata)(void*), long long timeout_ms);

The function returns a `RedisModuleBlockedClient` object, which is later
used in order to unblock the client. The arguments have the following
meaning:

* `ctx` is the command execution context as usually in the rest of the API.
* `reply_callback` is the callback, having the same prototype of a normal command function, that is called when the client is unblocked in order to return a reply to the client.
* `timeout_callback` is the callback, having the same prototype of a normal command function that is called when the client reached the `ms` timeout.
* `free_privdata` is the callback that is called in order to free the private data. Private data is a pointer to some data that is passed between the API used to unblock the client, to the callback that will send the reply to the client. We'll see how this mechanism works later in this document.
* `ms` is the timeout in milliseconds. When the timeout is reached, the timeout callback is called and the client is automatically aborted.

Once a client is blocked, it can be unblocked with the following API:

    int RedisModule_UnblockClient(RedisModuleBlockedClient *bc, void *privdata);

The function takes as argument the blocked client object returned by
the previous call to `RedisModule_BlockClient()`, and unblock the client.
Immediately before the client gets unblocked, the `reply_callback` function
specified when the client was blocked is called: this function will
have access to the `privdata` pointer used here.

IMPORTANT: The above function is thread safe, and can be called from within
a thread doing some work in order to implement the command that blocked
the client.

The `privdata` data will be freed automatically using the `free_privdata`
callback when the client is unblocked. This is useful **since the reply
callback may never be called** in case the client timeouts or disconnects
from the server, so it's important that it's up to an external function
to have the responsibility to free the data passed if needed.

To better understand how the API works, we can imagine writing a command
that blocks a client for one second, and then send as reply "Hello!".

Note: arity checks and other non important things are not implemented
int his command, in order to take the example simple.

    int Example_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv,
                             int argc)
    {
        RedisModuleBlockedClient *bc =
            RedisModule_BlockClient(ctx,reply_func,timeout_func,NULL,0);

        pthread_t tid;
        pthread_create(&tid,NULL,threadmain,bc);

        return REDISMODULE_OK;
    }

    void *threadmain(void *arg) {
        RedisModuleBlockedClient *bc = arg;

        sleep(1); /* Wait one second and unblock. */
        RedisModule_UnblockClient(bc,NULL);
    }

The above command blocks the client ASAP, spawning a thread that will
wait a second and will unblock the client. Let's check the reply and
timeout callbacks, which are in our case very similar, since they
just reply the client with a different reply type.

    int reply_func(RedisModuleCtx *ctx, RedisModuleString **argv,
                   int argc)
    {
        return RedisModule_ReplyWithSimpleString(ctx,"Hello!");
    }

    int timeout_func(RedisModuleCtx *ctx, RedisModuleString **argv,
                   int argc)
    {
        return RedisModule_ReplyWithNull(ctx);
    }

The reply callback just sends the "Hello!" string to the client.
The important bit here is that the reply callback is called when the
client is unblocked from the thread.

The timeout command returns `NULL`, as it often happens with actual
Redis blocking commands timing out.

Passing reply data when unblocking
---

The above example is simple to understand but lacks an important
real world aspect of an actual blocking command implementation: often
the reply function will need to know what to reply to the client,
and this information is often provided as the client is unblocked.

We could modify the above example so that the thread generates a
random number after waiting one second. You can think at it as an
actually expansive operation of some kind. Then this random number
can be passed to the reply function so that we return it to the command
caller. In order to make this working, we modify the functions as follow:

    void *threadmain(void *arg) {
        RedisModuleBlockedClient *bc = arg;

        sleep(1); /* Wait one second and unblock. */

        long *mynumber = RedisModule_Alloc(sizeof(long));
        *mynumber = rand();
        RedisModule_UnblockClient(bc,mynumber);
    }

As you can see, now the unblocking call is passing some private data,
that is the `mynumber` pointer, to the reply callback. In order to
obtain this private data, the reply callback will use the following
function:

    void *RedisModule_GetBlockedClientPrivateData(RedisModuleCtx *ctx);

So our reply callback is modified like that:

    int reply_func(RedisModuleCtx *ctx, RedisModuleString **argv,
                   int argc)
    {
        long *mynumber = RedisModule_GetBlockedClientPrivateData(ctx);
        /* IMPORTANT: don't free mynumber here, but in the
         * free privdata callback. */
        return RedisModule_ReplyWithLongLong(ctx,mynumber);
    }

Note that we also need to pass a `free_privdata` function when blocking
the client with `RedisModule_BlockClient()`, since the allocated
long value must be freed. Our callback will look like the following:

    void free_privdata(void *privdata) {
        RedisModule_Free(privdata);
    }

NOTE: It is important to stress that the private data is best freed in the
`free_privdata` callback because the reply function may not be called
if the client disconnects or timeout.

Also note that the private data is also accessible from the timeout
callback, always using the `GetBlockedClientPrivateData()` API.

Aborting the blocking of a client
---

One problem that sometimes arises is that we need to allocate resources
in order to implement the non blocking command. So we block the client,
then, for example, try to create a thread, but the thread creation function
returns an error. What to do in such a condition in order to recover? We
don't want to take the client blocked, nor we want to call `UnblockClient()`
because this will trigger the reply callback to be called.

In this case the best thing to do is to use the following function:

    int RedisModule_AbortBlock(RedisModuleBlockedClient *bc);

Practically this is how to use it:

    int Example_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv,
                             int argc)
    {
        RedisModuleBlockedClient *bc =
            RedisModule_BlockClient(ctx,reply_func,timeout_func,NULL,0);

        pthread_t tid;
        if (pthread_create(&tid,NULL,threadmain,bc) != 0) {
            RedisModule_AbortBlock(bc);
            RedisModule_ReplyWithError(ctx,"Sorry can't create a thread");
        }

        return REDISMODULE_OK;
    }

The client will be unblocked but the reply callback will not be called.

Implementing the command, reply and timeout callback using a single function
---

The following functions can be used in order to implement the reply and
callback with the same function that implements the primary command
function:

    int RedisModule_IsBlockedReplyRequest(RedisModuleCtx *ctx);
    int RedisModule_IsBlockedTimeoutRequest(RedisModuleCtx *ctx);

So I could rewrite the example command without using a separated
reply and timeout callback:

    int Example_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv,
                             int argc)
    {
        if (RedisModule_IsBlockedReplyRequest(ctx)) {
            long *mynumber = RedisModule_GetBlockedClientPrivateData(ctx);
            return RedisModule_ReplyWithLongLong(ctx,mynumber);
        } else if (RedisModule_IsBlockedTimeoutRequest) {
            return RedisModule_ReplyWithNull(ctx);
        }

        RedisModuleBlockedClient *bc =
            RedisModule_BlockClient(ctx,reply_func,timeout_func,NULL,0);

        pthread_t tid;
        if (pthread_create(&tid,NULL,threadmain,bc) != 0) {
            RedisModule_AbortBlock(bc);
            RedisModule_ReplyWithError(ctx,"Sorry can't create a thread");
        }

        return REDISMODULE_OK;
    }

Functionally is the same but there are people that will prefer the less
verbose implementation that concentrates most of the command logic in a
single function.

Working on copies of data inside a thread
---

An interesting pattern in order to work with threads implementing the
slow part of a command, is to work with a copy of the data, so that
while some operation is performed in a key, the user continues to see
the old version. However when the thread terminated its work, the
representations are swapped and the new, processed version, is used.

An example of this approach is the
[Neural Redis module](https://github.com/antirez/neural-redis)
where neural networks are trained in different threads while the
user can still execute and inspect their older versions.

Future work
---

An API is work in progress right now in order to allow Redis modules APIs
to be called in a safe way from threads, so that the threaded command
can access the data space and do incremental operations.

There is no ETA for this feature but it may appear in the course of the
Redis 4.0 release at some point.


Modules API for native types

Summary: How to use native types in a Redis module

Overview

Redis modules can access Redis built-in data structures both at high level,
by calling Redis commands, and at low level, by manipulating the data structures
directly.

By using these capabilities in order to build new abstractions on top of existing
Redis data structures, or by using strings DMA in order to encode modules
data structures into Redis strings, it is possible to create modules that
*feel like* they are exporting new data types. However, for more complex
problems, this is not enough, and the implementation of new data structures
inside the module is needed.

We call the ability of Redis modules to implement new data structures that
feel like native Redis ones **native types support**. This document describes
the API exported by the Redis modules system in order to create new data
structures and handle the serialization in RDB files, the rewriting process
in AOF, the type reporting via the [`TYPE`]() command, and so forth.

Overview of native types

A module exporting a native type is composed of the following main parts:

* The implementation of some kind of new data structure and of commands operating on the new data structure.
* A set of callbacks that handle: RDB saving, RDB loading, AOF rewriting, releasing of a value associated with a key, calculation of a value digest (hash) to be used with the `DEBUG DIGEST` command.
* A 9 characters name that is unique to each module native data type.
* An encoding version, used to persist into RDB files a module-specific data version, so that a module will be able to load older representations from RDB files.

While to handle RDB loading, saving and AOF rewriting may look complex as a first glance, the modules API provide very high level function for handling all this, without requiring the user to handle read/write errors, so in practical terms, writing a new data structure for Redis is a simple task.

A **very easy** to understand but complete example of native type implementation
is available inside the Redis distribution in the `/modules/hellotype.c` file.
The reader is encouraged to read the documentation by looking at this example
implementation to see how things are applied in the practice.

Register a new data type

In order to register a new native type into the Redis core, the module needs
to declare a global variable that will hold a reference to the data type.
The API to register the data type will return a data type reference that will
be stored in the global variable.

    static RedisModuleType *MyType;
    #define MYTYPE_ENCODING_VERSION 0

    int RedisModule_OnLoad(RedisModuleCtx *ctx) {
	RedisModuleTypeMethods tm = {
	    .version = REDISMODULE_TYPE_METHOD_VERSION,
	    .rdb_load = MyTypeRDBLoad,
	    .rdb_save = MyTypeRDBSave,
	    .aof_rewrite = MyTypeAOFRewrite,
	    .free = MyTypeFree
	};

        MyType = RedisModule_CreateDataType(ctx, "MyType-AZ",
		MYTYPE_ENCODING_VERSION, &tm);
        if (MyType == NULL) return REDISMODULE_ERR;
    }

As you can see from the example above, a single API call is needed in order to
register the new type. However a number of function pointers are passed as
arguments. Certain are optionals while some are mandatory. The above set
of methods *must* be passed, while `.digest` and `.mem_usage` are optional
and are currently not actually supported by the modules internals, so for
now you can just ignore them.

The `ctx` argument is the context that we receive in the `OnLoad` function.
The type `name` is a 9 character name in the character set that includes
from `A-Z`, `a-z`, `0-9`, plus the underscore `_` and minus `-` characters.

Note that **this name must be unique** for each data type in the Redis
ecosystem, so be creative, use both lower-case and upper case if it makes
sense, and try to use the convention of mixing the type name with the name
of the author of the module, to create a 9 character unique name.

**NOTE:** It is very important that the name is exactly 9 chars or the
registration of the type will fail. Read more to understand why.

For example if I'm building a *b-tree* data structure and my name is *antirez*
I'll call my type **btree1-az**. The name, converted to a 64 bit integer,
is stored inside the RDB file when saving the type, and will be used when the
RDB data is loaded in order to resolve what module can load the data. If Redis
finds no matching module, the integer is converted back to a name in order to
provide some clue to the user about what module is missing in order to load
the data.

The type name is also used as a reply for the [`TYPE`]() command when called
with a key holding the registered type.

The `encver` argument is the encoding version used by the module to store data
inside the RDB file. For example I can start with an encoding version of 0,
but later when I release version 2.0 of my module, I can switch encoding to
something better. The new module will register with an encoding version of 1,
so when it saves new RDB files, the new version will be stored on disk. However
when loading RDB files, the module `rdb_load` method will be called even if
there is data found for a different encoding version (and the encoding version
is passed as argument to `rdb_load`), so that the module can still load old
RDB files.

The last argument is a structure used in order to pass the type methods to the
registration function: `rdb_load`, `rdb_save`, `aof_rewrite`, `digest` and
`free` and `mem_usage` are all callbacks with the following prototypes and uses:

    typedef void *(*RedisModuleTypeLoadFunc)(RedisModuleIO *rdb, int encver);
    typedef void (*RedisModuleTypeSaveFunc)(RedisModuleIO *rdb, void *value);
    typedef void (*RedisModuleTypeRewriteFunc)(RedisModuleIO *aof, RedisModuleString *key, void *value);
    typedef size_t (*RedisModuleTypeMemUsageFunc)(void *value);
    typedef void (*RedisModuleTypeDigestFunc)(RedisModuleDigest *digest, void *value);
    typedef void (*RedisModuleTypeFreeFunc)(void *value);

* `rdb_load` is called when loading data from the RDB file. It loads data in the same format as `rdb_save` produces.
* `rdb_save` is called when saving data to the RDB file.
* `aof_rewrite` is called when the AOF is being rewritten, and the module needs to tell Redis what is the sequence of commands to recreate the content of a given key.
* `digest` is called when `DEBUG DIGEST` is executed and a key holding this module type is found. Currently this is not yet implemented so the function ca be left empty.
* `mem_usage` is called when the [`MEMORY`]() command asks for the total memory consumed by a specific key, and is used in order to get the amount of bytes used by the module value.
* `free` is called when a key with the module native type is deleted via [`DEL`]() or in any other mean, in order to let the module reclaim the memory associated with such a value.

Why module types require nine character names

When Redis persists to RDB files, modules specific data types require to
be persisted as well. Now RDB files are sequences of key-value pairs
like the following:

    [1 byte type] [key] [a type specific value]

The 1 byte type identifies strings, lists, sets, and so forth. In the case
of modules data, it is set to a special value of `module data`, but of
course this is not enough, we need the information needed to link a specific
value with a specific module type that is able to load and handle it.

So when we save a `type specific value` about a module, we prefix it with
a 64 bit integer. 64 bits is large enough to store the information needed
in order to lookup the module that can handle that specific type, but is
short enough that we can prefix each module value we store inside the RDB
without making the final RDB file too big. At the same time, this solution
of prefixing the value with a 64 bit *signature* does not require to do
strange things like defining in the RDB header a list of modules specific
types. Everything is pretty simple.

So, what you can store in 64 bits in order to identify a given module in
a reliable way? Well if you build a character set of 64 symbols, you can
easily store 9 characters of 6 bits, and you are left with 10 bits, that
are used in order to store the *encoding version* of the type, so that
the same type can evolve in the future and provide a different and more
efficient or updated serialization format for RDB files.

So the 64 bit prefix stored before each module value is like the following:

    6|6|6|6|6|6|6|6|6|10

The first 9 elements are 6-bits characters, the final 10 bits is the
encoding version.

When the RDB file is loaded back, it reads the 64 bit value, masks the final
10 bits, and searches for a matching module in the modules types cache.
When a matching one is found, the method to load the RDB file value is called
with the 10 bits encoding version as argument, so that the module knows
what version of the data layout to load, if it can support multiple versions.

Now the interesting thing about all this is that, if instead the module type
cannot be resolved, since there is no loaded module having this signature,
we can convert back the 64 bit value into a 9 characters name, and print
an error to the user that includes the module type name! So that she or he
immediately realizes what's wrong.

Set and get keys

After registering our new data type in the `RedisModule_OnLoad()` function,
we also need to be able to set Redis keys having as value our native type.

This normally happens in the context of commands that write data to a key.
The native types API allow to set and get keys to module native data types,
and to test if a given key is already associated to a value of a specific data
type.

The API uses the normal modules `RedisModule_OpenKey()` low level key access
interface in order to deal with this. This is an example of setting a
native type private data structure to a Redis key:

    RedisModuleKey *key = RedisModule_OpenKey(ctx,keyname,REDISMODULE_WRITE);
    struct some_private_struct *data = createMyDataStructure();
    RedisModule_ModuleTypeSetValue(key,MyType,data);

The function `RedisModule_ModuleTypeSetValue()` is used with a key handle open
for writing, and gets three arguments: the key handle, the reference to the
native type, as obtained during the type registration, and finally a `void*`
pointer that contains the private data implementing the module native type.

Note that Redis has no clues at all about what your data contains. It will
just call the callbacks you provided during the method registration in order
to perform operations on the type.

Similarly we can retrieve the private data from a key using this function:

    struct some_private_struct *data;
    data = RedisModule_ModuleTypeGetValue(key);

We can also test for a key to have our native type as value:

    if (RedisModule_ModuleTypeGetType(key) == MyType) {
        /* ... do something ... */
    }

However for the calls to do the right thing, we need to check if the key
is empty, if it contains a value of the right kind, and so forth. So
the idiomatic code to implement a command writing to our native type
is along these lines:

    RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
        REDISMODULE_READ|REDISMODULE_WRITE);
    int type = RedisModule_KeyType(key);
    if (type != REDISMODULE_KEYTYPE_EMPTY &&
        RedisModule_ModuleTypeGetType(key) != MyType)
    {
        return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
    }

Then if we successfully verified the key is not of the wrong type, and
we are going to write to it, we usually want to create a new data structure if
the key is empty, or retrieve the reference to the value associated to the
key if there is already one:

    /* Create an empty value object if the key is currently empty. */
    struct some_private_struct *data;
    if (type == REDISMODULE_KEYTYPE_EMPTY) {
        data = createMyDataStructure();
        RedisModule_ModuleTypeSetValue(key,MyTyke,data);
    } else {
        data = RedisModule_ModuleTypeGetValue(key);
    }
    /* Do something with 'data'... */

Free method

As already mentioned, when Redis needs to free a key holding a native type
value, it needs help from the module in order to release the memory. This
is the reason why we pass a `free` callback during the type registration:

    typedef void (*RedisModuleTypeFreeFunc)(void *value);

A trivial implementation of the free method can be something like this,
assuming our data structure is composed of a single allocation:

    void MyTypeFreeCallback(void *value) {
        RedisModule_Free(value);
    }

However a more real world one will call some function that performs a more
complex memory reclaiming, by casting the void pointer to some structure
and freeing all the resources composing the value.

RDB load and save methods

The RDB saving and loading callbacks need to create (and load back) a
representation of the data type on disk. Redis offers a high level API
that can automatically store inside the RDB file the following types:

* Unsigned 64 bit integers.
* Signed 64 bit integers.
* Doubles.
* Strings.

It is up to the module to find a viable representation using the above base
types. However note that while the integer and double values are stored
and loaded in an architecture and *endianness* agnostic way, if you use
the raw string saving API to, for example, save a structure on disk, you
have to care those details yourself.

This is the list of functions performing RDB saving and loading:

    void RedisModule_SaveUnsigned(RedisModuleIO *io, uint64_t value);
    uint64_t RedisModule_LoadUnsigned(RedisModuleIO *io);
    void RedisModule_SaveSigned(RedisModuleIO *io, int64_t value);
    int64_t RedisModule_LoadSigned(RedisModuleIO *io);
    void RedisModule_SaveString(RedisModuleIO *io, RedisModuleString *s);
    void RedisModule_SaveStringBuffer(RedisModuleIO *io, const char *str, size_t len);
    RedisModuleString *RedisModule_LoadString(RedisModuleIO *io);
    char *RedisModule_LoadStringBuffer(RedisModuleIO *io, size_t *lenptr);
    void RedisModule_SaveDouble(RedisModuleIO *io, double value);
    double RedisModule_LoadDouble(RedisModuleIO *io);

The functions don't require any error checking from the module, that can
always assume calls succeed.

As an example, imagine I've a native type that implements an array of
double values, with the following structure:

    struct double_array {
        size_t count;
        double *values;
    };

My `rdb_save` method may look like the following:

    void DoubleArrayRDBSave(RedisModuleIO *io, void *ptr) {
        struct dobule_array *da = ptr;
        RedisModule_SaveUnsigned(io,da->count);
        for (size_t j = 0; j < da->count; j++)
            RedisModule_SaveDouble(io,da->values[j]);
    }

What we did was to store the number of elements followed by each double
value. So when later we'll have to load the structure in the `rdb_load`
method we'll do something like this:

    void *DoubleArrayRDBLoad(RedisModuleIO *io, int encver) {
        if (encver != DOUBLE_ARRAY_ENC_VER) {
            /* We should actually log an error here, or try to implement
               the ability to load older versions of our data structure. */
            return NULL;
        }

        struct double_array *da;
        da = RedisModule_Alloc(sizeof(*da));
        da->count = RedisModule_LoadUnsigned(io);
        da->values = RedisModule_Alloc(da->count * sizeof(double));
        for (size_t j = 0; j < da->count; j++)
            da->values[j] = RedisModule_LoadDouble(io);
        return da;
    }

The load callback just reconstruct back the data structure from the data
we stored in the RDB file.

Note that while there is no error handling on the API that writes and reads
from disk, still the load callback can return NULL on errors in case what
it reads does not look correct. Redis will just panic in that case.

AOF rewriting

void RedisModule_EmitAOF(RedisModuleIO *io, const char *cmdname, const char *fmt, ...);

Allocate memory

Modules data types should try to use `RedisModule_Alloc()` functions family
in order to allocate, reallocate and release heap memory used to implement the native data structures (see the other Redis Modules documentation for detailed information).

This is not just useful in order for Redis to be able to account for the memory used by the module, but there are also more advantages:

* Redis uses the `jemalloc` allocator, that often prevents fragmentation problems that could be caused by using the libc allocator.
* When loading strings from the RDB file, the native types API is able to return strings allocated directly with `RedisModule_Alloc()`, so that the module can directly link this memory into the data structure representation, avoiding a useless copy of the data.

Even if you are using external libraries implementing your data structures, the
allocation functions provided by the module API is exactly compatible with
`malloc()`, `realloc()`, `free()` and `strdup()`, so converting the libraries
in order to use these functions should be trivial.

In case you have an external library that uses libc `malloc()`, and you want
to avoid replacing manually all the calls with the Redis Modules API calls,
an approach could be to use simple macros in order to replace the libc calls
with the Redis API calls. Something like this could work:

    #define malloc RedisModule_Alloc
    #define realloc RedisModule_Realloc
    #define free RedisModule_Free
    #define strdup RedisModule_Strdup

However take in mind that mixing libc calls with Redis API calls will result
into troubles and crashes, so if you replace calls using macros, you need to
make sure that all the calls are correctly replaced, and that the code with
the substituted calls will never, for example, attempt to call
`RedisModule_Free()` with a pointer allocated using libc `malloc()`.


Redis serialization protocol specification

Summary: Redis serialization protocol (RESP) is the wire protocol that clients implement

Overview

To communicate with the Redis server, Redis clients use a protocol called Redis Serialization Protocol (RESP).
While the protocol was designed specifically for Redis, you can use it for other client-server software projects.

RESP is a compromise among the following considerations:

* Simple to implement.
* Fast to parse.
* Human readable.

RESP can serialize different data types including integers, strings, and arrays.
It also features an error-specific type.
A client sends a request to the Redis server as an array of strings.
The array's contents are the command and its arguments that the server should execute.
The server's reply type is command-specific.

RESP is binary-safe and uses prefixed length to transfer bulk data so it does not require processing bulk data transferred from one process to another.

RESP is the protocol you should implement in your Redis client.

The protocol outlined here is used only for client-server communication.
[Redis Cluster]() uses a different binary protocol for exchanging messages between nodes.

RESP versions

Support for the first version of the RESP protocol was introduced in Redis 1.2.
Using RESP with Redis 1.2 was optional and had mainly served the purpose of working the kinks out of the protocol.

In Redis 2.0, the protocol's next version, a.k.a RESP2, became the standard communication method for clients with the Redis server.

[RESP3](https://github.com/redis/redis-specifications/blob/master/protocol/RESP3.md) is mostly a superset of RESP2 that mainly aims to make a client author's life a little bit easier.
Redis 6.0 introduced experimental opt-in support of RESP3's features (excluding streaming strings and streaming aggregates).
In addition, the introduction of the [`HELLO`]() command allows clients to handshake and upgrade the connection's protocol version (see [Client handshake](#client-handshake)).

From Redis version 7 and forward, both RESP2 and RESP3 clients can invoke all core commands.
However, commands may return differently typed replies for different protocol versions.
Each command has descriptions of RESP2 and RESP3 return values that you can reference.

Future versions of Redis may change the default protocol version, but it is unlikely that RESP2 will become entirely deprecated.
It is possible, however, that new features in upcoming versions will require the use of RESP3.

Network layer

A client connects to a Redis server by creating a TCP connection to its port (the default is 6379).

While RESP is technically non-TCP specific, the protocol is used exclusively with TCP connections (or equivalent stream-oriented connections like Unix sockets) in the context of Redis.

Request-Response model

The Redis server accepts commands composed of different arguments.
Then, the server processes the command and sends the reply back to the client.

This is the simplest model possible; however, there are some exceptions:

* Redis requests can be [pipelined](#multiple-commands-and-pipelining).
  Pipelining enables clients to send multiple commands at once and wait for replies later.
* When a RESP2 connection subscribes to a [Pub/Sub]() channel, the protocol changes semantics and becomes a *push* protocol.
  The client no longer requires sending commands because the server will automatically send new messages to the client (for the channels the client is subscribed to) as soon as they are received.
* The [`MONITOR`]() command.
  Invoking the [`MONITOR`]() command switches the connection to an ad-hoc push mode.
  The protocol of this mode is not specified but is obvious to parse.
* [Protected mode]().
  Connections opened from a non-loopback address to a Redis while in protected mode are denied and terminated by the server.
  Before terminating the connection, Redis unconditionally sends a `-DENIED` reply, regardless of whether the client writes to the socket.
* The [RESP3 Push type](#resp3-pushes).
  As the name suggests, a push type allows the server to send out-of-band data to the connection.
  The server may push data at any time, and the data isn't necessarily related to specific commands executed by the client.

Excluding these exceptions, the Redis protocol is a simple request-response protocol.

RESP protocol description

RESP is essentially a serialization protocol that supports several data types.
In RESP, the first byte of data determines its type.

Redis generally uses RESP as a [request-response](#request-response-model) protocol in the following way:

* Clients send commands to a Redis server as an [array](#arrays) of [bulk strings](#bulk-strings).
  The first (and sometimes also the second) bulk string in the array is the command's name.
  Subsequent elements of the array are the arguments for the command.
* The server replies with a RESP type.
  The reply's type is determined by the command's implementation and possibly by the client's protocol version.

RESP is a binary protocol that uses control sequences encoded in standard ASCII.
The `A` character, for example, is encoded with the binary byte of value 65.
Similarly, the characters CR (`\r`), LF (`\n`) and SP (` `) have binary byte values of 13, 10 and 32, respectively.

The `\r\n` (CRLF) is the protocol's _terminator_, which **always** separates its parts.

The first byte in an RESP-serialized payload always identifies its type.
Subsequent bytes constitute the type's contents.

We categorize every RESP data type as either _simple_, _bulk_ or _aggregate_.

Simple types are similar to scalars in programming languages that represent plain literal values. Booleans and Integers are such examples.

RESP strings are either _simple_ or _bulk_.
Simple strings never contain carriage return (`\r`) or line feed (`\n`) characters.
Bulk strings can contain any binary data and may also be referred to as _binary_ or _blob_.
Note that bulk strings may be further encoded and decoded, e.g. with a wide multi-byte encoding, by the client.

Aggregates, such as Arrays and Maps, can have varying numbers of sub-elements and nesting levels.

The following table summarizes the RESP data types that Redis supports:

| RESP data type | Minimal protocol version | Category | First byte |
| --- | --- | --- | --- |
| [Simple strings](#simple-strings) | RESP2 | Simple | `+` |
| [Simple Errors](#simple-errors) | RESP2 | Simple | `-` |
| [Integers](#integers) | RESP2 | Simple | `:` |
| [Bulk strings](#bulk-strings) | RESP2 | Aggregate | `$` |
| [Arrays](#arrays) | RESP2 | Aggregate | `*` |
| [Nulls](#nulls) | RESP3 | Simple | `_` |
| [Booleans](#booleans) | RESP3 | Simple | `#` |
| [Doubles](#doubles) | RESP3 | Simple | `,` |
| [Big numbers](#big-numbers) | RESP3 | Simple | `(` |
| [Bulk errors](#bulk-errors) | RESP3 | Aggregate | `!` |
| [Verbatim strings](#verbatim-strings) | RESP3 | Aggregate | `=` |
| [Maps](#maps) | RESP3 | Aggregate | `%` |
| [Attributes](#attributes) | RESP3 | Aggregate | | |
| [Sets](#sets) | RESP3 | Aggregate | `~` |
| [Pushes](#pushes) | RESP3 | Aggregate | `>` |

Simple strings

Simple strings are encoded as a plus (`+`) character, followed by a string.
The string mustn't contain a CR (`\r`) or LF (`\n`) character and is terminated by CRLF (i.e., `\r\n`).

Simple strings transmit short, non-binary strings with minimal overhead.
For example, many Redis commands reply with just "OK" on success.
The encoding of this Simple String is the following 5 bytes:

    +OK\r\n

When Redis replies with a simple string, a client library should return to the caller a string value composed of the first character after the `+` up to the end of the string, excluding the final CRLF bytes.

To send binary strings, use [bulk strings](#bulk-strings) instead.

Simple errors

RESP has specific data types for errors.
Simple errors, or simply just errors, are similar to [simple strings](#simple-strings), but their first character is the minus (`-`) character.
The difference between simple strings and errors in RESP is that clients should treat errors as exceptions, whereas the string encoded in the error type is the error message itself.

The basic format is:

    -Error message\r\n

Redis replies with an error only when something goes wrong, for example, when you try to operate against the wrong data type, or when the command does not exist.
The client should raise an exception when it receives an Error reply.

The following are examples of error replies:

    -ERR unknown command 'asdf'
    -WRONGTYPE Operation against a key holding the wrong kind of value

The first upper-case word after the `-`, up to the first space or newline, represents the kind of error returned.
This word is called an _error prefix_.
Note that the error prefix is a convention used by Redis rather than part of the RESP error type.

For example, in Redis, `ERR` is a generic error, whereas `WRONGTYPE` is a more specific error that implies that the client attempted an operation against the wrong data type.
The error prefix allows the client to understand the type of error returned by the server without checking the exact error message.

A client implementation can return different types of exceptions for various errors, or provide a generic way for trapping errors by directly providing the error name to the caller as a string.

However, such a feature should not be considered vital as it is rarely useful.
Also, simpler client implementations can return a generic error value, such as `false`.

Integers

This type is a CRLF-terminated string that represents a signed, base-10, 64-bit integer.

RESP encodes integers in the following way:

    :[<+|->]<value>\r\n

* The colon (`:`) as the first byte.
* An optional plus (`+`) or minus (`-`) as the sign.
* One or more decimal digits (`0`..`9`) as the integer's unsigned, base-10 value.
* The CRLF terminator.

For example, `:0\r\n` and `:1000\r\n` are integer replies (of zero and one thousand, respectively).

Many Redis commands return RESP integers, including [`INCR`](), [`LLEN`](), and [`LASTSAVE`]().
An integer, by itself, has no special meaning other than in the context of the command that returned it.
For example, it is an incremental number for [`INCR`](), a UNIX timestamp for [`LASTSAVE`](), and so forth.
However, the returned integer is guaranteed to be in the range of a signed 64-bit integer.

In some cases, integers can represent true and false Boolean values.
For instance, [`SISMEMBER`]() returns 1 for true and 0 for false.

Other commands, including [`SADD`](), [`SREM`](), and [`SETNX`](), return 1 when the data changes and 0 otherwise.

Bulk strings

A bulk string represents a single binary string.
The string can be of any size, but by default, Redis limits it to 512 MB (see the `proto-max-bulk-len` configuration directive).

RESP encodes bulk strings in the following way:

    $<length>\r\n<data>\r\n

* The dollar sign (`$`) as the first byte.
* One or more decimal digits (`0`..`9`) as the string's length, in bytes, as an unsigned, base-10 value.
* The CRLF terminator.
* The data.
* A final CRLF.

So the string "hello" is encoded as follows:

    $5\r\nhello\r\n

The empty string's encoding is:

    $0\r\n\r\n

#### Null bulk strings
Whereas RESP3 has a dedicated data type for [null values](#nulls), RESP2 has no such type.
Instead, due to historical reasons, the representation of null values in RESP2 is via predetermined forms of the [bulk strings](#bulk-strings) and [arrays](#arrays) types.

The null bulk string represents a non-existing value.
The [`GET`]() command returns the Null Bulk String when the target key doesn't exist.

It is encoded as a bulk string with the length of negative one (-1), like so:

    $-1\r\n

A Redis client should return a nil object when the server replies with a null bulk string rather than the empty string.
For example, a Ruby library should return `nil` while a C library should return `NULL` (or set a special flag in the reply object).

Arrays

Clients send commands to the Redis server as RESP arrays.
Similarly, some Redis commands that return collections of elements use arrays as their replies.
An example is the [`LRANGE`]() command that returns elements of a list.

RESP Arrays' encoding uses the following format:

    *<number-of-elements>\r\n<element-1>...<element-n>

* An asterisk (`*`) as the first byte.
* One or more decimal digits (`0`..`9`) as the number of elements in the array as an unsigned, base-10 value.
* The CRLF terminator.
* An additional RESP type for every element of the array.

So an empty Array is just the following:

    *0\r\n

Whereas the encoding of an array consisting of the two bulk strings "hello" and "world" is:

    *2\r\n$5\r\nhello\r\n$5\r\nworld\r\n

As you can see, after the `*<count>CRLF` part prefixing the array, the other data types that compose the array are concatenated one after the other.
For example, an Array of three integers is encoded as follows:

    *3\r\n:1\r\n:2\r\n:3\r\n

Arrays can contain mixed data types.
For instance, the following encoding is of a list of four integers and a bulk string:

    *5\r\n
    :1\r\n
    :2\r\n
    :3\r\n
    :4\r\n
    $5\r\n
    hello\r\n

(The raw RESP encoding is split into multiple lines for readability).

The first line the server sent is `*5\r\n`.
This numeric value tells the client that five reply types are about to follow it.
Then, every successive reply constitutes an element in the array.

All of the aggregate RESP types support nesting.
For example, a nested array of two arrays is encoded as follows:

    *2\r\n
    *3\r\n
    :1\r\n
    :2\r\n
    :3\r\n
    *2\r\n
    +Hello\r\n
    -World\r\n

(The raw RESP encoding is split into multiple lines for readability).

The above encodes a two-element array.
The first element is an array that, in turn, contains three integers (1, 2, 3).
The second element is another array containing a simple string and an error.

In some places, the RESP Array type may be referred to as _multi bulk_.
The two are the same.

#### Null arrays
Whereas RESP3 has a dedicated data type for [null values](#nulls), RESP2 has no such type. Instead, due to historical reasons, the representation of null values in RESP2 is via predetermined forms of the [Bulk Strings](#bulk-strings) and [arrays](#arrays) types.

Null arrays exist as an alternative way of representing a null value.
For instance, when the [`BLPOP`]() command times out, it returns a null array.

The encoding of a null array is that of an array with the length of -1, i.e.:

    *-1\r\n

When Redis replies with a null array, the client should return a null object rather than an empty array.
This is necessary to distinguish between an empty list and a different condition (for instance, the timeout condition of the [`BLPOP`]() command).

#### Null elements in arrays
Single elements of an array may be [null bulk string](#null-bulk-strings).
This is used in Redis replies to signal that these elements are missing and not empty strings. This can happen, for example, with the [`SORT`]() command when used with the `GET pattern` option
if the specified key is missing.

Here's an example of an array reply containing a null element:

    *3\r\n
    $5\r\n
    hello\r\n
    $-1\r\n
    $5\r\n
    world\r\n

Above, the second element is null.
The client library should return to its caller something like this:

    ["hello",nil,"world"]

Nulls

The null data type represents non-existent values.

Nulls' encoding is the underscore (`_`) character, followed by the CRLF terminator (`\r\n`).
Here's Null's raw RESP encoding:

    _\r\n

Due to historical reasons, RESP2 features two specially crafted values for representing null values of bulk strings and arrays.
This duality has always been a redundancy that added zero semantical value to the protocol itself.

The null type, introduced in RESP3, aims to fix this wrong.

<a name="boolean-reply">

Booleans

RESP booleans are encoded as follows:

    #<t|f>\r\n

* The octothorpe character (`#`) as the first byte.
* A `t` character for true values, or an `f` character for false ones.
* The CRLF terminator.

Doubles

The Double RESP type encodes a double-precision floating point value.
Doubles are encoded as follows:

    ,[<+|->]<integral>[.<fractional>][<E|e>[sign]<exponent>]\r\n

* The comma character (`,`) as the first byte.
* An optional plus (`+`) or minus (`-`) as the sign.
* One or more decimal digits (`0`..`9`) as an unsigned, base-10 integral value.
* An optional dot (`.`), followed by one or more decimal digits (`0`..`9`) as an unsigned, base-10 fractional value.
* An optional capital or lowercase letter E (`E` or `e`), followed by an optional plus (`+`) or minus (`-`) as the exponent's sign, ending with one or more decimal digits (`0`..`9`) as an unsigned, base-10 exponent value.
* The CRLF terminator.

Here's the encoding of the number 1.23:

    ,1.23\r\n

Because the fractional part is optional, the integer value of ten (10) can, therefore, be RESP-encoded both as an integer as well as a double:

    :10\r\n
    ,10\r\n

In such cases, the Redis client should return native integer and double values, respectively, providing that these types are supported by the language of its implementation.

The positive infinity, negative infinity and NaN values are encoded as follows:

    ,inf\r\n
    ,-inf\r\n
    ,nan\r\n

Big numbers

This type can encode integer values outside the range of signed 64-bit integers.

Big numbers use the following encoding:

    ([+|-]<number>\r\n

* The left parenthesis character (`(`) as the first byte.
* An optional plus (`+`) or minus (`-`) as the sign.
* One or more decimal digits (`0`..`9`) as an unsigned, base-10 value.
* The CRLF terminator.

Example:

    (3492890328409238509324850943850943825024385\r\n

Big numbers can be positive or negative but can't include fractionals.
Client libraries written in languages with a big number type should return a big number.
When big numbers aren't supported, the client should return a string and, when possible, signal to the caller that the reply is a big integer (depending on the API used by the client library).

Bulk errors

This type combines the purpose of [simple errors](#simple-errors) with the expressive power of [bulk strings](#bulk-strings).

It is encoded as:

    !<length>\r\n<error>\r\n

* An exclamation mark (`!`) as the first byte.
* One or more decimal digits (`0`..`9`) as the error's length, in bytes, as an unsigned, base-10 value.
* The CRLF terminator.
* The error itself.
* A final CRLF.

As a convention, the error begins with an uppercase (space-delimited) word that conveys the error message.

For instance, the error "SYNTAX invalid syntax" is represented by the following protocol encoding:

    !21\r\n
    SYNTAX invalid syntax\r\n

(The raw RESP encoding is split into multiple lines for readability).

<a name="verbatim-string-reply">

Verbatim strings

This type is similar to the [bulk string](#bulk-strings), with the addition of providing a hint about the data's encoding.

A verbatim string's RESP encoding is as follows:

    =<length>\r\n<encoding>:<data>\r\n

* An equal sign (`=`) as the first byte.
* One or more decimal digits (`0`..`9`) as the string's total length, in bytes, as an unsigned, base-10 value.
* The CRLF terminator.
* Exactly three (3) bytes represent the data's encoding.
* The colon (`:`) character separates the encoding and data.
* The data.
* A final CRLF.

Example:

    =15\r\n
    txt:Some string\r\n

(The raw RESP encoding is split into multiple lines for readability).

Some client libraries may ignore the difference between this type and the string type and return a native string in both cases.
However, interactive clients, such as command line interfaces (e.g., [`redis-cli`]()), can use this type and know that their output should be presented to the human user as is and without quoting the string.

For example, the Redis command [`INFO`]() outputs a report that includes newlines.
When using RESP3, `redis-cli` displays it correctly because it is sent as a Verbatim String reply (with its three bytes being "txt").
When using RESP2, however, the `redis-cli` is hard-coded to look for the [`INFO`]() command to ensure its correct display to the user.

Maps

The RESP map encodes a collection of key-value tuples, i.e., a dictionary or a hash.

It is encoded as follows:

    %<number-of-entries>\r\n<key-1><value-1>...<key-n><value-n>

* A percent character (`%`) as the first byte.
* One or more decimal digits (`0`..`9`) as the number of entries, or key-value tuples, in the map as an unsigned, base-10 value.
* The CRLF terminator.
* Two additional RESP types for every key and value in the map.

For example, the following JSON object:

    {
        "first": 1,
        "second": 2
    }

Can be encoded in RESP like so:

    %2\r\n
    +first\r\n
    :1\r\n
    +second\r\n
    :2\r\n

(The raw RESP encoding is split into multiple lines for readability).

Both map keys and values can be any of RESP's types.

Redis clients should return the idiomatic dictionary type that their language provides.
However, low-level programming languages (such as C, for example) will likely return an array along with type information that indicates to the caller that it is a dictionary.

RESP2 doesn't have a map type.
A map in RESP2 is represented by a flat array containing the keys and the values.
The first element is a key, followed by the corresponding value, then the next key and so on, like this:
`key1, value1, key2, value2, ...`.

Attributes

The attribute type is exactly like the Map type, but instead of a `%` character as the first byte, the `|` character is used. Attributes describe a dictionary exactly like the Map type. However the client should not consider such a dictionary part of the reply, but as auxiliary data that augments the reply.

Note: in the examples below, indentation is shown only for clarity; the additional whitespace would not be part of a real reply.

For example, newer versions of Redis may include the ability to report the popularity of keys for every executed command. The reply to the command `MGET a b` may be the following:

    |1\r\n
        +key-popularity\r\n
        %2\r\n
            $1\r\n
            a\r\n
            ,0.1923\r\n
            $1\r\n
            b\r\n
            ,0.0012\r\n
    *2\r\n
        :2039123\r\n
        :9543892\r\n

The actual reply to `MGET` is just the two item array `[2039123, 9543892]`. The returned attributes specify the popularity, or frequency of requests, given as floating point numbers ranging from `0.0` to `1.0`, of the keys mentioned in the original command. Note: the actual implementation in Redis may differ.

When a client reads a reply and encounters an attribute type, it should read the attribute, and continue reading the reply. The attribute reply should be accumulated separately, and the user should have a way to access such attributes. For instance, if we imagine a session in an higher level language, something like this could happen:

[code example]

Attributes can appear anywhere before a valid part of the protocol identifying a given type, and supply information only about the part of the reply that immediately follows. For example:

    *3\r\n
        :1\r\n
        :2\r\n
        |1\r\n
            +ttl\r\n
            :3600\r\n
        :3\r\n

In the above example the third element of the array has associated auxiliary information of `{ttl:3600}`. Note that it's not up to the client library to interpret the attributes, but it should pass them to the caller in a sensible way.

Sets

Sets are somewhat like [Arrays](#arrays) but are unordered and should only contain unique elements.

RESP set's encoding is:

    ~<number-of-elements>\r\n<element-1>...<element-n>

* A tilde (`~`) as the first byte.
* One or more decimal digits (`0`..`9`) as the number of elements in the set as an unsigned, base-10 value.
* The CRLF terminator.
* An additional RESP type for every element of the Set.

Clients should return the native set type if it is available in their programming language.
Alternatively, in the absence of a native set type, an array coupled with type information can be used (in C, for example).

Pushes

RESP's pushes contain out-of-band data.
They are an exception to the protocol's request-response model and provide a generic _push mode_ for connections.

Push events are encoded similarly to [arrays](#arrays), differing only in their first byte:

    ><number-of-elements>\r\n<element-1>...<element-n>

* A greater-than sign (`>`) as the first byte.
* One or more decimal digits (`0`..`9`) as the number of elements in the message as an unsigned, base-10 value.
* The CRLF terminator.
* An additional RESP type for every element of the push event.

Pushed data may precede or follow any of RESP's data types but never inside them.
That means a client won't find push data in the middle of a map reply, for example.
It also means that pushed data may appear before or after a command's reply, as well as by itself (without calling any command).

Clients should react to pushes by invoking a callback that implements their handling of the pushed data.

Client handshake

New RESP connections should begin the session by calling the [`HELLO`]() command.
This practice accomplishes two things:

1. It allows servers to be backward compatible with RESP2 versions.
  This is needed in Redis to make the transition to version 3 of the protocol gentler.
2. The [`HELLO`]() command returns information about the server and the protocol that the client can use for different goals.

The [`HELLO`]() command has the following high-level syntax:

    HELLO <protocol-version> [optional-arguments]

The first argument of the command is the protocol version we want the connection to be set.
By default, the connection starts in RESP2 mode.
If we specify a connection version that is too big and unsupported by the server, it should reply with a `-NOPROTO` error. Example:

    Client: HELLO 4
    Server: -NOPROTO sorry, this protocol version is not supported.

At that point, the client may retry with a lower protocol version.

Similarly, the client can easily detect a server that is only able to speak RESP2:

    Client: HELLO 3
    Server: -ERR unknown command 'HELLO'

The client can then proceed and use RESP2 to communicate with the server.

Note that even if the protocol's version is supported, the [`HELLO`]() command may return an error, perform no action and remain in RESP2 mode.
For example, when used with invalid authentication credentials in the command's optional `AUTH` clause:

    Client: HELLO 3 AUTH default mypassword
    Server: -ERR invalid password
    (the connection remains in RESP2 mode)

A successful reply to the [`HELLO`]() command is a map reply.
The information in the reply is partly server-dependent, but certain fields are mandatory for all the RESP3 implementations:
* **server**: "redis" (or other software name).
* **version**: the server's version.
* **proto**: the highest supported version of the RESP protocol.

In Redis' RESP3 implementation, the following fields are also emitted:

* **id**: the connection's identifier (ID).
* **mode**: "standalone", "sentinel" or "cluster".
* **role**: "master" or "replica".
* **modules**: list of loaded modules as an Array of Bulk Strings.

Sending commands to a Redis server

Now that you are familiar with the RESP serialization format, you can use it to help write a Redis client library.
We can further specify how the interaction between the client and the server works:

* A client sends the Redis server an [array](#arrays) consisting of only bulk strings.
* A Redis server replies to clients, sending any valid RESP data type as a reply.

So, for example, a typical interaction could be the following.

The client sends the command `LLEN mylist` to get the length of the list stored at the key _mylist_.
Then the server replies with an [integer](#integers) reply as in the following example (`C:` is the client, `S:` the server).

    C: *2\r\n
    C: $4\r\n
    C: LLEN\r\n
    C: $6\r\n
    C: mylist\r\n

    S: :48293\r\n

As usual, we separate different parts of the protocol with newlines for simplicity, but the actual interaction is the client sending `*2\r\n$4\r\nLLEN\r\n$6\r\nmylist\r\n` as a whole.

Multiple commands and pipelining

A client can use the same connection to issue multiple commands.
Pipelining is supported, so multiple commands can be sent with a single write operation by the client.
The client can skip reading replies and continue to send the commands one after the other.
All the replies can be read at the end.

For more information, see [Pipelining]().

Inline commands

Sometimes you may need to send a command to the Redis server but only have `telnet` available.
While the Redis protocol is simple to implement, it is not ideal for interactive sessions, and `redis-cli` may not always be available.
For this reason, Redis also accepts commands in the _inline command_ format.

The following example demonstrates a server/client exchange using an inline command (the server chat starts with `S:`, the client chat with `C:`):

    C: PING
    S: +PONG

Here's another example of an inline command where the server returns an integer:

    C: EXISTS somekey
    S: :0

Basically, to issue an inline command, you write space-separated arguments in a telnet session.
Since no command starts with `*` (the identifying byte of RESP Arrays), Redis detects this condition and parses your command inline.

High-performance parser for the Redis protocol

While the Redis protocol is human-readable and easy to implement, its implementation can exhibit performance similar to that of a binary protocol.

RESP uses prefixed lengths to transfer bulk data.
That makes scanning the payload for special characters unnecessary (unlike parsing JSON, for example).
For the same reason, quoting and escaping the payload isn't needed.

Reading the length of aggregate types (for example, bulk strings or arrays) can be processed with code that performs a single operation per character while at the same time scanning for the CR character.

Example (in C):

[code example]

After the first CR is identified, it can be skipped along with the following LF without further processing.
Then, the bulk data can be read with a single read operation that doesn't inspect the payload in any way.
Finally, the remaining CR and LF characters are discarded without additional processing.

While comparable in performance to a binary protocol, the Redis protocol is significantly more straightforward to implement in most high-level languages, reducing the number of bugs in client software.

Tips for Redis client authors

* For testing purposes, use [Lua's type conversions]() to have Redis reply with any RESP2/RESP3 needed.
  As an example, a RESP3 double can be generated like so:
  [code example]

#include <stdio.h>

int main(void) {
    unsigned char *p = "$123\r\n";
    int len = 0;

    p++;
    while(*p != '\r') {
        len = (len*10)+(*p - '0');
        p++;
    }

    /* Now p points at '\r', and the length is in len. */
    printf("%d\n", len);
    return 0;
}

EVAL "return { double = tonumber(ARGV[1]) }" 0 1e0


Sentinel client spec

Summary: How to build clients for Redis Sentinel

Overview

Redis Sentinel is a monitoring solution for Redis instances that handles
automatic failover of Redis masters and service discovery (who is the current
master for a given group of instances?). Since Sentinel is both responsible
for reconfiguring instances during failovers, and providing configurations to
clients connecting to Redis masters or replicas, clients are required to have
explicit support for Redis Sentinel.

This document is targeted at Redis clients developers that want to support Sentinel in their clients implementation with the following goals:

* Automatic configuration of clients via Sentinel.
* Improved safety of Redis Sentinel automatic failover.

For details about how Redis Sentinel works, please check the [Redis Documentation](), as this document only contains information needed for Redis client developers, and it is expected that readers are familiar with the way Redis Sentinel works.

Redis service discovery via Sentinel

Redis Sentinel identifies every master with a name like "stats" or "cache".
Every name actually identifies a *group of instances*, composed of a master
and a variable number of replicas.

The address of the Redis master that is used for a specific purpose inside a network may change after events like an automatic failover, a manually triggered failover (for instance in order to upgrade a Redis instance), and other reasons.

Normally Redis clients have some kind of hard-coded configuration that specifies the address of a Redis master instance within a network as IP address and port number. However if the master address changes, manual intervention in every client is needed.

A Redis client supporting Sentinel can automatically discover the address of a Redis master from the master name using Redis Sentinel. So instead of a hard coded IP address and port, a client supporting Sentinel should optionally be able to take as input:

* A list of ip:port pairs pointing to known Sentinel instances.
* The name of the service, like "cache" or "timelines".

This is the procedure a client should follow in order to obtain the master address starting from the list of Sentinels and the service name.

Step 1: connect to the first Sentinel

The client should iterate the list of Sentinel addresses. For every address it should try to connect to the Sentinel, using a short timeout (in the order of a few hundreds of milliseconds). On errors or timeouts the next Sentinel address should be tried.

If all the Sentinel addresses were tried without success, an error should be returned to the client.

The first Sentinel replying to the client request should be put at the start of the list, so that at the next reconnection, we'll try first the Sentinel that was reachable in the previous connection attempt, minimizing latency.

Step 2: ask for the master address

Once a connection with a Sentinel is established, the client should retry to execute the following command on the Sentinel:

    SENTINEL get-master-addr-by-name master-name

Where *master-name* should be replaced with the actual service name specified by the user.

The result from this call can be one of the following two replies:

* An ip:port pair.
* A null reply. This means Sentinel does not know this master.

If an ip:port pair is received, this address should be used to connect to the Redis master. Otherwise if a null reply is received, the client should try the next Sentinel in the list.

Step 3: call the ROLE command in the target instance

Once the client discovered the address of the master instance, it should
attempt a connection with the master, and call the [`ROLE`]() command in order
to verify the role of the instance is actually a master.

If the [`ROLE`]() commands is not available (it was introduced in Redis 2.8.12), a client may resort to the `INFO replication` command parsing the `role:` field of the output.

If the instance is not a master as expected, the client should wait a short amount of time (a few hundreds of milliseconds) and should try again starting from Step 1.

Handle reconnections

Once the service name is resolved into the master address and a connection is established with the Redis master instance, every time a reconnection is needed, the client should resolve again the address using Sentinels restarting from Step 1. For instance Sentinel should contacted again the following cases:

* If the client reconnects after a timeout or socket error.
* If the client reconnects because it was explicitly closed or reconnected by the user.

In the above cases and any other case where the client lost the connection with the Redis server, the client should resolve the master address again.

Sentinel failover disconnection

Starting with Redis 2.8.12, when Redis Sentinel changes the configuration of
an instance, for example promoting a replica to a master, demoting a master to
replicate to the new master after a failover, or simply changing the master
address of a stale replica instance, it sends a `CLIENT KILL type normal`
command to the instance in order to make sure all the clients are disconnected
from the reconfigured instance. This will force clients to resolve the master
address again.

If the client will contact a Sentinel with yet not updated information, the verification of the Redis instance role via the [`ROLE`]() command will fail, allowing the client to detect that the contacted Sentinel provided stale information, and will try again.

Note: it is possible that a stale master returns online at the same time a client contacts a stale Sentinel instance, so the client may connect with a stale master, and yet the ROLE output will match. However when the master is back again Sentinel will try to demote it to replica, triggering a new disconnection. The same reasoning applies to connecting to stale replicas that will get reconfigured to replicate with a different master.

Connect to replicas

Sometimes clients are interested to connect to replicas, for example in order to scale read requests. This protocol supports connecting to replicas by modifying step 2 slightly. Instead of calling the following command:

    SENTINEL get-master-addr-by-name master-name

The clients should call instead:

    SENTINEL replicas master-name

In order to retrieve a list of replica instances.

Symmetrically the client should verify with the [`ROLE`]() command that the
instance is actually a replica, in order to avoid scaling read queries with
the master.

Connection pools

For clients implementing connection pools, on reconnection of a single connection, the Sentinel should be contacted again, and in case of a master address change all the existing connections should be closed and connected to the new address.

Error reporting

The client should correctly return the information to the user in case of errors. Specifically:

* If no Sentinel can be contacted (so that the client was never able to get the reply to `SENTINEL get-master-addr-by-name`), an error that clearly states that Redis Sentinel is unreachable should be returned.
* If all the Sentinels in the pool replied with a null reply, the user should be informed with an error that Sentinels don't know this master name.

Sentinels list automatic refresh

Optionally once a successful reply to `get-master-addr-by-name` is received, a client may update its internal list of Sentinel nodes following this procedure:

* Obtain a list of other Sentinels for this master using the command `SENTINEL sentinels <master-name>`.
* Add every ip:port pair not already existing in our list at the end of the list.

It is not needed for a client to be able to make the list persistent updating its own configuration. The ability to upgrade the in-memory representation of the list of Sentinels can be already useful to improve reliability.

Subscribe to Sentinel events to improve responsiveness

The [Sentinel documentation]() shows how clients can connect to
Sentinel instances using Pub/Sub in order to subscribe to changes in the
Redis instances configurations.

This mechanism can be used in order to speedup the reconfiguration of clients,
that is, clients may listen to Pub/Sub in order to know when a configuration
change happened in order to run the three steps protocol explained in this
document in order to resolve the new Redis master (or replica) address.

However update messages received via Pub/Sub should not substitute the
above procedure, since there is no guarantee that a client is able to
receive all the update messages.

Additional information

For additional information or to discuss specific aspects of this guidelines, please drop a message to the [Redis Google Group](https://groups.google.com/group/redis-db).