title: Virtual memory (deprecated)

title: Content

text:
**Note: This document was written by the creator of Redis, Salvatore Sanfilippo, early in the development of Redis (c. 2010). Virtual Memory was deprecated in Redis 2.4 and removed in Redis 2.6, so this documentation exists only for historical interest.**

This document details the internals of the Redis Virtual Memory subsystem prior to Redis 2.6. The intended audience is not the final user but programmers willing to understand or modify the Virtual Memory implementation.

Keys vs Values: what is swapped out?
---

The goal of the VM subsystem is to free memory transferring Redis Objects from memory to disk. This is a very generic command, but specifically, Redis transfers only objects associated with _values_. In order to understand better this concept we'll show, using the DEBUG command, how a key holding a value looks from the point of view of the Redis internals:

    redis> set foo bar
    OK
    redis> debug object foo
    Key at:0x100101d00 refcount:1, value at:0x100101ce0 refcount:1 encoding:raw serializedlength:4

As you can see from the above output, the Redis top level hash table maps Redis Objects (keys) to other Redis Objects (values). The Virtual Memory is only able to swap _values_ on disk, the objects associated to _keys_ are always taken in memory: this trade off guarantees very good lookup performances, as one of the main design goals of the Redis VM is to have performances similar to Redis with VM disabled when the part of the dataset frequently used fits in RAM.

How does a swapped value looks like internally
---

When an object is swapped out, this is what happens in the hash table entry:

 * The key continues to hold a Redis Object representing the key.
 * The value is set to NULL

So you may wonder where we store the information that a given value (associated to a given key) was swapped out. Just in the key object!

This is how the Redis Object structure _robj_ looks like:

    /* The actual Redis Object */
    typedef struct redisObject {
        void *ptr;
        unsigned char type;
        unsigned char encoding;
        unsigned char storage;  /* If this object is a key, where is the value?
                                 * REDIS_VM_MEMORY, REDIS_VM_SWAPPED, ... */
        unsigned char vtype; /* If this object is a key, and value is swapped out,
                              * this is the type of the swapped out object. */
        int refcount;
        /* VM fields, this are only allocated if VM is active, otherwise the
         * object allocation function will just allocate
         * sizeof(redisObject) minus sizeof(redisObjectVM), so using
         * Redis without VM active will not have any overhead. */
        struct redisObjectVM vm;
    } robj;

As you can see there are a few fields about VM. The most important one is _storage_, that can be one of this values:

 * `REDIS_VM_MEMORY`: the associated value is in memory.
 * `REDIS_VM_SWAPPED`: the associated values is swapped, and the value entry of the hash table is just set to NULL.
 * `REDIS_VM_LOADING`: the value is swapped on disk, the entry is NULL, but there is a job to load the object from the swap to the memory (this field is only used when threaded VM is active).
 * `REDIS_VM_SWAPPING`: the value is in memory, the entry is a pointer to the actual Redis Object, but there is an I/O job in order to transfer this value to the swap file.

If an object is swapped on disk (`REDIS_VM_SWAPPED` or `REDIS_VM_LOADING`), how do we know where it is stored, what type it is, and so forth? That's simple: the _vtype_ field is set to the original type of the Redis object swapped, while the _vm_ field (that is a _redisObjectVM_ structure) holds information about the location of the object. This is the definition of this additional structure:

    /* The VM object structure */
    struct redisObjectVM {
        off_t page;         /* the page at which the object is stored on disk */
        off_t usedpages;    /* number of pages used on disk */
        time_t atime;       /* Last access time */
    } vm;

As you can see the structure contains the page at which the object is located in the swap file, the number of pages used, and the last access time of the object (this is very useful for the algorithm that select what object is a good candidate for swapping, as we want to transfer on disk objects that are rarely accessed).

As you can see, while all the other fields are using unused bytes in the old Redis Object structure (we had some free bit due to natural memory alignment concerns), the _vm_ field is new, and indeed uses additional memory. Should we pay such a memory cost even when VM is disabled? No! This is the code to create a new Redis Object:

    ... some code ...
            if (server.vm_enabled) {
                pthread_mutex_unlock(&server.obj_freelist_mutex);
                o = zmalloc(sizeof(*o));
            } else {
                o = zmalloc(sizeof(*o)-sizeof(struct redisObjectVM));
            }
    ... some code ...

As you can see if the VM system is not enabled we allocate just `sizeof(*o)-sizeof(struct redisObjectVM)` of memory. Given that the _vm_ field is the last in the object structure, and that this fields are never accessed if VM is disabled, we are safe and Redis without VM does not pay the memory overhead.

The Swap File
---

The next step in order to understand how the VM subsystem works is understanding how objects are stored inside the swap file. The good news is that's not some kind of special format, we just use the same format used to store the objects in .rdb files, that are the usual dump files produced by Redis using the [`SAVE`](/commands/save) command.

The swap file is composed of a given number of pages, where every page size is a given number of bytes. This parameters can be changed in redis.conf, since different Redis instances may work better with different values: it depends on the actual data you store inside it. The following are the default values:

    vm-page-size 32
    vm-pages 134217728

Redis takes a "bitmap" (a contiguous array of bits set to zero or one) in memory, every bit represent a page of the swap file on disk: if a given bit is set to 1, it represents a page that is already used (there is some Redis Object stored there), while if the corresponding bit is zero, the page is free.

Taking this bitmap (that will call the page table) in memory is a huge win in terms of performances, and the memory used is small: we just need 1 bit for every page on disk. For instance in the example below 134217728 pages of 32 bytes each (4GB swap file) is using just 16 MB of RAM for the page table.

Transferring objects from memory to swap
---

In order to transfer an object from memory to disk we need to perform the following steps (assuming non threaded VM, just a simple blocking approach):

 * Find how many pages are needed in order to store this object on the swap file. This is trivially accomplished just calling the function `rdbSavedObjectPages` that returns the number of pages used by an object on disk. Note that this function does not duplicate the .rdb saving code just to understand what will be the length *after* an object will be saved on disk, we use the trick of opening /dev/null and writing the object there, finally calling `ftello` in order check the amount of bytes required. What we do basically is to save the object on a virtual very fast file, that is, /dev/null.
 * Now that we know how many pages are required in the swap file, we need to find this number of contiguous free pages inside the swap file. This task is accomplished by the `vmFindContiguousPages` function. As you can guess this function may fail if the swap is full, or so fragmented that we can't easily find the required number of contiguous free pages. When this happens we just abort the swapping of the object, that will continue to live in memory.
 * Finally we can write the object on disk, at the specified position, just calling the function `vmWriteObjectOnSwap`.

As you can guess once the object was correctly written in the swap file, it is freed from memory, the storage field in the associated key is set to `REDIS_VM_SWAPPED`, and the used pages are marked as used in the page table.

Loading objects back in memory
---

Loading an object from swap to memory is simpler, as we already know where the object is located and how many pages it is using. We also know the type of the object (the loading functions are required to know this information, as there is no header or any other information about the object type on disk), but this is stored in the _vtype_ field of the associated key as already seen above.

Calling the function `vmLoadObject` passing the key object associated to the value object we want to load back is enough. The function will also take care of fixing the storage type of the key (that will be `REDIS_VM_MEMORY`), marking the pages as freed in the page table, and so forth.

The return value of the function is the loaded Redis Object itself, that we'll have to set again as value in the main hash table (instead of the NULL value we put in place of the object pointer when the value was originally swapped out).

How blocking VM works
---

Now we have all the building blocks in order to describe how the blocking VM works. First of all, an important detail about configuration. In order to enable blocking VM in Redis `server.vm_max_threads` must be set to zero.
We'll see later how this max number of threads info is used in the threaded VM, for now all it's needed to now is that Redis reverts to fully blocking VM when this is set to zero.

We also need to introduce another important VM parameter, that is, `server.vm_max_memory`. This parameter is very important as it is used in order to trigger swapping: Redis will try to swap objects only if it is using more memory than the max memory setting, otherwise there is no need to swap as we are matching the user requested memory usage.

Blocking VM swapping
---

Swapping of object from memory to disk happens in the cron function. This function used to be called every second, while in the recent Redis versions on git it is called every 100 milliseconds (that is, 10 times per second).
If this function detects we are out of memory, that is, the memory used is greater than the vm-max-memory setting, it starts transferring objects from memory to disk in a loop calling the function `vmSwapOneObect`. This function takes just one argument, if 0 it will swap objects in a blocking way, otherwise if it is 1, I/O threads are used. In the blocking scenario we just call it with zero as argument.

vmSwapOneObject acts performing the following steps:

 * The key space in inspected in order to find a good candidate for swapping (we'll see later what a good candidate for swapping is).
 * The associated value is transferred to disk, in a blocking way.
 * The key storage field is set to `REDIS_VM_SWAPPED`, while the _vm_ fields of the object are set to the right values (the page index where the object was swapped, and the number of pages used to swap it).
 * Finally the value object is freed and the value entry of the hash table is set to NULL.

The function is called again and again until one of the following happens: there is no way to swap more objects because either the swap file is full or nearly all the objects are already transferred on disk, or simply the memory usage is already under the vm-max-memory parameter.

What values to swap when we are out of memory?
---

Understanding what's a good candidate for swapping is not too hard. A few objects at random are sampled, and for each their _swappability_ is commuted as:

    swappability = age*log(size_in_memory)

The age is the number of seconds the key was not requested, while size_in_memory is a fast estimation of the amount of memory (in bytes) used by the object in memory. So we try to swap out objects that are rarely accessed, and we try to swap bigger objects over smaller one, but the latter is a less important factor (because of the logarithmic function used). This is because we don't want bigger objects to be swapped out and in too often as the bigger the object the more I/O and CPU is required in order to transfer it.

Blocking VM loading
---

What happens if an operation against a key associated with a swapped out object is requested? For instance Redis may just happen to process the following command:

    GET foo

If the value object of the `foo` key is swapped we need to load it back in memory before processing the operation. In Redis the key lookup process is centralized in the `lookupKeyRead` and `lookupKeyWrite` functions, this two functions are used in the implementation of all the Redis commands accessing the keyspace, so we have a single point in the code where to handle the loading of the key from the swap file to memory.

So this is what happens:

 * The user calls some command having as argument a swapped key
 * The command implementation calls the lookup function
 * The lookup function search for the key in the top level hash table. If the value associated with the requested key is swapped (we can see that checking the _storage_ field of the key object), we load it back in memory in a blocking way before to return to the user.

This is pretty straightforward, but things will get more _interesting_ with the threads. From the point of view of the blocking VM the only real problem is the saving of the dataset using another process, that is, handling [`BGSAVE`](/commands/bgsave) and [`BGREWRITEAOF`](/commands/bgrewriteaof) commands.

Background saving when VM is active
---

The default Redis way to persist on disk is to create .rdb files using a child process. Redis calls the fork() system call in order to create a child, that has the exact copy of the in memory dataset, since fork duplicates the whole program memory space (actually thanks to a technique called Copy on Write memory pages are shared between the parent and child process, so the fork() call will not require too much memory).

In the child process we have a copy of the dataset in a given point in the time. Other commands issued by clients will just be served by the parent process and will not modify the child data.

The child process will just store the whole dataset into the dump.rdb file and finally will exit. But what happens when the VM is active? Values can be swapped out so we don't have all the data in memory, and we need to access the swap file in order to retrieve the swapped values. While child process is saving the swap file is shared between the parent and child process, since:

* The parent process needs to access the swap file in order to load values back into memory if an operation against swapped out values are performed.
* The child process needs to access the swap file in order to retrieve the full dataset while saving the data set on disk.

In order to avoid problems while both the processes are accessing the same swap file we do a simple thing, that is, not allowing values to be swapped out in the parent process while a background saving is in progress. This way both the processes will access the swap file in read only. This approach has the problem that while the child process is saving no new values can be transferred on the swap file even if Redis is using more memory than the max memory parameters dictates. This is usually not a problem as the background saving will terminate in a short amount of time and if still needed a percentage of values will be swapped on disk ASAP.

An alternative to this scenario is to enable the Append Only File that will have this problem only when a log rewrite is performed using the [`BGREWRITEAOF`](/commands/bgrewriteaof) command.

The problem with the blocking VM
---

The problem of blocking VM is that... it's blocking :)
This is not a problem when Redis is used in batch processing activities, but for real-time usage one of the good points of Redis is the low latency. The blocking VM will have bad latency behaviors as when a client is accessing a swapped out value, or when Redis needs to swap out values, no other clients will be served in the meantime.

Swapping out keys should happen in background. Similarly when a client is accessing a swapped out value other clients accessing in memory values should be served mostly as fast as when VM is disabled. Only the clients dealing with swapped out keys should be delayed.

All this limitations called for a non-blocking VM implementation.

Threaded VM
---

There are basically three main ways to turn the blocking VM into a non blocking one.
* 1: One way is obvious, and in my opinion, not a good idea at all, that is, turning Redis itself into a threaded server: if every request is served by a different thread automatically other clients don't need to wait for blocked ones. Redis is fast, exports atomic operations, has no locks, and is just 10k lines of code, *because* it is single threaded, so this was not an option for me.
* 2: Using non-blocking I/O against the swap file. After all you can think Redis already event-loop based, why don't just handle disk I/O in a non-blocking fashion? I also discarded this possibility because of two main reasons. One is that non blocking file operations, unlike sockets, are an incompatibility nightmare. It's not just like calling select, you need to use OS-specific things. The other problem is that the I/O is just one part of the time consumed to handle VM, another big part is the CPU used in order to encode/decode data to/from the swap file. This is I picked option three, that is...
* 3: Using I/O threads, that is, a pool of threads handling the swap I/O operations. This is what the Redis VM is using, so let's detail how this works.

I/O Threads
---

The threaded VM design goals where the following, in order of importance:

 * Simple implementation, little room for race conditions, simple locking, VM system more or less completely decoupled from the rest of Redis code.
 * Good performances, no locks for clients accessing values in memory.
 * Ability to decode/encode objects in the I/O threads.

The above goals resulted in an implementation where the Redis main thread (the one serving actual clients) and the I/O threads communicate using a queue of jobs, with a single mutex.
Basically when main thread requires some work done in the background by some I/O thread, it pushes an I/O job structure in the `server.io_newjobs` queue (that is, just a linked list). If there are no active I/O threads, one is started. At this point some I/O thread will process the I/O job, and the result of the processing is pushed in the `server.io_processed` queue. The I/O thread will send a byte using an UNIX pipe to the main thread in order to signal that a new job was processed and the result is ready to be processed.

This is how the `iojob` structure looks like:

    typedef struct iojob {
        int type;   /* Request type, REDIS_IOJOB_* */
        redisDb *db;/* Redis database */
        robj *key;  /* This I/O request is about swapping this key */
        robj *val;  /* the value to swap for REDIS_IOREQ_*_SWAP, otherwise this
                     * field is populated by the I/O thread for REDIS_IOREQ_LOAD. */
        off_t page; /* Swap page where to read/write the object */
        off_t pages; /* Swap pages needed to save object. PREPARE_SWAP return val */
        int canceled; /* True if this command was canceled by blocking side of VM */
        pthread_t thread; /* ID of the thread processing this entry */
    } iojob;

There are just three type of jobs that an I/O thread can perform (the type is specified by the `type` field of the structure):

* `REDIS_IOJOB_LOAD`: load the value associated to a given key from swap to memory. The object offset inside the swap file is `page`, the object type is `key->vtype`. The result of this operation will populate the `val` field of the structure.
* `REDIS_IOJOB_PREPARE_SWAP`: compute the number of pages needed in order to save the object pointed by `val` into the swap. The result of this operation will populate the `pages` field.
* `REDIS_IOJOB_DO_SWAP`: Transfer the object pointed by `val` to the swap file, at page offset `page`.

The main thread delegates just the above three tasks. All the rest is handled by the I/O thread itself, for instance finding a suitable range of free pages in the swap file page table (that is a fast operation), deciding what object to swap, altering the storage field of a Redis object to reflect the current state of a value.

Non blocking VM as probabilistic enhancement of blocking VM
---

So now we have a way to request background jobs dealing with slow VM operations. How to add this to the mix of the rest of the work done by the main thread? While blocking VM was aware that an object was swapped out just when the object was looked up, this is too late for us: in C it is not trivial to start a background job in the middle of the command, leave the function, and re-enter in the same point the computation when the I/O thread finished what we requested (that is, no co-routines or continuations or alike).

Fortunately there was a much, much simpler way to do this. And we love simple things: basically consider the VM implementation a blocking one, but add an optimization (using non the no blocking VM operations we are able to perform) to make the blocking *very* unlikely.

This is what we do:

 * Every time a client sends us a command, *before* the command is executed, we examine the argument vector of the command in search for swapped keys. After all we know for every command what arguments are keys, as the Redis command format is pretty simple.
 * If we detect that at least a key in the requested command is swapped on disk, we block the client instead of really issuing the command. For every swapped value associated to a requested key, an I/O job is created, in order to bring the values back in memory. The main thread continues the execution of the event loop, without caring about the blocked client.
 * In the meanwhile, I/O threads are loading values in memory. Every time an I/O thread finished loading a value, it sends a byte to the main thread using an UNIX pipe. The pipe file descriptor has a readable event associated in the main thread event loop, that is the function `vmThreadedIOCompletedJob`. If this function detects that all the values needed for a blocked client were loaded, the client is restarted and the original command called.

So you can think of this as a blocked VM that almost always happen to have the right keys in memory, since we pause clients that are going to issue commands about swapped out values until this values are loaded.

If the function checking what argument is a key fails in some way, there is no problem: the lookup function will see that a given key is associated to a swapped out value and will block loading it. So our non blocking VM reverts to a blocking one when it is not possible to anticipate what keys are touched.

For instance in the case of the [`SORT`](/commands/sort) command used together with the [`GET`](/commands/get) or `BY` options, it is not trivial to know beforehand what keys will be requested, so at least in the first implementation, `SORT BY/GET` resorts to the blocking VM implementation.

Blocking clients on swapped keys
---

How to block clients? To suspend a client in an event-loop based server is pretty trivial. All we do is canceling its read handler. Sometimes we do something different (for instance for BLPOP) that is just marking the client as blocked, but not processing new data (just accumulating the new data into input buffers).

Aborting I/O jobs
---

There is something hard to solve about the interactions between our blocking and non blocking VM, that is, what happens if a blocking operation starts about a key that is also "interested" by a non blocking operation at the same time?

For instance while SORT BY is executed, a few keys are being loaded in a blocking manner by the sort command. At the same time, another client may request the same keys with a simple _GET key_ command, that will trigger the creation of an I/O job to load the key in background.

The only simple way to deal with this problem is to be able to kill I/O jobs in the main thread, so that if a key that we want to load or swap in a blocking way is in the `REDIS_VM_LOADING` or `REDIS_VM_SWAPPING` state (that is, there is an I/O job about this key), we can just kill the I/O job about this key, and go ahead with the blocking operation we want to perform.

This is not as trivial as it is. In a given moment an I/O job can be in one of the following three queues:

 * server.io_newjobs: the job was already queued but no thread is handling it.
 * server.io_processing: the job is being processed by an I/O thread.
 * server.io_processed: the job was already processed.
The function able to kill an I/O job is `vmCancelThreadedIOJob`, and this is what it does:
 * If the job is in the newjobs queue, that's simple, removing the iojob structure from the queue is enough as no thread is still executing any operation.
 * If the job is in the processing queue, a thread is messing with our job (and possibly with the associated object!). The only thing we can do is waiting for the item to move to the next queue in a *blocking way*. Fortunately this condition happens very rarely so it's not a performance problem.
 * If the job is in the processed queue, we just mark it as _canceled_ marking setting the `canceled` field to 1 in the iojob structure. The function processing completed jobs will just ignored and free the job instead of really processing it.

Questions?
---

This document is in no way complete, the only way to get the whole picture is reading the source code, but it should be a good introduction in order to make the code review / understanding a lot simpler.

Something is not clear about this page? Please leave a comment and I'll try to address the issue possibly integrating the answer in this document.


title: Redis design draft #2 (historical)

summary: A design for the RDB format written in the early days of Redis

page_type: content

title: Overview

role: overview

text: **Note: this document was written by the creator of Redis, Salvatore Sanfilippo, early in the development of Redis (c. 2013), as part of a series of design drafts. This is preserved for historical interest.**

title: Redis Design Draft 2 -- RDB version 7 info fields

role: content

text:
* Author: Salvatore Sanfilippo `antirez@gmail.com`
* GitHub issue [#1048](https://github.com/redis/redis/issues/1048)

title: History of revisions

role: history

text: 1.0, 10 April 2013 - Initial draft.

title: Overview

text:
The Redis RDB format lacks a simple way to add info fields to an RDB file
without causing a backward compatibility issue even if the added meta data
is not required in order to load data from the RDB file.

For example thanks to the info fields specified in this document it will
be possible to add to RDB information like file creation time, Redis version
generating the file, and any other useful information, in a way that not
every field is required for an RDB version 7 file to be correctly processed.

Also with minimal changes it will be possible to add RDB version 7 support to
Redis 2.6 without actually supporting the additional fields but just skipping
them when loading an RDB file.

RDB info fields may have semantic meaning if needed, so that the presence
of the field may add information about the data set specified in the RDB
file format, however when an info field is required to be correctly decoded
in order to understand and load the data set content of the RDB file, the
RDB file format must be increased so that previous versions of Redis will not
attempt to load it.

However currently the info fields are designed to only hold additional
information that are not useful to load the dataset, but can better specify
how the RDB file was created.

title: Info fields representation

text:
The RDB format 6 has the following layout:

* A 9 bytes magic "REDIS0006"
* key-value pairs
* An EOF opcode
* CRC64 checksum

The proposal for RDB format 7 is to add the optional fields immediately
after the first 9 bytes magic, so that the new format will be:

* A 9 bytes magic "REDIS0007"
* Info field 1
* Info field 2
* ...
* Info field N
* Info field end-of-fields
* key-value pairs
* An EOF opcode
* CRC64 checksum

Every single info field has the following structure:

* A 16 bit identifier
* A 64 bit data length
* A data section of the exact length as specified

Both the identifier and the data length are stored in little endian byte
ordering.

The special identifier 0 means that there are no other info fields, and that
the remaining of the RDB file contains the key-value pairs.

title: Handling of info fields

text:
A program can simply skip every info field it does not understand, as long
as the RDB version matches the one that it is capable to load.

title: Specification of info fields IDs and content.

text:
#### Info field 0 -- End of info fields

This just means there are no longer info fields to process.

#### Info field 1 -- Creation date

This field represents the unix time at which the RDB file was created.
The format of the unix time is a 64 bit little endian integer representing
seconds since 1th January 1970.

#### Info field 2 -- Redis version

This field represents a null-terminated string containing the Redis version
that generated the file, as displayed in the Redis version INFO field.


title: Redis signal handling

summary: How Redis handles common Unix signals

title: Overview

text:
This document provides information about how Redis reacts to different POSIX signals such as `SIGTERM` and `SIGSEGV`.

The information in this document **only applies to Redis version 2.6 or greater**.

title: SIGTERM and SIGINT

text:
The `SIGTERM` and `SIGINT` signals tell Redis to shut down gracefully. When the server receives this signal,
it does not immediately exit. Instead, it schedules
a shutdown similar to the one performed by the [`SHUTDOWN`](/commands/shutdown) command. The scheduled shutdown starts as soon as possible, specifically as long as the
current command in execution terminates (if any), with a possible additional
delay of 0.1 seconds or less.

If the server is blocked by a long-running Lua script,
kill the script with [`SCRIPT KILL`](/commands/script-kill) if possible. The scheduled shutdown will
run just after the script is killed or terminates spontaneously.

This shutdown process includes the following actions:

* If there are any replicas lagging behind in replication:
  * Pause clients attempting to write with [`CLIENT PAUSE`](/commands/client-pause) and the `WRITE` option.
  * Wait up to the configured `shutdown-timeout` (default 10 seconds) for replicas to catch up with the master's replication offset.
* If a background child is saving the RDB file or performing an AOF rewrite, the child process is killed.
* If the AOF is active, Redis calls the `fsync` system call on the AOF file descriptor to flush the buffers on disk.
* If Redis is configured to persist on disk using RDB files, a synchronous (blocking) save is performed. Since the save is synchronous, it doesn't use any additional memory.
* If the server is daemonized, the PID file is removed.
* If the Unix domain socket is enabled, it gets removed.
* The server exits with an exit code of zero.

IF the RDB file can't be saved, the shutdown fails, and the server continues to run in order to ensure no data loss.
Likewise, if the user just turned on AOF, and the server triggered the first AOF rewrite in order to create the initial AOF file but this file can't be saved, the shutdown fails and the server continues to run.
Since Redis 2.6.11, no further attempt to shut down will be made unless a new `SIGTERM` is received or the [`SHUTDOWN`](/commands/shutdown) command is issued.

Since Redis 7.0, the server waits for lagging replicas up to a configurable `shutdown-timeout`, 10 seconds by default, before shutting down.
This provides a best effort to minimize the risk of data loss in a situation where no save points are configured and AOF is deactivated.
Before version 7.0, shutting down a heavily loaded master node in a diskless setup was more likely to result in data loss.
To minimize the risk of data loss in such setups, trigger a manual [`FAILOVER`](/commands/failover) (or [`CLUSTER FAILOVER`](/commands/cluster-failover)) to demote the master to a replica and promote one of the replicas to a new master before shutting down a master node.

title: SIGSEGV, SIGBUS, SIGFPE and SIGILL

text:
The following signals are handled as a Redis crash:

* SIGSEGV
* SIGBUS
* SIGFPE
* SIGILL

Once one of these signals is trapped, Redis stops any current operation and performs the following actions:

* Adds a bug report to the log file. This includes a stack trace, dump of registers, and information about the state of clients.
* Since Redis 2.8, a fast memory test is performed as a first check of the reliability of the crashing system.
* If the server was daemonized, the PID file is removed.
* Finally the server unregisters its own signal handler for the received signal and resends the same signal to itself to make sure that the default action is performed, such as dumping the core on the file system.

title: What happens when a child process gets killed

text:
When the child performing the Append Only File rewrite gets killed by a signal,
Redis handles this as an error and discards the (probably partial or corrupted)
AOF file. It will attempt the rewrite again later.

When the child performing an RDB save is killed, Redis handles the
condition as a more severe error. While the failure of an
AOF file rewrite can cause AOF file enlargement, failed RDB file
creation reduces durability.

As a result of the child producing the RDB file being killed by a signal,
or when the child exits with an error (non zero exit code), Redis enters
a special error condition where no further write command is accepted.

* Redis will continue to reply to read commands.
* Redis will reply to all write commands with a `MISCONFIG` error.

This error condition will persist until it becomes possible to create an RDB file successfully.

title: Kill the RDB file without errors

text:
Sometimes the user may want to kill the RDB-saving child process without
generating an error. Since Redis version 2.6.10, this can be done using the signal `SIGUSR1`. This signal is handled in a special way:
it kills the child process like any other signal, but the parent process will
not detect this as a critical error and will continue to serve write
requests.


title: Probabilistic data structure commands

summary: Lists probabilistic data structure commands and provides links to the command reference pages.

title: Overview

text: The following tables list probabilistic data structure commands. See the command links for more information about each command's syntax, arguments, and examples.

title: Bloom filter commands

text:
| Command | Redis
Software | Redis
Cloud | Description |
|:--------|:----------------------|:-----------------|:------|
| [BF.ADD]() | ✅ Supported

 | <nobr>✅ Flexible & Annual</nobr>
✅ Free & Fixed</nobr> | Adds an item to the filter. |
| [BF.EXISTS]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Checks if an item exists in the filter. |
| [BF.INFO]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns information about a Bloom filter. |
| [BF.INSERT]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Adds multiple items to a filter. If the key does not exist, it creates a new filter. |
| [BF.LOADCHUNK]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Restores a Bloom filter previously saved with [BF.SCANDUMP](). |
| [BF.MADD]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Adds multiple items to the filter. |
| [BF.MEXISTS]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | For multiple items, checks if each item exists in the filter. |
| [BF.RESERVE]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Creates a Bloom filter. Sets the false positive rate and capacity. |
| [BF.SCANDUMP]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Starts an incremental save of a Bloom filter. |

title: Cuckoo filter commands

text:
| Command | Redis
Software | Redis
Cloud | Description |
|:--------|:----------------------|:-----------------|:------|
| [CF.ADD]() | ✅ Supported

 | <nobr>✅ Flexible & Annual</nobr>
✅ Free & Fixed</nobr> | Adds an item to a filter. |
| [CF.ADDNX]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Adds an item to a filter only if the item does not already exist. |
| [CF.COUNT]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns the probable number of times an item occurs in the filter. |
| [CF.DEL]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Removes one instance of an item from the filter. |
| [CF.EXISTS]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Checks if an item exists in the filter. |
| [CF.INFO]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns information about a cuckoo filter. |
| [CF.INSERT]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Adds multiple items to a filter. Optionally sets the capacity if the filter does not already exist. |
| [CF.INSERTNX]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Adds multiple items to a filter if they do not already exist. Optionally sets the capacity if the filter does not already exist. |
| [CF.LOADCHUNK]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Restores a cuckoo filter previously saved with [CF.SCANDUMP](). |
| [CF.MEXISTS]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | For multiple items, checks if each item exists in the filter. |
| [CF.RESERVE]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Creates a cuckoo filter and sets its capacity. |
| [CF.SCANDUMP]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Starts an incremental save of a cuckoo filter. |

title: Count-min sketch commands

text:
| Command | Redis
Software | Redis
Cloud | Description |
|:--------|:----------------------|:-----------------|:------|
| [CMS.INCRBY]() | ✅ Supported

 | <nobr>✅ Flexible & Annual</nobr>
✅ Free & Fixed</nobr> | Increases item counts. |
| [CMS.INFO]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns width, depth, and total count of the sketch. |
| [CMS.INITBYDIM]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Initializes a count-min sketch to the specified dimensions (width and depth). |
| [CMS.INITBYPROB]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Initializes a count-min sketch to allow the specified overestimation percent for the item count and the probability of overestimation. |
| [CMS.MERGE]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Merges several sketches into one sketch. |
| [CMS.QUERY]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns the count for one or more items in a sketch. |

title: Top-k commands

text:
| Command | Redis
Software | Redis
Cloud | Description |
|:--------|:----------------------|:-----------------|:------|
| [TOPK.ADD]() | ✅ Supported

 | <nobr>✅ Flexible & Annual</nobr>
✅ Free & Fixed</nobr> | Adds an item to the data structure. |
| [TOPK.COUNT]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns probable item counts. |
| [TOPK.INCRBY]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Increases the score of an item by the specified number. |
| [TOPK.INFO]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns the number of required items (k), width, depth, and decay values. |
| [TOPK.LIST]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns the keys of items in the top-k list. Optionally returns their item counts. |
| [TOPK.QUERY]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Checks whether an item is one of top-k items. |
| [TOPK.RESERVE]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Initializes a top-k with the specified number of top occurring items to keep, width, depth, and decay. |

title: T-digest sketch commands

text:
| Command | Redis
Software | Redis
Cloud | Description |
|:--------|:----------------------|:-----------------|:------|
| [TDIGEST.ADD]() | ✅ Supported

 | <nobr>✅ Flexible & Annual</nobr>
✅ Free & Fixed</nobr> | Adds one or more samples to a t-digest sketch. |
| [TDIGEST.CDF]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Estimates the fraction of all observations which are less than or equal to the specified value. |
| [TDIGEST.CREATE]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Allocates memory and initializes a t-digest sketch. |
| [TDIGEST.INFO]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns information about the t-digest sketch. |
| [TDIGEST.MAX]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns the maximum value from the sketch. |
| [TDIGEST.MERGE]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Copies values from one sketch to another. |
| [TDIGEST.MIN]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns the minimum value from the sketch. |
| [TDIGEST.QUANTILE]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Estimates one or more cutoffs. |
| [TDIGEST.RESET]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Resets the sketch and reinitializes it. |
| [TDIGEST.TRIMMED_MEAN]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Estimates the mean value from the sketch, excluding values outside the specified range. |


title: Probabilistic data structure configuration compatibility with Redis Software

summary: Probabilistic data structure configuration settings supported by Redis Software and Redis Cloud.

title: Configure probabilistic data structures in Redis Software

role: configuration

text:
[Redis Software]() lets you manually change any [RedisBloom configuration setting](configuration/#redisbloom-configuration-parameters).

To change the RedisBloom configuration using the Redis Software Cluster Manager UI:

  1. From the **Databases** list, select the database, then click **Configuration**.

  1. Select the **Edit** button.

  1. In the **Capabilities** section, click **Parameters**.

  1. After you finish editing the module's configuration parameters, click **Done** to close the parameter editor.

  1. Click **Save**.

title: Configure probabilistic data structures in Redis Cloud

text: [Redis Cloud]() does not let you configure RedisBloom manually. However, if you have a Flexible or Annual [subscription](), you can contact [support](https://redis.com/company/support/) to request a configuration change. You cannot change RedisBloom configuration for Free or Fixed subscriptions.

title: Configuration settings

text: See [configuration parameters]() in the Develop section for parameter details and compatibility with Redis Software and Redis Cloud.


title: Probabilistic data structures

summary: Probabilistic data structures

title: Probabilistic data structure commands

title: Overview

text:
A Bloom filter is a probabilistic data structure which provides an
efficient way to verify that an entry is certainly *not* in a set. This
makes it especially ideal when trying to search for items on
expensive-to-access resources (such as over a network or disk): If I
have a large on-disk database and I want to know if the key *foo* exists
in it, I can query the Bloom filter first, which tells me with a
certainty whether it potentially exists (and then the disk lookup can
continue) or whether it does *not* exist, and in this case I can forego
the expensive disk lookup and simply send a negative reply up the stack.

While it's possible to use other data structures (such as a hash table)
to perform this, Bloom filters are also especially useful in that they
occupy very little space per element, typically counted in the number of
*bits* (not bytes!). There exists a percentage of false positives
(which is controllable), but for an initial test of whether a key exists
in a set, they provide excellent speed and most importantly excellent
space efficiency.

Bloom filters are used in a wide variety of applications such as ad
serving - making sure a user doesn't see an ad too often; likewise in
content recommendation systems - ensuring recommendations don't appear
too often, in databases - quickly checking if an entry exists in a
table before accessing it on disk, and so on.

title: How Bloom filters work

text:
Most of the literature on Bloom filter uses highly symbolic and/or
mathematical descriptions to describe it. If you're mathematically
challenged like yours truly, you might find my explanation more useful.

A Bloom filter is an array of many bits. When an element is 'added' to a
bloom filter, the element is hashed. Then *bit\[hashval % nbits\]* is
set to 1. This looks fairly similar to how buckets in a hash table are
mapped. To check if an item is present or not, the hash is computed and
the filter sees if the corresponding bit is set or not.

Of course, this is subject to collisions. If a collision happens, the
filter returns a false positive - indicating that the entry is
indeed found (note that a bloom filter never returns a false
negative, that is, claim that something does not exist when it fact it
is present).

In order to reduce the risk of collisions, an entry may use more than
one bit: the entry is hashed *bits_per_element (bpe)* times with a
different seed for each iteration resulting in a different hash value,
and for each hash value, the corresponding *hash % nbits* bit is set. To
check if an entry exists, the candidate key is also hashed *bpe* times,
and if any corresponding bit is *unset*, then it can be determined with
certainty that the item does *not* exist.

The actual value of *bpe* is determined at the time the filter is
created. Generally the more bits per element, the lower the likelihood
of false positives.

In the example above, all three bits would need to be set in order for
the filter to return a positive result.

Another value affecting the accuracy of a Bloom filter is its *fill
ratio*, or how many bits in the filter are actually set. If a filter has
a vast majority of bits set, the likelihood of any specific lookup
returning false is decreased, and thus the possibility of the filter
returning false positives is increased.

title: Scalable Bloom filters

text:
Typically Bloom filters must be created with a foreknowledge of how many
entries they contain. The *bpe* number needs to be fixed, and
likewise, the width of the bit array is also fixed.
Unlike hash tables, Bloom filters cannot be "rebalanced" because there
is no way to know *which* entries are part of the filter (the filter can
only determine whether a given entry is *not* present, but does not
actually store the entries which *are* present).

In order to allow Bloom filters to 'scale' and be able to accommodate
more elements than they've been designed to, they may be stacked. Once a
single Bloom filter reaches capacity, a new one is created atop it.
Typically the new filter has greater capacity than the previous
one in order to reduce the likelihood of needing to stack yet another
filter.

In a stackable (scalable) Bloom filter, checking for membership now
involves inspecting each layer for presence. Adding new items now
involves checking that it does not exist beforehand, and adding it to
the current filter. Hashes still only need to be computed once, however.

When creating a Bloom filter - even a scalable one, it's important to
have a good idea of how many items it is expected to contain. A filter
whose initial layer can only contain a small number of elements will
degrade performance significantly because it will take more layers to
reach a larger capacity.

title: More info

text:
- [Probabilistic data structures quick start]()
- [Probabilistic data structure commands]()
- [Probabilistic data structure configuration]()
- [RedisBloom source](https://github.com/RedisBloom/RedisBloom/)


title: Graph commands

summary: Lists graph commands and provides links to the command reference pages.

title: Content

text:
The following table lists graph commands. See the command links for more information about each command's syntax, arguments, and examples.

| Command | Redis Software | Redis Cloud
Flexible & Annual | Redis Cloud
Free & Fixed | Description |
|:--------|:----------------------|:-----------------|:-----------------|:------|
| [GRAPH.CONFIG GET](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.config-get.md) | <nobr>❌ Not supported | <nobr>⚠ Not supported[2](#table-note-2) | <nobr>❌ Not supported</nobr> | Returns the current value of a [RedisGraph configuration parameter](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/configuration.md#redisgraph-configuration-parameters). |
| [GRAPH.CONFIG SET](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.config-set.md) | <nobr>⚠ Not supported[1](#table-note-1) | <nobr>⚠ Not supported[2](#table-note-2) | <nobr>❌ Not supported</nobr> | Changes the value of a [RedisGraph configuration parameter](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/configuration.md#redisgraph-configuration-parameters). |
| [GRAPH.DELETE](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.delete.md) | ✅ Supported | ✅ Supported | ✅ Supported</nobr> | Removes the graph and its entities. |
| [GRAPH.EXPLAIN](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.explain.md) | ✅ Supported | ✅ Supported | ✅ Supported</nobr> | Displays the query execution plan but does not run the query. |
| [GRAPH.LIST](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.list.md) | ✅ Supported | ✅ Supported | ✅ Supported</nobr> | Lists all graph keys. |
| [GRAPH.PROFILE](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.profile.md) | ✅ Supported | ✅ Supported | ✅ Supported</nobr> | Runs a query and displays the execution plan with metrics for each operation. |
| [GRAPH.QUERY](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.query.md) | ✅ Supported | ✅ Supported | ✅ Supported</nobr> | Runs a query against a graph. Supports a variety of [clauses](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.query.md#query-structure) and [functions](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.query.md#functions). |
| [GRAPH.RO_QUERY](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.ro_query.md) | ✅ Supported | ✅ Supported | ✅ Supported</nobr> | Runs a read-only query against a graph. |
| [GRAPH.SLOWLOG](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.slowlog.md) | ✅ Supported | ✅ Supported | ✅ Supported</nobr> | Returns the slowest 10 queries run against a specific graph. |

1. Use [`rladmin`]() or the [REST API]() to change RedisGraph configuration for Redis Software. See [RedisGraph configuration compatibility with Redis Software]() for more information and examples.

2. [Contact support](https://redis.com/company/support/) to view the current RedisGraph configuration values or request configuration changes for Flexible or Annual Redis Cloud subscriptions.


title: Graph configuration compatibility with Redis Software

summary: Graph configuration settings supported by Redis Software.

title: Content

text:
You cannot use `GRAPH.CONFIG SET` to configure RedisGraph in [Redis Software]() or [Redis Cloud](). Instead, use one of the following methods.

For Redis Cloud:

- _Flexible or Annual [subscriptions]()_: contact [support](https://redis.com/company/support/) to request a configuration change.

- _Free or Fixed subscriptions_: you cannot change RedisGraph configuration.

For Redis Software, use either:

- [`rladmin tune db`]():

    [code example]

- [Configure module]() REST API request:

    [code example]

| Setting | Redis
Software | Redis
Cloud | Notes |
|:--------|:----------------------|:-----------------|:------|
| [CACHE_SIZE](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/configuration.md#cache_size) | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> | Default: 25 |
| [MAX_QUEUED_QUERIES](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/configuration.md#max_queued_queries) | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> | Default: 25 |
| [NODE_CREATION_BUFFER](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/configuration.md#node_creation_buffer) | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> | Default: 16384 |
| [OMP_THREAD_COUNT](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/configuration.md#omp_thread_count) | ✅ Supported

 | ✅ Flexible & Annual\*
<nobr>❌ Free & Fixed</nobr> | \* Updates automatically when you change your plan.

Redis Software default: Set by plan

Redis Cloud defaults:
• Flexible & Annual: Set by plan
• Free & Fixed: 1
 |
| [QUERY_MEM_CAPACITY](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/configuration.md#query_mem_capacity) | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> | Default: 100000000 |
| [RESULTSET_SIZE](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/configuration.md#resultset_size) | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> | Default: 10000 |
| [THREAD_COUNT](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/configuration.md#thread_count) | ✅ Supported

 | ✅ Flexible & Annual\*
<nobr>❌ Free & Fixed</nobr> | \* Updates automatically when you change your plan.

Redis Software default: Set by plan

Redis Cloud defaults:
• Flexible & Annual: Set by plan
• Free & Fixed: 1
 |
| [TIMEOUT](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/configuration.md#timeout) | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> | Default: 100 |
| [VKEY_MAX_ENTITY_COUNT](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/configuration.md#vkey_max_entity_count) | ✅ Supported

 | ✅ Flexible & Annual
<nobr>❌ Free & Fixed</nobr> | Default: 100000 |

code:
$ rladmin tune db db:<ID|name> module_name graph \
        module_config_params "setting-name setting-value"

section_id: content

code:
POST /v1/modules/config/bdb/<ID>
    {
      "modules": [
        {
          "module_name": "graph",
          "module_args": "setting-name setting-value"
        }
      ]
    }


title: Graph quick start

summary: Graph quick start

title: Prerequisites

text:
For this quick start tutorial, you need:

- A Redis database with the RedisGraph module enabled. You can use either:

    - A [Redis Cloud]() database

    - A [Redis Software]() database

- [`redis-cli`]() command-line tool

- [`redis-py`](https://github.com/redis/redis-py) client library v4.1.0 or later

title: Graph with `redis-cli`

text: To begin, [connect to your database]() with `redis-cli`.

title: Create a graph

text:
When you create a graph, you can define nodes and the relationships between them with this format:

`(:<node 1>)-[:<relationship>]->(:<node 2>)`

To define multiple nodes and relationships in a single creation query, separate the entries with commas.

For example, use the [`CREATE`](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.query.md#create) query to create a new graph of motorcycle riders and teams participating in the MotoGP league:

[code example]

title: Add nodes

text:
You can add new nodes to a previously created graph:

[code example]

title: Add relationships

text:
To create new relationships between nodes of a graph:

[code example]

title: Query the graph

text:
After you create a graph, you can use the [`GRAPH.QUERY`](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.query.md) command to query the graph's data.

The following example returns which motorcycle riders compete for team Yamaha:

[code example]

You can also use [functions](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.query.md#functions) to create more complex queries.

For example, you can use the `count` function to check how many riders represent team Honda:

[code example]

title: Delete nodes

text:
You can use the [`DELETE`](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.query.md#delete) query to remove a specific node and its relationships from the graph:

[code example]

title: Delete relationships

text:
You can also use the [`DELETE`](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.query.md#delete) query to delete a node's relationships without removing any nodes:

[code example]

title: Delete a graph

text:
To delete an entire graph, including all nodes and relationships, run the [`GRAPH.DELETE`](https://github.com/RedisGraph/RedisGraph/blob/master/docs/commands/graph.delete.md) command:

[code example]

title: Graph with Python

text:
If you want to use graphs within an application, you can use one of these [client libraries](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/clients.md).

The following example uses the Redis Python client library [redis-py](https://github.com/redis/redis-py), which supports graph commands as of v4.1.0.

This Python code creates a graph that represents friendships between users on a social media website. It also shows how to run queries and change relationships between users.

[code example]

Example output:

[code example]

title: Visualize graphs with Redis Insight

text:
You can use the [Redis Insight]() workbench to visualize the relationships between the nodes of your graph.

1. Connect to your database with Redis Insight. You can [connect manually]() or use the [auto-discovery]() feature.

1. Select the **Workbench** button.

1. Enter a graph query in the text editor.

   For example, this query returns all nodes and relationships in the graph:

   [code example]

1. Select **Run** to run the query.

After you run a query, the output log displays a visual representation of your graph's nodes and relationships:

title: More info

text:
- [Graph commands]()
- [RedisGraph client libraries](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/clients.md)
- [RedisGraph documentation on GitHub](https://github.com/RedisGraph/RedisGraph/tree/master/docs/docs)

code:
127.0.0.1:12543> GRAPH.QUERY MotoGP "CREATE (:Rider {name:'Valentino Rossi'})-[:rides]->(:Team {name:'Yamaha'}), (:Rider {name:'Dani Pedrosa'})-[:rides]->(:Team {name:'Honda'}), (:Rider {name:'Andrea Dovizioso'})-[:rides]->(:Team {name:'Ducati'})"
1) 1) "Labels added: 2"
   2) "Nodes created: 6"
   3) "Properties set: 6"
   4) "Relationships created: 3"
   5) "Cached execution: 0"
   6) "Query internal execution time: 0.385472 milliseconds"

section_id: create-a-graph

code:
127.0.0.1:12543> GRAPH.QUERY MotoGP "CREATE (:Rider {name:'Jorge Lorenzo'})"
1) 1) "Nodes created: 1"
   2) "Properties set: 1"
   3) "Cached execution: 0"
   4) "Query internal execution time: 0.185841 milliseconds"

section_id: add-nodes

code:
127.0.0.1:12543> GRAPH.QUERY MotoGP "MATCH (r:Rider), (t:Team) WHERE r.name = 'Jorge Lorenzo' and t.name = 'Honda' CREATE (r)-[:rides]->(t)"
1) 1) "Relationships created: 1"
   2) "Cached execution: 0"
   3) "Query internal execution time: 0.356578 milliseconds"

section_id: add-relationships

code:
127.0.0.1:12543> GRAPH.QUERY MotoGP "MATCH (r:Rider)-[:rides]->(t:Team) WHERE t.name = 'Yamaha' RETURN r,t"
1) 1) "r"
   2) "t"
2) 1) 1) 1) 1) "id"
            2) "0"
         2) 1) "labels"
            2) 1) "Rider"
         3) 1) "properties"
            2) 1) 1) "name"
                  2) "Valentino Rossi"
      2) 1) 1) "id"
            2) "1"
         2) 1) "labels"
            2) 1) "Team"
         3) 1) "properties"
            2) 1) 1) "name"
                  2) "Yamaha"
3) 1) "Cached execution: 0"
   2) "Query internal execution time: 0.500535 milliseconds"

section_id: query-the-graph

code:
127.0.0.1:12543> GRAPH.QUERY MotoGP "MATCH (r:Rider)-[:rides]->(t:Team {name:'Honda'}) RETURN count(r)"
1) 1) "count(r)"
2) 1) 1) "2"
3) 1) "Cached execution: 0"
   2) "Query internal execution time: 0.445760 milliseconds"

code:
127.0.0.1:12543> GRAPH.QUERY MotoGP "MATCH (r:Rider {name: 'Dani Pedrosa'}) DELETE r"
1) 1) "Nodes deleted: 1"
   2) "Relationships deleted: 1"
   3) "Cached execution: 0"
   4) "Query internal execution time: 0.276815 milliseconds"

section_id: delete-nodes

code:
127.0.0.1:12543> GRAPH.QUERY MotoGP "MATCH (:Rider {name: 'Valentino Rossi'})-[r:rides]->() DELETE r"
1) 1) "Relationships deleted: 1"
   2) "Cached execution: 0"
   3) "Query internal execution time: 0.348346 milliseconds"

section_id: delete-relationships

code:
127.0.0.1:12543> GRAPH.DELETE MotoGP
"Graph removed, internal execution time: 0.013138 milliseconds"

section_id: delete-a-graph

code:
import redis
from redis.commands.graph.edge import Edge
from redis.commands.graph.node import Node

# Connect to a database
r = redis.Redis(host="<endpoint>", port="<port>",
                password="<password>")

# Create nodes that represent users
users = { "Alex": Node(label="Person", properties={"name": "Alex", "age": 35}),
          "Jun": Node(label="Person", properties={"name": "Jun", "age": 33}),
          "Taylor": Node(label="Person", properties={"name": "Taylor", "age": 28}),
          "Noor": Node(label="Person", properties={"name": "Noor", "age": 41}) }

# Define a graph called SocialMedia
social_graph = r.graph("SocialMedia")

# Add users to the graph as nodes
for key in users.keys():
    social_graph.add_node(users[key])

# Add relationships between user nodes
social_graph.add_edge( Edge(users["Alex"], "friends", users["Jun"]) )
# Make the relationship bidirectional
social_graph.add_edge( Edge(users["Jun"], "friends", users["Alex"]) )

social_graph.add_edge( Edge(users["Jun"], "friends", users["Taylor"]) )
social_graph.add_edge( Edge(users["Taylor"], "friends", users["Jun"]) )

social_graph.add_edge( Edge(users["Jun"], "friends", users["Noor"]) )
social_graph.add_edge( Edge(users["Noor"], "friends", users["Jun"]) )

social_graph.add_edge( Edge(users["Alex"], "friends", users["Noor"]) )
social_graph.add_edge( Edge(users["Noor"], "friends", users["Alex"]) )

# Create the graph in the database
social_graph.commit()

# Query the graph to find out how many friends Alex has
result1 = social_graph.query("MATCH (p1:Person {name: 'Alex'})-[:friends]->(p2:Person) RETURN count(p2)")
print("Alex's original friend count:", result1.result_set)

# Delete a relationship without deleting any user nodes
social_graph.query("MATCH (:Person {name: 'Alex'})<-[f:friends]->(:Person {name: 'Jun'}) DELETE f")

# Query the graph again to see Alex's updated friend count
result2 = social_graph.query("MATCH (p1:Person {name: 'Alex'})-[:friends]->(p2:Person) RETURN count(p2)")
print("Alex's updated friend count:", result2.result_set)

# Delete the entire graph
social_graph.delete()

section_id: graph-with-python

code:
$ ./quick_start.py
Alex's original friend count: [[2]]
Alex's updated friend count: [[1]]

code: GRAPH.QUERY MotoGP "MATCH (n) RETURN n"

section_id: visualize-graphs-with-redis-insight


title: Graph

summary: RedisGraph is a queryable graph database built on Redis.

summary: Graph quick start

title: Graph quick start

title: Graph commands

title: Overview

text: RedisGraph is the first queryable [Property Graph](https://github.com/opencypher/openCypher/blob/master/docs/property-graph-model.adoc) database to use [sparse matrices](https://en.wikipedia.org/wiki/Sparse_matrix) to represent the [adjacency matrix](https://en.wikipedia.org/wiki/Adjacency_matrix) in graphs and [linear algebra](http://faculty.cse.tamu.edu/davis/GraphBLAS.html) to query the graph.

title: Primary features

text:
- Based on the [Property Graph Model](https://github.com/opencypher/openCypher/blob/master/docs/property-graph-model.adoc)
    - Nodes (vertices) and Relationships (edges) that may have attributes
    - Nodes that can be labeled
    - Relationships have a relationship type
- Graphs represented as sparse adjacency matrices
- [Cypher](http://www.opencypher.org/) as query language
    - Cypher queries translated into linear algebra expressions

title: Commands

role: syntax

text: See [Graph commands]() for a list of graph commands that are supported in Redis Software.

title: Clients

text: For a list of available RedisGraph clients, see [RedisGraph client libraries](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/clients.md).

title: Configuration

text:
See [Configuration parameters](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/configuration.md) for the complete list of RedisGraph configuration parameters.

To learn which configuration parameters are supported in Redis Software, see [Graph configuration compatibility with Redis Software]().

title: Design

text: For an overview of RedisGraph's design, see [RedisGraph: A High Performance In-Memory Graph Database](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/design/_index.md).

title: Client specification

text: RedisGraph client libraries should follow the [Technical specification for writing RedisGraph client libraries](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/design/client_spec.md).

title: Result set structure

role: returns

text: [RedisGraph result set structure](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/design/result_structure.md) describes the format RedisGraph uses to print data when using `redis-cli`.

title: GRAPH.BULK endpoint

text:
You can import new graphs from CSV files using `redisgraph-bulk-loader`.

See the [`GRAPH.BULK` implementation details](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/design/bulk_spec.md) for more information.

title: Data types

text:
RedisGraph supports a number of distinct data types, including:

- Graph types:
    - Nodes
    - Relationships
    - Paths

- Scalar types:
    - Strings
    - Booleans
    - Integers
    - Floating-point values
    - Geospatial points
    - Null

- Collection types:
    - Arrays
    - Maps

See [RedisGraph data types](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/datatypes.md) for details.

title: Cypher coverage

text: RedisGraph implements a subset of the Cypher language. See [Cypher coverage](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/cypher_support.md) for more information.

title: Path algorithms

text: To learn how to find single-pair and single-source paths using `algo.SPpaths` and `algo.SSpaths`, see [Path algorithms](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/path_algorithm.md).

title: Known limitations

text: RedisGraph has several [known limitations](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/known_limitations.md).

title: More info

text:
- [Graph quick start]()

- [RedisGraph documentation on GitHub](https://github.com/RedisGraph/RedisGraph/tree/master/docs/docs)

- [References](https://github.com/RedisGraph/RedisGraph/blob/master/docs/docs/References.md)

- [RedisGraph source](https://github.com/RedisGraph/RedisGraph/)


title: Deprecated Redis Open Source features and modules

summary: Features (modules) that have been deprecated or replaced with a new version.

summary: Trigger and execute JavaScript functions in the Redis process

title: Triggers and functions

title: Graph

title: Content

text: The following features (modules) have been deprecated or replaced with a new version:


title: Binary data

summary: Working with binary data

title: Overview

text:
By default, triggers and functions will decode all data as a string and will raise error on failures. Though useful for most users, sometimes there is a need to work with binary data. In order to do so, the library developer has to consider the following:

1. Binary function arguments
2. Binary command results
3. Binary key names on [keyspace triggers]()
4. Binary data on [stream triggers]()

title: Binary function arguments

text:
It is possible to instruct triggers and functions not to decode function arguments as `JS` `Strings` using the [redis.functionFlags.RAW_ARGUMENTS]() function flag. In this case, the function arguments will be given as `JS` `ArrayBuffer`. Example:

[code example]

The above example will allow us to set `key` and `val` even if those are binary data. Run example:

[code example]

Notice that the `call` function also accepts `JS` `ArrayBuffer` arguments.

title: Binary command results

text:
Getting function arguments as binary data is not enough. We might want to read binary data from a Redis key. In order to do this we can use the `callRaw` function, which will not decode the result as a `JS` `String` and instead will return the result as a `JS` `ArrayBuffer`. Example:

[code example]

The above example will be able to fetch binary data and return it to the user. For example:

[code example]

Notice that a `JS` `ArrayBuffer` can be returned by a function, it will be returned to the client as `bulk string`.

title: Binary keys names on database triggers

text:
On [keyspace triggers](), if the key name that triggered the event is binary, the `data.key` field will be NULL. The `data.key_raw` field is always provided as a `JS` `ArrayBuffer` and can be used as in the following example:

[code example]

Run example:

[code example]

For more information see [keyspace triggers]().

title: Binary data on stream consumers

text:
On [stream triggers](), if the key name is binary. The `data.stream_name` field will be NULL. The `data.stream_name_raw` field is always provided as a `JS` `ArrayBuffer` and can be used in this case. In addition, if the content of the steam is binary, it will also appear as `null` under `data.record`. In this case, it is possible to use `data.record` (which always exists) and contains the data as a `JS` `ArrayBuffer`. Example:

[code example]

Run Example:

[code example]

code:
#!js api_version=1.0 name=lib
redis.registerFunction("my_set",
    (c, key, val) => {
        return c.call("set", key, val);
    },
    {
        flags: [redis.functionFlags.RAW_ARGUMENTS]
    }
);

section_id: binary-function-arguments

code:
#!js api_version=1.0 name=lib
redis.registerFunction("my_get",
    (c, key) => {
        return c.callRaw("get", key);
    },
    {
        flags: [redis.functionFalgs.RAW_ARGUMENTS]
    }
);

section_id: binary-command-results

code:
#!js api_version=1.0 name=lib
/* The following is just an example, in general it is discourage to use globals. */
var n_notifications = 0;
var last_key = null;
var last_key_raw = null;
redis.registerKeySpaceTrigger("consumer", "", function(client, data) {
    if (data.event == "set") {
        n_notifications += 1;
        last_data = data.key;
        last_key_raw = data.key_raw;
    }
});

redis.registerFunction("notifications_stats", async function(){
    return [
        n_notifications,
        last_key,
        last_key_raw
    ];
});

section_id: binary-keys-names-on-database-triggers

code:
#!js api_version=1.0 name=lib
/* The following is just an example, in general it is discourage to use globals. */
var last_key = null;
var last_key_raw = null;
var last_data = null;
var last_data_raw = null;
redis.registerFunction("stats", function(){
    return [
        last_key,
        last_key_raw,
        last_data,
        last_data_raw
    ];
})
redis.registerStreamTrigger("consumer", new Uint8Array([255]).buffer, function(c, data){
    last_key = data.stream_name;
    last_key_raw = data.stream_name_raw;
    last_data = data.record;
    last_data_raw = data.record_raw;
})

section_id: binary-data-on-stream-consumers


title: Cluster support

summary: Cluster support for triggers and functions

title: Overview

text:
**Notice**: On a Redis Open Source cluster, before executing any gears function, you must send a `REDISGEARS_2.REFRESHCLUSTER` command to all the shards so that they will be aware of the cluster topology. Without this step, each shard will act as a single Redis Open Source instance.

Triggers and functions support cross-shard operation on Redis clusters. This means that it is possible to call a function that will be invoked on another shard. We call such a function a remote function.

Just like local functions, remote function must be declared on library load time using `redis.registerClusterFunction` API. The following example declares a remote function that returns the number of keys on the shard:

[code example]

`redis.registerClusterFunction` is passed the remote function name, which will be used later to call the remote function, and the remote function code. The remote function must be a Coroutine (async function) and it is executed in the background on the remote shard. For more information about async function, please refer to [sync and async]() page.

We have couple of options for calling a remote function. These options are exposed through the async client that is given to a Coroutine:

* `async_client.runOnShards` - run the remote function on all the shards (including the current shard). Returns a promise that, once resolved, will give two nested arrays, the first contains another array with the results from all the shards and the other contains an array of errors (`[[res1, res2, ...],[err1, err2, ..]]`).
* `async_client.runOnKey` - run the remote function on the shard responsible for a given key. Returns a promise that, once resolved, will give the result from the remote function execution or raise an exception in the case of an error.

The following example registers a function that will return the total number of keys on the cluster. The function will use the remote function defined above:

[code example]

First, the function executes a remote function on all shards that returns the number of keys on each shard, then the function adds all the results and returns total number of keys for all shards.

Here's the complete example:

[code example]

title: Arguments and results serialization

role: parameters

text:
It is possible to pass arguments to the remote function. The arguments will be given to the remote function after the `async_client`. The following example shows how to get a value of a key from any shard in the cluster:

[code example]

The function `test` will return the value of `x` regardless of which shard the function was executed on.

The remote function arguments and results are serialized in the following way:

1. If the argument is of type `ArrayBuffer`, the data will be sent as is.
2. Otherwise, RedisGears will try to serialize the give arguments (or the return value) as JSON using `JSON.stringify`. A serialization failure will cause an error to be raised.

title: Execution timeout

text:
Remote functions will not be permitted to run forever and will timeout. The timeout period can be configured using [remote-task-default-timeout]().
When using `async_client.runOnShards` API, the timeout will be added as error to the error array. When using `async_client.runOnKey`, a timeout will cause an exception to be raised.

title: Remote function limitations

text:
All the limitations listed on [coroutines]() also apply to remote functions. Remote function also come with some extra limitations:

* Remote functions can only perform read operations. An attempt to perform a write operation will result in an error.
* Remote function are not guaranteed to succeed (if the shard crashed for example). In such cases a timeout error will be given.

code:
redis.registerClusterFunction("dbsize", async(async_client) => {
    return client.block((async_client) => {
        return client.call("dbsize").toString();
    });
});

section_id: overview

code:
redis.registerAsyncFunction("my_dbsize", async(async_client) => {
    let res = await async_client.runOnShards("dbsize");
    let results = res[0];
    let errors = res[1];
    if (errors.length > 0) {
        return errors;
    }
    let sum = BigInt(0);
    results.forEach((element) => sum+=BigInt(element));
    return sum;
});

code:
#!js name=lib api_version=1.0

redis.registerClusterFunction("dbsize", async(client) => {
    return client.block((client) => {
        return client.call("dbsize").toString();
    });
});

redis.registerAsyncFunction("test", async(async_client) => {
    let res = await async_client.runOnShards("dbsize");
    let results = res[0];
    let errors = res[1];
    if (errors.length > 0) {
        return errors;
    }
    let sum = BigInt(0);
    results.forEach((element) => sum+=BigInt(element));
    return sum;
});

code:
#!js name=lib api_version=1.0
const remote_get = "remote_get";

redis.registerClusterFunction(remote_get, async(client, key) => {
    let res = client.block((client) => {
        return client.call("get", key);
    });
    return res;
});

redis.registerAsyncFunction("test", async (async_client) => {
    return await async_client.runOnKey("x", remote_get, "x");
});

section_id: arguments-and-results-serialization


title: Function flags

summary: Function flags for JavaScript functions

title: Content

text:
When registering a function, it is possible to include additional information about its behavior. This information is known as function flags. Function flags are an optional argument that can be specified after the function implementation. The following flags are supported:
1. `redis.functionFlags.NO_WRITES`: This flag indicates that the function does not perform any write commands. Enabling this flag allows a function to be executed on read-only replicas or in out-of-memory (OOM) situations. Redis enforces this flag's behavior, meaning that any attempt to call a write command within a function that has this flag set will result in an exception.
2. `redis.functionFlags.ALLOW_OOM`: By default, Redis prevents any function from running in an OOM scenario. However, this flag allows overriding this behavior and running a function even when there is a memory shortage. Enabling this flag is considered unsafe and may cause Redis to exceed the `maxmemory` limit. **Users should only enable this flag if they are certain that their function does not consume additional memory.** For example, it is safe to run a function that only deletes data during an OOM situation.
3. `redis.functionFlags.RAW_ARGUMENTS`: By default, Redis attempts to decode all function arguments as `JS` `String`s. If the decoding fails, an error is returned to the client. However, when this flag is set, Redis avoids string decoding and passes the argument as a `JS` `ArrayBuffer` instead.

The following example shows how to set the `redis.functionFlags.NO_WRITES` flag:

[code example]

Run example:

[code example]

code:
#!js api_version=1.0 name=lib

redis.registerFunction('my_ping',
    function(client){
        return client.call('ping');
    },
    {
        flags: [redis.functionFlags.NO_WRITES]
    }
);

code:
127.0.0.1:6379> TFCALL lib.my_ping 0
"PONG"
127.0.0.1:6379> config set maxmemory 1
OK
127.0.0.1:6379> TFCALL lib.my_ping 0
"PONG"


title: Concepts

summary: Details about triggers, the JavaScript API, and advanced concepts

summary: Different ways of executing JavaScript functions

title: Triggers

summary: Overview of the JavaScript API

title: JavaScript API

summary: Sync and async functions

title: Sync and async

title: Cluster support

summary: Function flags for JavaScript functions

title: Function flags

summary: How to use configuration in JavaScript functions

title: Library configuration

summary: Converting RESP to and from JavaScript

title: RESP & JavaScript

summary: Working with binary data

title: Binary data

page_type: index


title: JavaScript API

summary: Overview of the JavaScript API

title: Redis object

text:
* Since version: 2.0.0

The triggers and functions JavaScript API provides a singleton instance of an object named *redis*. The *redis* instance enables registered functions to interact with the Redis server on which they are running. Following is the API provided by the *redis* instance.

title: `redis.registerFunction`

text:
* Since version: 2.0.0

Register a new function that can later be invoke using `TFCALL` command.

[code example]

title: `redis.registerAsyncFunction`

text:
* Since version: 2.0.0

Register a new async function that can later be invoke using `TFCALLASYNC` command.

[code example]

title: `redis.registerKeySpaceTrigger`

text:
* Since version: 2.0.0

Register a key space notification trigger that will run whenever a key space notification fired.

[code example]

title: `redis.registerStreamTrigger`

text:
* Since version: 2.0.0

Register a stream trigger that will be invoke whenever a data is added to a stream.

[code example]

title: `redis.registerClusterFunction`

text:
* Since version: 2.0.0

Register a cluster function that can later be called using `async_client.runOnKey()` or `async_client.runOnShards()`

[code example]

title: `redis.config`

text:
* Since version: 2.0.0

[code example]

[code example]

title: Client object

text: Client object that is used to perform operations on Redis.

title: `client.call`

text:
* Since version: 2.0.0

Run a command on Redis. The command is executed on the current Redis instrance.

[code example]

title: `client.callRaw`

text:
* Since version: 2.0.0

Run a command on Redis but does not perform UTF8 decoding on the result. The command is executed on the current Redis instrance.

[code example]

title: `client.callAsync`

text:
* Since version: 2.0.0

Call a command on Redis. Allow Redis to block the execution if needed (like `blpop` command) and return the result asynchronously. Returns a promise object that will be resolved when the command invocation finished.

[code example]

title: `client.callAsyncRaw`

text:
* Since version: 2.0.0

Call a command on Redis. Allow Redis to block the execution if needed (like `blpop` command) and return the result asynchronously. Returns a promise object that will be resolved when the command invocation finished. The command is executed on the current Redis instrance.

[code example]

title: `client.isBlockAllowed`

text:
* Since version: 2.0.0

Return true if it is allow to return promise from the function callback. In case it is allowed and a promise is return, Redis will wait for the promise to be fulfilled and only then will return the function result.

[code example]

title: `client.executeAsync`

text:
* Since version: 2.0.0

Execute the given function asynchroniusly. Return a promise that will be fulfilled when the promise will be resolved/rejected.

[code example]

title: Async client object

text:
Background client object that is used to perform background operation on Redis.
This client is given to any background task that runs as a JS coroutine.

title: `async_client.block`

text:
* Since version: 2.0.0

Blocks Redis for command invocation. All the command that are executed inside the given function is considered atomic and will be wrapped with `Multi/Exec` and send to the replica/AOF.

[code example]

title: `async_client.runOnKey`

text:
* Since version: 2.0.0

Runs a remote function on a given key. If the key located on the current shard on which we currently runs on, the remote function will run write away. Otherwise the remote function will run on the remote shard. Returns a promise which will be fulfilled when the invocation finishes. Notice that the remote function must return a json serializable result so we can serialize the result back to the original shard.

Notice that remote function can only perform read operations, not writes are allowed.

[code example]

title: `async_client.runOnShards`

text:
* Since version: 2.0.0

Runs a remote function on all the shards. Returns a promise which will be fulfilled when the invocation finishes on all the shards.

The result is array of 2 elements, the first is another array of all the results. The second is an array of all the errors happened durring the invocation. Notice that the remote function must return a json serializable result so we can serialize the result back to the original shard.

Notice that remote function can only perform read operations, not writes are allowed.

[code example]

code:
//name and callback mandatory
//object with optional arguments
redis.registerFunction(
  'foo', //name
  function(client, args) {}, //callback
  {
    description: 'The description',
    flags: [redis.functionFlags.NO_WRITES, redis.functionFlags.ALLOW_OOM]
  } // optional arguments
);

section_id: redis-registerfunction

code:
redis.registerAsyncFunction(
  'foo', //Function name
  function(client, args){}, //callback
  {
    description: 'description',
    flags: [redis.functionFlags.NO_WRITES, redis.functionFlags.ALLOW_OOM]
  } //optional arguments
);

section_id: redis-registerasyncfunction

code:
redis.registerKeySpaceTrigger(
  'foo', // trigger name
  'keys:*', //key prefix
  function(client, data) {}, //callback
  {
    description: 'description'
    onTriggerFired: function(client, data){}
  } //optional arguments
)

section_id: redis-registerkeyspacetrigger

code:
redis.registerStreamTrigger(
  'foo', //trigger name
  'stream:*', //prefix
  function(client, data){},//callback
  {
    description: 'Description'
    window: 1,
    isStreamTrimmed: false
  } //optional arguments
)

section_id: redis-registerstreamtrigger

code:
redis.registerClusterFunction(
  'foo', //name
  async function(client, ...args){} //callback
)

section_id: redis-registerclusterfunction

code: redis-cli -x RG.FUNCTION LOAD UPGRADE CONFIG '{"example_config": "example_config_value"}' < main.js

section_id: redis-config

code: var configExample = redis.config.example_config;

code:
client.call(
  '',
  ...args
)

section_id: client-call

code:
client.callRaw(
  '',
  ...args
)

section_id: client-callraw

code:
client.callAsync(
  '',
  ...args
)

section_id: client-callasync

section_id: client-callasyncraw

code: client.isBlockAllowed(); // True / False

section_id: client-isblockallowed

code:
client.executeAsync(
  async function(client){}
)

section_id: client-executeasync

code:
async_client.block(
  function(client){}
)

section_id: async-client-block

code:
async_client.runOnKey(
  'key1', // key
  'foo', // function name
  ...args
)

section_id: async-client-runonkey

code:
async_client.runOnShards(
  'foo', //name
  ...args //arguments
)

section_id: async-client-runonshards


title: Library configuration

title: Content

text:
When writing a library, you may want to provide a loading configuration so that different users can use the same library with slightly different behaviour, without changing the base code. For example, assume you write a library that adds a `__last_updated__` field to a hash (you can see how it can also be done with [keyspace triggers]()), the code will look like this:

[code example]

Run example:

[code example]

The problem with the above code is that the `__last_update__` field is hard coded. What if we want to allow the user to configure it at runtime? Triggers and functions provide for specifying a library configuration at load time using a [`CONFIG`]() argument that is passed to the `TFUNCTION LOAD` command. The configuration argument accepts a string representation of a JSON object. The JSON will be provided to the library as a JS object under the `redis.config` variable. We can change the above example to accept the `__last_update__` field name as a library configuration. The code will look like this:

[code example]

Notice that in the above example we first set `last_update_field_name` to `__last_update__`, the default value in cases where a value is not provided by the configuration. Then we check if we have `last_update_field_name` in our configuration and if we do we use it. We can now load our function with a [`CONFIG`]() argument:

[code example]

We can see that the last update field name is `last_update`:

[code example]

Notice, triggers and functions only provides the library with the JSON configuration. **It's the library's responsibility to verify the correctness of the given configuration**.

code:
#!js api_version=1.0 name=lib

redis.registerFunction("hset", function(client, key, field, val){
    // get the current time in ms
    var curr_time = client.call("time")[0];
    return client.call('hset', key, field, val, "__last_update__", curr_time);
});

section_id: content

code:
#!js api_version=1.0 name=lib

var last_update_field_name = "__last_update__"

if (redis.config.last_update_field_name !== undefined) {
    if (typeof redis.config.last_update_field_name != 'string') {
        throw "last_update_field_name must be a string";
    }
    last_update_field_name = redis.config.last_update_field_name
}

redis.registerFunction("hset", function(client, key, field, val){
    // get the current time in ms
    var curr_time = client.call("time")[0];
    return client.call('hset', key, field, val, last_update_field_name, curr_time);
});

code:
> redis-cli -x TFUNCTION LOAD REPLACE CONFIG '{"last_update_field_name":"last_update"}' < <path to code file>
OK


title: RESP & JavaScript

summary: Converting RESP to and from JavaScript

title: Overview

text:
When running Redis commands from within a function using the `client.call` API, the reply is parsed as a resp3 reply and converted to a JS object using the following rules:

| resp 3            | JS object type                                                                                |
|-------------------|------------------------------------------------------------------------------------------------------------------------------------------------|
| `status`          | `StringObject` with a field called `__reply_type` and value `status` (or error if failed to convert to utf8)                                   |
| `bulk string`     | JS `String` (or error if failed to convert to utf8)                                                                                |
| `Error`           | Raise JS exception                                                                                |
| `long`            | JS big integer                                                                                |
| `double`          | JS number                                                                                |
| `array`           | JS array                                                                                |
| `map`             | JS object                                                                                |
| `set`             | JS set                                                                                |
| `bool`            | JS boolean                                                                                |
| `big number`      | `StringObject` with a field called `__reply_type` and value `big_number`                                                                       |
| `verbatim string` | `StringObject` with 2 additional fields: 1. `__reply_type` and value `verbatim` 2. `__format` with the value of the ext in the verbatim string (or error if failed to convert to utf8) |
| `null`            | JS null                                                                                |
|                   |                                                                                |

When running Redis commands from within a function using the `client.callRaw` API, the reply is parsed as a resp3 reply and converted to a JS object using the following rules:

| resp 3            | JS object type                                                                                |
|-------------------|------------------------------------------------------------------------------------------------------------------------------------------------|
| `status`          | JS `ArrayBuffer` with a field called `__reply_type` and value `status`                                                                        |
| `bulk string`     | JS `ArrayBuffer`                                                                                |
| `Error`           | Raise JS exception                                                                                |
| `long`            | JS big integer                                                                                |
| `double`          | JS number                                                                                |
| `array`           | JS array                                                                                |
| `map`             | JS object                                                                                |
| `set`             | JS set                                                                                |
| `bool`            | JS boolean                                                                                |
| `big number`      | `StringObject` with a field called `__reply_type` and value `big_number`                                                                       |
| `verbatim string` | JS `ArrayBuffer` with 2 additional fields: 1. `__reply_type` and value `verbatim` 2. `__format` with the value of the ext in the verbatim string |
| `null`            | JS null                                                                                |
|                   |                                                                                |

title: JavaScript to RESP conversion

text:
| JS type                                                          | RESP2         | RESP3                                  |
|------------------------------------------------------------------|---------------|----------------------------------------|
| `string`                                                         | `bulk string` | `bulk string`                          |
| `string` object with field `__reply_type=status`                 | `status`      | `status`                               |
| Exception                                                        | `error`       | `error`                                |
| `big integer`                                                    | `long`        | `long`                                 |
| `number`                                                         | `bulk string` | `double`                               |
| `array`                                                          | `array`       | `array`                                |
| `map`                                                            | `array`       | `map`                                  |
| `set`                                                            | `array`       | `set`                                  |
| `bool`                                                           | `long`        | `bool`                                 |
| `string` object with field`__reply_type=varbatim` and `__format=txt` | `bulk string` | `verbatim string` with format as `txt` |
| `null`                                                           | resp2 `null`  | resp3 `null`                           |