title: JavaReader

summary: A template for creating custom readers.

title: Overview

text: The `JavaReader` is an abstract class that allows you to create a custom reader in Java.

title: Create a custom reader

text:
To create a custom reader:

- Extend the `JavaReader` class
- Override the `iterator()` function

title: Custom reader example

text:
The implementation of the [`KeysOnlyReader`]() class shows how to create a custom reader with `JavaReader`:

[code example]

code:
import java.util.Iterator;

import gears.GearsBuilder;

/**
 * A reader that only reads key names from the key space
 *
 */
public class KeysOnlyReader extends JavaReader<String> {

	/**
	 *
	 */
	private static final long serialVersionUID = 1L;

	private String scanSize;
	private String pattern;

	/**
	 * Create a new KeysOnlyReader reader
	 * @param scanSize - the size to use with the scan command
	 * @param pattern - the pattern of the keys to read
	 */
	public KeysOnlyReader(int scanSize, String pattern) {
		this.scanSize = Integer.toString(scanSize);
		this.pattern = pattern;
	}

	/**
	 * Create a new KeysOnlyReader reader with default pattern (*) and default
	 * scan size (10000)
	 */
	public KeysOnlyReader() {
		this(10000, "*");
	}

	@Override
	public Iterator<String> iterator() {
		return new Iterator<String>() {

			String cursor = "0";
			int currIndex = 0;
			Object[] keys = null;
			boolean isDone = false;
			String nextKey = null;

			private String innerNext() {
				while(!isDone) {
					if(keys == null) {
						Object[] res = (Object[]) GearsBuilder.execute("scan",
										cursor == null ? "0" : cursor,
										"MATCH", pattern, "COUNT", scanSize);
						keys = (Object[])res[1];
						cursor = (String)res[0];
						currIndex = 0;
					}
					if(currIndex < keys.length) {
						return (String) keys[currIndex++];
					}

					keys = null;
					if(cursor.charAt(0) == '0') {
						isDone = true;
					}
				}
				return null;
			}

			@Override
			public boolean hasNext() {
				if(nextKey == null) {
					nextKey = innerNext();
				}
				return !isDone;
			}

			@Override
			public String next() {
				String temp = nextKey != null ? nextKey : innerNext();
				nextKey = innerNext();
				return temp;
			}

		};
	}

}

section_id: custom-reader-example


title: KeysOnlyReader

summary: Gets key names from a database.

page_type: content

title: Overview

role: overview

text: The `KeysOnlyReader` only extracts the key names from a database.

title: Constructors

role: content

text:
You can use one of these constructors to create a new `KeysOnlyReader` object:

[code example]

title: Parameters

role: parameters

text:
| Name | Type | Default value | Description |
|------|------|---------------|-------------|
| pattern | string | "\*" (match all keys) | Get all keys that match this pattern |
| scanSize | integer | 10000 | The scan command's size limit |

title: Output records

role: returns

text: Each output record is a string that represents the key's name.

title: Examples

role: example

text:
Get all keys in the database:

[code example]

Only get keys that start with "user:":

[code example]

code:
public KeysOnlyReader()

public KeysOnlyReader(int scanSize, String pattern)

section_id: constructors

code: KeysOnlyReader reader = new KeysOnlyReader();

section_id: examples

code: KeysOnlyReader reader = new KeysOnlyReader(1000, "user:*");


title: KeysReader

summary: Gets keys and their values from a database.

title: Overview

text:
Creates records from the keys and values stored in a Redis database.

 Currently only supports string and hash data types. For other data types, it will only extract the key name.

title: Constructors

text:
You can use one of these constructors to create a new `KeysReader` object:

[code example]

title: Parameters

text:
| Name | Type | Default value | Description |
|------|------|---------------|-------------|
| commands | array of strings | null | The commands that this reader is registered on |
| eventTypes | array of strings | null | The [event types]() to register on (usually the command name) |
| keyTypes | array of strings | null | The key types to register on |
| noScan | boolean | false | Whether or not to scan the key space or just read the pattern as is |
| pattern/prefix | string | "\*" (match all keys) | The reader will get all keys that match this pattern |
| readValues | boolean | true | Whether or not to read the keys' values |

title: Output records

text:
Creates a `KeysReaderRecord` for each matching key in the database.

| Name | Type | Description |
|------|------|-------------|
| key | string | The name of the key |
| type | long | The core Redis type: 'string', 'hash', 'list', 'set', 'zset', or 'stream' |
| event | string | The event that triggered the execution (null if using the run function) |
| stringVal | string | The key's value for string data types |
| hashVal | Map<String,String> | The key's value for hash data types |
| listVal | List<String> | The key's value for list data types |
| setVal | Set<String> | The key's value for set data types |

title: Examples

text:
Here's a basic example of a `KeysReader` that creates records for all keys in the database:

[code example]

In the following example, the `KeysReader` creates records for all keys in the database that start with "person:". When registered, it only runs for hashes after `HSET` and `DEL` events occur.

[code example]

code:
public KeysReader()

public KeysReader(String pattern)

public KeysReader(String prefix, boolean readValues)

public KeysReader(String pattern, boolean noScan, boolean readValues)

public KeysReader(String prefix,
                  boolean readValues,
                  String[] eventTypes,
                  String[] keyTypes)

public KeysReader(String pattern,
                  boolean noScan,
                  boolean readValues,
                  String[] eventTypes,
                  String[] keyTypes)

code: KeysReader reader = new KeysReader();

code:
String[] eventTypes = {"HSET", "DEL"};
String[] keyTypes = {"HASH"};
KeysReader reader = new KeysReader("person:*", false, true, eventTypes, keyTypes);


title: ShardsIDReader

summary: Gets the shard ID for each shard in a database.

title: Overview

text:
The `ShardsIDReader` creates a single record on each shard that represents the current shard's ID.

Use this reader when you want to run an operation on each shard in the database.

title: Parameters

title: Output records

text: Creates a record for each shard with the shard's cluster identifier.

title: Example

text: [code example]

code: ShardsIDReader reader = new ShardsIDReader();

section_id: example


title: StreamReader

summary: Reads Redis stream data.

title: Overview

text: Reads Redis stream data.

title: Parameters

text:
| Name | Type | Default value | Description |
|------|------|---------------|-------------|
| batchSize | integer | 1 | The number of new messages that will cause the functions to run |
| duration | integer | 0 | How many seconds to wait before execution, regardless of batch size |
| failurePolicy | FailurePolicy | FailurePolicy.CONTINUE | How to handle execution failure (CONTINUE/ABORT/RETRY) |
| failureRetryInterval | integer | 5000 | The number of seconds to wait before retrying |
| pattern | string | "\*" (match all keys) | The pattern of keys that store streams |
| startId | string | "0-0" | Start reading from this stream ID |
| trimStream | boolean | true | Whether or not to trim the stream |

title: Output records

text:
Creates a record for each message in the input stream.

Each record is a `HashMap<String, Object>` with the following fields:

| Name | Type | Description |
|------|------|-------------|
| id | string | The message's ID |
| key | string | The stream key name |
| value | HashMap<String, byte[]> | The message's data |

title: Examples

text:
The following example creates a `StreamReader` with default values:

[code example]

To change the parameter values for a `StreamReader`, use their setter methods:

[code example]

code:
StreamReader reader = new StreamReader();
// Get streams for keys that match "weather"
reader.setPattern("weather");
// Run RedisGears functions after every 10 messages
reader.setBatchSize(10);


title: RedisGears JVM commands

summary: Outputs information about existing Java sessions.

title: RG.JDUMPSESSIONS

summary: Executes a Java function.

title: RG.JEXECUTE

title: Overview

text: Use a Redis client like `redis-cli` to send commands to the RedisGears JVM plugin.

title: JVM plugin commands

text: Ignore any commands that start with `RG.PY` while using the JVM plugin. The `RG.PY` commands are for the Python plugin.


title: RG.JDUMPSESSIONS

title: Overview

text:
[code example]

Outputs information about existing Java sessions.

When you run the `RG.JEXECUTE` command, it creates a Java session.

title: Arguments

text:
| Name | Description |
|------|-------------|
| VERBOSE | Output more details about registrations. |
| SESSIONS | Only output information about sessions that appears in the given list. Can only be the last argument. |

title: Returns

text: Returns information about existing Java sessions.

title: Examples

text:
Get information for all sessions:

[code example]

Get more detailed information about a specific session:

[code example]

code: RG.JDUMPSESSIONS [VERBOSE] [SESSIONS s1 s2 ...]

section_id: overview

code:
> redis-cli RG.JDUMPSESSIONS
1)  1) "mainClass"
    2) "gears_experiments.test"
    3) "version"
    4) (integer) 1
    5) "description"
    6) "foo"
    7) "upgradeData"
    8) (nil)
    9) "jar"
   10) "/home/user/work/RedisGears/plugins/jvmplugin/-jars/6876b8b78ccfc2ad764edc7ede590f573bd7260b.jar"
   11) "refCount"
   12) (integer) 2
   13) "linked"
   14) "true"
   15) "ts"
   16) "false"
   17) "registrations"
   18) 1) "0000000000000000000000000000000000000000-1"

code:
> redis-cli RG.JDUMPSESSIONS VERBOSE SESSIONS gears_experiments.test
1)  1) "mainClass"
    2) "gears_experiments.test"
    3) "version"
    4) (integer) 1
    5) "description"
    6) "foo"
    7) "upgradeData"
    8) (nil)
    9) "jar"
   10) "/home/user/work/RedisGears/plugins/jvmplugin/-jars/6876b8b78ccfc2ad764edc7ede590f573bd7260b.jar"
   11) "refCount"
   12) (integer) 2
   13) "linked"
   14) "true"
   15) "ts"
   16) "false"
   17) "registrations"
   18) 1)  1) "id"
           2) "0000000000000000000000000000000000000000-1"
           3) "reader"
           4) "CommandReader"
           5) "desc"
           6) (nil)
           7) "RegistrationData"
           8)  1) "mode"
               2) "async"
               3) "numTriggered"
               4) (integer) 0
               5) "numSuccess"
               6) (integer) 0
               7) "numFailures"
               8) (integer) 0
               9) "numAborted"
              10) (integer) 0
              11) "lastRunDurationMS"
              12) (integer) 0
              13) "totalRunDurationMS"
              14) (integer) 0
              15) "avgRunDurationMS"
              16) "-nan"
              17) "lastError"
              18) (nil)
              19) "args"
              20) 1) "trigger"
                  2) "test"
                  3) "inorder"
                  4) (integer) 0
           9) "ExecutionThreadPool"
          10) "JVMPool"


title: RG.JEXECUTE

summary: Executes a Java function.

title: Overview

text:
[code example]

Executes a Java function.

The code runs immediately if it uses [`GearsBuilder.run()`](). Code that uses [`GearsBuilder.register()`]() will run later, every time certain events occur in the database.

title: Arguments

text:
| Name | Description |
|------|-------------|
| path.to.main.class | The path to the main class in the JAR |
| JAR file | A JAR file that contains the RedisGears code to run or register |
| UPGRADE | Upgrades registered code to a new version |

title: Returns

text:
If the executed code calls [`GearsBuilder.run()`](), it returns the output of the executed code.

For registered code, it returns the string "`OK`" instead.

title: Examples

text:
The executed code in this example [runs]() immediately:

[code example]

This example [registers]() the RedisGears code to run every time certain database events occur:

[code example]

Here's an example of how to upgrade registered code to a new version:

[code example]

code: RG.JEXECUTE <path.to.main.class> [UPGRADE] <JAR file>

code:
$ redis-cli -x RG.JEXECUTE com.domain.packagename.Reviews < /tmp/rgjvmtest-0.0.1-SNAPSHOT.jar
1) 1) "3.6666666666666665"
2) (empty array)

code:
$ redis-cli -x RG.JEXECUTE com.domain.packagename.App < /tmp/rgjvmtest-0.0.1-SNAPSHOT.jar
OK

code:
$ redis-cli -x RG.JEXECUTE com.domain.packagename.App UPGRADE < /tmp/rgjvmtest-0.0.2-SNAPSHOT.jar
OK


title: RedisGears JVM plugin

summary: The RedisGears JVM plugin allows you to run RedisGears functions in the Java virtual machine.

summary: A quick start to learn how to use RedisGears with Java.

title: RedisGears JVM quick start

title: Install RedisGears and the JVM plugin

title: RedisGears JVM commands

title: RedisGears Java classes and functions

title: Java-based RedisGears recipes

title: Overview

text:
With the [RedisGears JVM plugin](https://github.com/RedisGears/JVMPlugin), you can write RedisGears functions in [Java](https://en.wikipedia.org/wiki/Java_(programming_language)) and run them on a [Redis Software]() cluster. It currently supports [JVM](https://en.wikipedia.org/wiki/Java_virtual_machine) version 11.

Similar to the Python plugin, the JVM plugin allows both batch processing and event-driven processing.

Before you can run RedisGears with Java, you will need to [install the RedisGears module and the JVM plugin]() on your Redis Software cluster and [enable them for your database]().

Once you have written your code, compile and package it into a [JAR](https://en.wikipedia.org/wiki/JAR_(file_format)) file and upload it to a node on your Redis Software cluster. Use the `RG.JEXECUTE` command with the `redis-cli` command-line tool to run your code.

title: More info

text:
- [RedisGears JVM quick start]()
- [RedisGears Java classes and functions]()
- [RedisGears recipes]()


title: Install RedisGears and the JVM plugin

title: Overview

text: Before you can use RedisGears with the JVM, you need to install the RedisGears module and JVM plugin on your Redis Software cluster and enable them for a database.

title: Prerequisites

text:
1. Redis Software v6.0.12 or later

1. [Created a Redis Software cluster]()

1. [Added nodes to the cluster]()

1. [Installed RedisGears and the JVM plugin]()

title: Enable RedisGears for a database

text:
1. From the Redis Software admin console's **databases** page, select the **Add** button to create a new database:

1. Confirm that you want to create a new Redis database with the **Next** button.

1. On the **create database** page, give your database a name.

1. For **Redis Modules**, select the **Add** button and choose RedisGears from the **Module** dropdown list.

1. Select **Add Configuration**, enter `Plugin gears_jvm` in the box, then select the **OK** button:

You can configure additional JVM options in this box. For example:

`Plugin gears_jvm JvmOptions `<nobr>`'-Dproperty1=value1`</nobr> <nobr>`-Dproperty2=value2'`</nobr>

1. Select the **Activate** button.

title: Verify the install

text:
Run the `RG.JSTATS` command from a database shard to view statistics and verify that you set up RedisGears and the JVM plugin correctly:

[code example]

section_id: verify-the-install


title: RedisGears JVM quick start

title: Prerequisites

text:
For this quick start, you need:

- A Redis Software cluster with the [RedisGears module and JVM plugin installed]() and [enabled on a database]()
- `redis-cli` with connectivity to a Redis database

title: Tutorial

title: Create a Maven project

text:
1. Create a new [Maven project](https://maven.apache.org/guides/getting-started/index.html#).

1. Add the following sections to the [pom.xml](https://maven.apache.org/guides/introduction/introduction-to-the-pom.html) file:

    [code example]

1. Add example code for either [batch processing](#batch-processing) or [event processing](#event-processing) to your project's `main` function.

title: Build a JAR

text:
Use the Maven command-line tool or an IDE plugin to compile and package your code into a JAR file:

[code example]

title: Upload the JAR

text: Upload your JAR file to a node in the Redis Software cluster. You will need to use the destination filepath when you run your code.

title: Run RedisGears Java code

text:
Use the `RG.JEXECUTE` command to run your code:

[code example]

When you use [`GearsBuilder.run()`](), `RG.JEXECUTE` runs your code immediately.

However, if you use [`GearsBuilder.register()`](), `RG.JEXECUTE` only outputs an `OK` message if it registers successfully. Your registered code will run whenever certain database events occur.

title: Example code

text: You can use these code examples with your own Maven project to try out batch processing or event processing with the RedisGears JVM plugin.

title: Batch processing

text:
If you use the [`GearsBuilder.run()`]() function within your code, then the functions you add to the pipeline will run exactly once when you use `RG.JEXECUTE` with your JAR file.

The following example calculates the average rating of all restaurant reviews stored in your database.

#### Add data to the database

1. Connect to your database with `redis-cli`:

    [code example]

1. Add a few review hashes to the database with the `HSET` command:

    [code example]

#### Example code

[code example]

#### Example output

[code example]

title: Event processing

text:
If you use the [`GearsBuilder.register()`]() function in your code, then the functions you add to the pipeline will run every time a certain database event occurs.

The following example registers a pipeline of functions to automatically update the maximum age every time you add a new person hash to your database.

#### Example code

[code example]

#### Example event processing

After you register your code with the `RG.JEXECUTE` command, add some data to the database and check the value of `age:maximum` to verify that it runs correctly.

1. Connect to your database with `redis-cli`:

    [code example]

1. Add a hash that represents a person with `HSET`:

    [code example]

1. The current value of `age:maximum` should match Alex's age:

    [code example]

1. Add another person with a higher age and then check that `age:maximum` updated automatically:

    [code example]

1. Add a person with a lower age and verify that `age:maximum` did not change:

    [code example]

code:
<repositories>
        <repository>
            <id>snapshots-repo</id>
            <url>https://oss.sonatype.org/content/repositories/snapshots</url>
        </repository>
    </repositories>

    <dependencies>
        <dependency>
            <groupId>com.redislabs</groupId>
            <artifactId>gear_runtime</artifactId>
            <version>0.0.3-SNAPSHOT</version>
        </dependency>
    </dependencies>

section_id: create-a-maven-project

code: $ mvn package

section_id: build-a-jar

code: $ redis-cli -x -h {host} -p {port} RG.JEXECUTE {package.MainClass} < {filepath}/{JAR name}.jar

section_id: run-redisgears-java-code

code: $ redis-cli -h <host> -p <port>

section_id: batch-processing

code:
127.0.0.1:12000> HSET review:1 user "Alex L" message "My new favorite restaurant!" rating 5
    (integer) 3
    127.0.0.1:12000> HSET review:2 user "Anonymous user" message "Kind of overpriced" rating 2
    (integer) 3
    127.0.0.1:12000> HSET review:3 user "Francis J" message "They have a really unique menu." rating 4
    (integer) 3
    127.0.0.1:12000> exit

code:
import java.io.Serializable;
import gears.GearsBuilder;
import gears.readers.KeysReader;
import gears.records.KeysReaderRecord;

public class Reviews implements Serializable
{

	private static final long serialVersionUID = 1L;
	int count; // Total number of reviews
	int ratingsSum; // Sum of all review ratings

    // Reviews constructor
	public Reviews(int count, int ratingsSum) {
		this.count = count;
		this.ratingsSum = ratingsSum;
	}

    public static void main(String args[])
    {
        // Create the reader that will pass data to the pipe
        KeysReader reader = new KeysReader();

        // Create the data pipe builder
        GearsBuilder<KeysReaderRecord> gb = GearsBuilder.CreateGearsBuilder(reader);

		gb.filter(r->{
			// Filter out any keys that are not reviews
			return r.getKey().startsWith("review:");
		}).map(r->{
			// Extract the rating field
			return r.getHashVal().get("rating");
		})
		.accumulate(new Reviews(0, 0), (accumulator, record)-> {
			// Count the reviews and add up all of their ratings
			accumulator.count++;
			accumulator.ratingsSum += Integer.parseInt(record);
			return accumulator;
		}).map(r->{
			// Calculate the average rating
			return Double.valueOf(((double) r.ratingsSum) / r.count);
		});

        // Run the data through the pipeline immediately
        gb.run();
    }
}

code:
$ redis-cli -x -h {host} -p {port} \
    RG.JEXECUTE com.domain.packagename.Reviews < /tmp/rgjvmtest-0.0.1-SNAPSHOT.jar
1) 1) "3.6666666666666665"
2) (empty array)

code:
import gears.GearsBuilder;
import gears.readers.KeysReader;
import gears.records.KeysReaderRecord;

public class App
{
    public static void main(String args[])
    {
        // Create the reader that will pass data to the pipe
        KeysReader reader = new KeysReader();

        // Create the data pipe builder
        GearsBuilder<KeysReaderRecord> gb = GearsBuilder.CreateGearsBuilder(reader);

        // Only process keys that start with "person:"
        gb.filter(r->{
        	return r.getKey().startsWith("person:");
       	});

        // Compare each person's age to the current maximum age
        gb.foreach(r->{
        	String newAgeStr = r.getHashVal().get("age");
        	int newAge = Integer.parseInt(newAgeStr);

        	// Get the current maximum age
        	String maxAgeKey = "age:maximum";
        	String maxAgeStr = (String) GearsBuilder.execute("GET", maxAgeKey);

        	int maxAge = 0; // Initialize to 0
        	if (maxAgeStr != null) {
                // Convert the maximum age to an integer
        		maxAge = Integer.parseInt(maxAgeStr);
        	}

        	// Update the maximum age if a new age is higher
        	if (newAge > maxAge) {
        		GearsBuilder.execute("SET", maxAgeKey, newAgeStr);
        	}
        });

        // Store this pipeline of functions and
        // run them when a new person key is added
        gb.register(ExecutionMode.SYNC);
        // Note: ExecutionMode defaults to ASYNC
        // if you call register() without any arguments
    }
}

section_id: event-processing

code:
127.0.0.1:12000> HSET person:1 name "Alex" age 24
    (integer) 2


title: Java-based RedisGears recipes

summary: Write-behind, write-through, and read-through caching between Redis and other databases (SQL or NoSQL).

title: Content

text: A RedisGears recipe is a collection of RedisGears functions, often designed to perform a specific, complex task.


title: Write-behind, write-through, and read-through caching

title: Overview

text:
The [rghibernate](https://github.com/RedisGears/rghibernate) recipe uses RedisGears functions and the [Hibernate](https://hibernate.org/) framework to implement write-behind, write-through, and read-through caching.

These caching strategies allow applications to simply connect to a Redis cache layer instead of an underlying database. Whenever the application updates data in the cache, Redis also syncs the data in the backend database.

The underlying database could be an SQL database like MySQL, so you will need to provide an XML file that tells rghibernate how to map data between Redis and the other database.

title: Differences from RGSync

text:
[RGSync](https://github.com/RedisGears/rgsync):
- Python-based recipe
- Programmable

[rghibernate](https://github.com/RedisGears/rghibernate):
- Java-based recipe
- Uses the Hibernate framework
- Configurable rather than programmable

title: Set up write-behind and read-through

text:
To set up write-behind caching, first build an rghibernate JAR and register it with RedisGears.

Then, register the following configuration files:

- Connector XML: Tells Redis how to connect to the underlying database.

- Mapping XML: Shows how to map data between the two databases, such as mapping Redis hashes to MySQL tables.

title: Register rghibernate JAR

text:
1. Download the rghibernate JAR from the [download center](https://redis.com/redis-enterprise-software/download-center/modules/).

1. Upload the JAR to a Redis node.

1. Register rghibernate with RedisGears:

    [code example]

title: Configure database connection

role: configuration

text:
1. Create a [connector XML file](#connector-xml) with the configuration to connect Redis to an underlying database.

1. Upload the file to a Redis node.

1. Register the connector configuration:

    [code example]

title: Configure data mapping

text:
1. Create a [mapping XML file](#mapping-xml) that defines how Redis maps data to an underlying database.

1. Upload the file to a Redis node.

1. Register the mapping configuration for write-behind:

    [code example]`

1. Register the same mapping configuration for read-through:

    [code example]`

title: Example configuration

text: Here are some example configuration files for connecting to databases and mapping data.

title: Connector XML

text:
This configuration file contains connection details for an underlying MySQL database:

[code example]

title: Mapping XML

text:
The following XML maps Redis hashes, which represent students, to a MySQL table:

[code example]

title: Commands

role: syntax

text:
Run rghibernate commands with `RG.TRIGGER`:

[code example]

To pass a file to a command like `SYNC.REGISTERCONNECTOR` or `SYNC.REGISTERSOURCE`, use the <nobr>`redis-cli -x`</nobr> option:

[code example]

title: Command list

text:
| Command | Description |
|---------|-------------|
| [SYNC.REGISTERCONNECTOR](#syncregisterconnector) | Register a new connector |
| SYNC.UNREGISTERCONNECTOR | Unregister a connector (cannot have sources attached) |
| [SYNC.REGISTERSOURCE](#syncregistersource) | Extra configuration based on policy |
| SYNC.UNREGISTERSOURCE | Unregister a source |
| SYNC.INFO SOURCES | Dump all sources |
| SYNC.INFO CONNECTORS | Dump all connectors |

title: SYNC.REGISTERCONNECTOR

text:
[code example]

| Name | Description |
|------|-------------|
| connector name | Name to give to your connector |
| batch size | The size of the data sent to the backend in batches |
| timeout | After this timeout, sends data to the backend even if the batch size was not reached |
| retry interval | Retry interval on error |
| connector xml | Hibernate XML definition of the connector |

title: SYNC.REGISTERSOURCE

text:
[code example]

| Name | Description |
|------|-------------|
| source name | Name to give to your source |
| connector name | Connector to send the data to |
| policy | WriteBehind/WriteThrough/ReadThrough:

• On WriteThrough, an extra argument is WriteTimeout

• On ReadThrough, an extra argument is expire (0 for no expire) |
| mapping xml | Hibernate XML definition of the mapping |

code: $ redis-cli -x RG.JEXECUTE com.redislabs.WriteBehind < {filepath}/rghibernate-0.1.1-jar-with-dependencies.jar

section_id: register-rghibernate-jar

code:
> redis-cli -x RG.TRIGGER SYNC.REGISTERCONNECTOR mysql 1000 10 5 < src/test/resources/mysql_hibernate.cfg.xml
    1) "OK"

section_id: configure-database-connection

code:
> redis-cli -x RG.TRIGGER SYNC.REGISTERSOURCE StudentWrite mysql WriteBehind < src/test/resources/Student.hbm.xml
    1) "OK"

section_id: configure-data-mapping

code:
> redis-cli -x RG.TRIGGER SYNC.REGISTERSOURCE StudentRead mysql ReadThrough 0 < src/test/resources/Student.hbm.xml
    1) "OK"

code:
<!DOCTYPE hibernate-configuration PUBLIC
        "-//Hibernate/Hibernate Configuration DTD 3.0//EN"
  "http://www.hibernate.org/dtd/hibernate-configuration-3.0.dtd">

<hibernate-configuration>
  <session-factory>
    <!-- JDBC Database connection settings -->
    <property name="connection.driver_class">org.mariadb.jdbc.Driver</property>
    <property name="connection.url">jdbc:mysql://localhost:3306/test?allowPublicKeyRetrieval=true&useSSL=false</property>
    <property name="connection.username">user</property>
    <property name="connection.password">pass</property>
    <!-- JDBC connection pool settings ... using built-in test pool -->
    <property name="connection.pool_size">1</property>
    <!-- Echo the SQL to stdout -->
    <property name="show_sql">false</property>
    <!-- Set the current session context -->
    <property name="current_session_context_class">thread</property>
    <!-- Drop and re-create the database schema on startup -->
    <property name="hbm2ddl.auto">update</property>
    <!-- dbcp connection pool configuration -->
    <property name="hibernate.dbcp.initialSize">5</property>
    <property name="hibernate.dbcp.maxTotal">20</property>
    <property name="hibernate.dbcp.maxIdle">10</property>
    <property name="hibernate.dbcp.minIdle">5</property>
    <property name="hibernate.dbcp.maxWaitMillis">-1</property>
  </session-factory>
</hibernate-configuration>

section_id: connector-xml

code:
<?xml version="1.0" encoding="UTF-8"?>
<hibernate-mapping xmlns="http://www.hibernate.org/xsd/hibernate-mapping"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.hibernate.org/xsd/hibernate-mapping
    http://www.hibernate.org/xsd/hibernate-mapping/hibernate-mapping-4.0.xsd">

  <class entity-name="Student" table="student">
          <tuplizer entity-mode="dynamic-map" class="org.hibernate.tuple.entity.DynamicMapEntityTuplizer"/>
          <id name="id" type="integer" length="50" column="id"/>
          <property name="firstName" column="first_name" type="string"/>
          <property name="lastName" column="last_name" type="string"/>
          <property name="email" column="email" type="string" not-null="true"/>
          <property name="age" column="age" type="integer"/>
  </class>

</hibernate-mapping>

section_id: mapping-xml

code: redis-cli RG.TRIGGER SYNC.<COMMAND>

section_id: commands

code: redis-cli -x RG.TRIGGER SYNC.{COMMAND} {arg1 arg2 ...} < {file}

code:
$ redis-cli -x RG.TRIGGER SYNC.REGISTERCONNECTOR \
    {connector name} {batch size} {timeout} {retry interval} \
    < {connector xml}

section_id: sync-registerconnector

code:
redis-cli -x RG.TRIGGER SYNC.REGISTERSOURCE \
    {source name} {connector name} {policy} \
    < {mapping xml}

section_id: sync-registersource


title: RedisGears Python plugin

summary: The RedisGears Python plugin allows you to run RedisGears functions with Python.

summary: A quick start to learn how to use RedisGears with Python.

title: RedisGears Python quick start

title: Install RedisGears and the Python plugin

title: Python-based RedisGears recipes

title: Overview

text:
With the RedisGears Python plugin, you can write RedisGears functions in [Python](https://www.python.org/) and run them on a [Redis Software]() cluster.

The Python plugin allows both batch processing and event-driven processing.

Before you can run RedisGears with Python, you will need to [install the RedisGears module and the Python plugin]() on your Redis Software cluster and [enable them for your database]().

Once you have written your code, upload it to a node on your Redis Software cluster. Use the `RG.PYEXECUTE` command with the `redis-cli` command-line tool to run your code.

title: More info

text:
- [RedisGears Python quick start]()
- [RedisGears recipes]()


title: Overview

text: Before you can use RedisGears with Python, you need to install the RedisGears module and Python plugin on your Redis Software cluster and enable them for a database.

title: Prerequisites

text:
1. Redis Software v6.0.12 or later

1. [Created a Redis Software cluster]()

1. [Added nodes to the cluster]()

1. [Installed RedisGears and the Python plugin]()

title: Enable RedisGears for a database

text:
1. From the Redis Software admin console's **databases** page, select the **Add** button to create a new database:

1. Confirm that you want to create a new Redis database with the **Next** button.

1. On the **create database** page, give your database a name.

1. For **Redis Modules**, select the **Add** button and choose RedisGears from the **Module** dropdown list.

1. Select **Add Configuration**, enter <nobr>`Plugin gears_python CreateVenv 1`</nobr> in the box, then select the **OK** button:

Only RedisGears v1.2 and later require this configuration.

1. Select the **Activate** button.

title: Verify the install

text:
Run the `RG.PYSTATS` command to view statistics and verify that you set up RedisGears and the Python plugin correctly:

[code example]

code: redis> RG.PYSTATS


title: RedisGears Python quick start

title: Overview

text:
For this tutorial, you need:

- Either:
    - A Redis Software cluster with the [RedisGears module and Python plugin installed]() and [enabled on a database]()
    - A Redis Open Source database with the RedisGears module
- `redis-cli` with connectivity to a Redis database

title: RedisGears basics

text:
In this quick start guide, we'll see how to use RedisGears to perform batch processing and event processing.

With RedisGears, **batch processing** means processing the data already stored in a Redis database. **Event processing** means processing changes to the Redis key space.

The examples below assume an empty Redis database.

title: Batch processing

text:
Let's start with the simplest example. From the `redis-cli`, run the following command:

[code example]

This command doesn't do much; it simply iterates over the keyspace. Let's add a key and run it again:

[code example]

We've added a single string, and you can see that this gears function processes it, even though it does nothing with the data. Let's actually do something with the data. So first, we'll add a few more strings:

[code example]

We now have three strings in our database. Suppose we want to perform a unique word count on these strings. We can write a RedisGears function to do this just that. So open a file called `wordcount.py`, and add the following code:

[code example]

There are two ways to load files into RedisGears. For production deployments, we recommend using the special [`gears-cli`](https://github.com/gears-project/gears-cli). However, for the purpose of this demonstration, the easiest way is to pass the filename through the `redis-cli` command, like so:

[code example]

The results here show the number of occurrences of each word in all of our strings. So, we've effectively processed the data in our Redis database all at once, in a batch.

title: Event processing

text:
You may have noticed that all of the RedisGears functions above end with a call to `run()`. This indicates that the function should be run immediately on the data in the Redis database. But what if you want to process data as it arrives in Redis? In that case, your functions will end with a call to `register()`, which will store the function and apply it as events occur in Redis.

Let's see how to register a function. First, suppose we're writing hashes to our database that represent users. They take the following form:

[code example]

Each hash has two fields, one containing a name and the other an age. Now, suppose we want to keep a record of the maximum age of all users. We can register a RedisGears function to do this. Open up a file called `maxage.py`, and add the following code:

[code example]

You can see here that we define two methods: `age()` and `compare_and_swap()`. Even if you're not familiar with Python, you should be able to see what the methods do.

Below the method definitions is the RedisGears data flow that we're defining. Notice that at the end we call `register()` to register the function to listen for events.

To load this function into RedisGears, run the following:

[code example]

Now start the `redis-cli`, and create a couple of hashes:

[code example]

To see if the RedisGears function is working, check the value of `age:maximum`:

[code example]

title: Next steps

text: You should now have a basic idea of how to run RedisGears functions for batch and event processing. If you're interested in write-behind caching, see our [write-behind caching]() overview.

code:
redis.cloud:6379> RG.PYEXECUTE "GearsBuilder().run()"
1) (empty array)
2) (empty array)

code:
redis.cloud:6379> SET message "hello world"
OK
redis.cloud:6379> RG.PYEXECUTE "GearsBuilder().run()"
1) 1) "{'event': None, 'key': 'message', 'type': 'string', 'value': 'hello world'}"
2) (empty array)

code:
redis.cloud::6379> SET message:2 "hello galaxy"
OK
redis.cloud:6379> SET message:3 "hello universe"
OK

code:
gb = GearsBuilder()
gb.map(lambda x: x['value'])     # map each key object to its string value
gb.flatmap(lambda x: x.split())  # split each string into a list of words
gb.countby()                     # run a count-unique on these words
gb.run()

code:
$ redis-cli rg.pyexecute "`cat wordcount.py`"
1) 1) "{'key': 'world', 'value': 1}"
   2) "{'key': 'galaxy', 'value': 1}"
   3) "{'key': 'hello', 'value': 3}"
   4) "{'key': 'universe', 'value': 1}"
2) (empty array)

code:
redis.cloud:6379> HSET person:3 name "Summer Smith" age 17
(integer) 2
redis.cloud:6379> HSET person:4 name "James Jameson" age 21
(integer) 2

code:
def age(x):
  ''' Extracts the age from a person's record '''
  return int(x['value']['age'])

def compare_and_swap(x):
  ''' Checks and sets the current maximum '''
  k = 'age:maximum'
  v = execute('GET', k)   # read key's current value
  v = int(v) if v else 0  # initialize to 0 if None
  if x > v:               # if a new maximum found
    execute('SET', k, x)  # set key to new value

# Event handling function registration
gb = GearsBuilder()
gb.map(age) # Extract the 'age' field from each hash
gb.foreach(compare_and_swap) # Compare the max age to the value stored at age:maximum
gb.register(prefix='person:*') # Only process keys matching the pattern 'person:*'

code: $ redis-cli RG.PYEXECUTE "`cat maxage.py`"

code:
redis.cloud:6379> HSET person:5 name "Marek Michalski" age 17
(integer) 2
redis.cloud:6379> HSET person:6 name "Noya Beit" age 21
(integer) 2

code:
redis.cloud:6379> GET age:maximum
"21"


title: Python-based RedisGears recipes

summary: Write-behind and write-through caching between Redis and other databases (SQL or NoSQL).

title: Write-behind caching

title: Content

text: A RedisGears recipe is a collection of RedisGears functions, often designed to perform a specific, complex task.


title: Write-behind caching

title: Overview

text:
Write-behind is a caching strategy in which the cache layer itself connects to the backing database. This means that your applications need only ever connect to your cache layer, and the cache then reads from or updates the backing database as needed. Redis currently supports write-behind caching in [Redis Software]().

Here's how these caching patterns work:

1. Your application uses the cache for reads and writes.
1. The cache syncs any changed data to the backing database asynchronously.

<!-- ### Read-through caching

1. Your application reads from the cache. If there's a cache hit, the data is returned.
1. If there's cache miss, the cache retrieves the data from your backing database (think Oracle, PostgreSQL, etc.). The data is then stored in the cache and returned. -->

title: Install the write-behind recipe

role: setup

text:
The write-behind recipe comes with two types of dependencies:

- Drivers to connect to the backend database
- Python libraries for the RedisGears functions

In most cases all of these can be provisioned to RedisGears before the functions are uploaded.
However, root access for the driver on the host is required in some cases, for example with Oracle drivers.

title: Install Oracle driver (optional)

text:
If you want to do write-behind with an Oracle database:

1. [Download the Oracle driver](https://www.oracle.com/database/technologies/instant-client/linux-x86-64-downloads.html).
1. On each RS node in the cluster, follow the installation instructions on the download page for the Oracle driver.

title: Import requirements

text:
1. Install [Python 3](https://www.python.org/downloads/).
1. To install the [gears-cli](https://pypi.org/project/gears-cli/), run:
    [code example]

1. Download [rgsync offline package](https://redislabs.com/download-center/modules/).
1. Import the requirements:

    [code example]

You can be more efficient and import only the requirements you need, but rgsync is always required and can be combined with one or more of these packages according to your backend database:

- redisgears-requirement-v1-snowflake-sqlalchemy-linux-\<os>-x64.zip
- redisgears-requirement-v1-PyMySQL-linux-\<os>-x64.zip
- redisgears-requirement-v1-cx-Oracle-linux-\<os>-x64.zip
- redisgears-requirement-v1-cassandra-driver-linux-\<os>-x64.zip

This list can be different or more extensive in newer versions.
The `module.json` file in the module package lists the dependencies for the module.

1. From the CLI of a node or of a client that is connected to the database, check that the requirements were imported successfully with:

    [code example]

    To connect to the database from a client, run:

    [code example]

    This command returns a list of all available requirements.

title: Register the functions

text:
The following is a RedisGears recipe that shows how to use the write-behind pattern to map data from Redis hashes to MySQL tables.

The recipe maps all Redis hashes with the prefix <nobr>`person:<id>`</nobr> to the MySQL table `persons`, with `<id>` being the primary key and mapped to the `person_id` column.
Similarly, it maps all hashes with the prefix <nobr>`car:<id>`</nobr> to the `cars` table.

[code example]

Go to the [rgsync website](https://pypi.org/project/rgsync/) to get the replication options and the configuration options for the database and mapping.

1. Create a Python file with the configuration mapping according to your specific needs.
1. Run `gears-cli` with your custom file:

    [code example]

title: Write-behind caching with RedisGears

text:
[RGSync](https://github.com/RedisGears/rgsync) is a RedisGears recipe that uses the write-behind pattern to synchronize Redis data structures to a backing data store (Oracle, MySQL, Cassandra, and Snowflake are currently supported).

Here's how you can create a write-behind function using the `RGSync` recipe against MySQL. This function will synchronize a set of fields in a Redis hash to the columns in a MySQL row. The key name for the Redis hash must have the form `{name}:{id}`, where `id` is the primary key for the row we want to keep in sync with. So, for example, suppose our hashes look like this:

[code example]

The data in this hash would correspond to a row with ID of 5. But we need to define the mapping from our hash field to our columns. In our function, we do this using a Python dictionary:

[code example]

This is saying that the `last_name` field in our hash should be synchronized with the 'last' field in our row. So, now we're ready to write our RedisGears function.

First, load the necessary libraries and connect to MySQL:

[code example]

Next, create a connector object, specifying the table name and the name of the column storing the primary key:

[code example]

After that, define your field-column mappings:

[code example]

Finally, use these initialized objects to create an `RGWRiteBehind` object:

[code example]

Notice here that you can version this function registration with the `version` argument, as newer versions with the same `name` will replace the old one.

If you've placed all of this code into a file (`example.py`), then you can load it into your Redis deployment using the [`gears-cli`](https://github.com/RedisGears/gears-cli) tool:

[code example]

title: Secret management

text:
You may not want to store database credentials in your RedisGears functions. To avoid this, you can pass these credentials as module parameters and then reference them from your functions.

The code below shows how to reference these credentials when creating a backing database connection.

[code example]

Notice that for each credential, we define a Python function that returns the specified module parameter. We then provide each function reference when we instantiate `MySqlConnection`.

This code references three parameters: `MySqlUser`, `MySqlPassword`, and `MySqlDB`. In Redis Software, you can set these parameters using the `rladmin` tool. The command to set these parameters takes the following form:

[code example]

To set the `MySqlPassword` parameter to "Password123!" on a database named "user-api", you would run the this `rladmin` command:

[code example]

Once a connection is successfully established, RedisGears will not attempt to reconnect until a disconnect occurs.
This means that updates to the parameters that store the secrets will not take effect immediately,
but they will be used for all subsequent connection attempts.

code: pip install gears-cli

section_id: import-requirements

code:
# gears-cli import-requirements \
        --host HOST [ --port PORT ] --password PASSWORD \
        --requirements-path rgsync-99.99.99.linux-bionic-x64.zip

code: redis-cli RG.PYDUMPREQS

code: redis-cli -h <FQDN_of_node> -p <host> [-a <password>]

code:
from rgsync import RGWriteBehind
from rgsync.Connectors import MySqlConnector, MySqlConnection

'''
Create MySQL connection object
'''
connection = MySqlConnection('demouser', 'Password123!', 'localhost:3306/test')

'''
Create MySQL persons connector
persons - MySQL table to put the data
person_id - primary key
'''
personConnector = MySqlConnector(connection, 'persons', 'person_id')

personsMappings = {
	'first_name':'first',
	'last_name':'last',
	'age':'age'
}

RGWriteBehind(GB, keysPrefix='person', mappings=personsMappings, connector=personConnector, name='PersonsWriteBehind', version='99.99.99')

'''
Create MySQL car connector
cars - MySQL table to put the data
car_id - primary key
'''
carConnector = MySqlConnector(connection, 'cars', 'car_id')

carsMappings = {
	'id':'id',
	'color':'color'
}

RGWriteBehind(GB, keysPrefix='cars', mappings=carsMappings, connector=carConnector, name='CarsWriteBehind', version='99.99.99')

section_id: register-the-functions

code: gears-cli run --host <host> --port <post> --password <password> <yourfile>.py

code:
redis.cloud:6379> HSET person:5 first_name Marek last_name Michalski age 27
(integer) 2

section_id: write-behind-caching-with-redisgears

code:
from rgsync import RGWriteBehind
from rgsync.Connectors import MySqlConnector, MySqlConnection

connection = MySqlConnection('myapplication', 'Password123!', 'localhost:3306/test')

code: personConnector = MySqlConnector(mySqlConnection, 'persons', 'id')

code:
RGWriteBehind(GB,
              name='PersonsWriteBehind',
              version='1',
              connector=personsConnector,
              keysPrefix='person',
              mappings=personsMappings)

code:
gears-cli --host <host>
          --port <post>
          --password <password>
          example.py
          REQUIREMENTS rgsync

code:
def User():
	return configGet('MySqlUser')
def Password():
	return configGet('MySqlPassword')
def DB():
	return configGet('MySqlDB')

connection = MySqlConnection(User, Password, DB)

section_id: secret-management

code: rladmin tune db [DB-NAME] module_name rg module_config_params "[PARAM-NAME] [PARAM-VALUE]"

code: rladmin tune db user-api module_name rg module_config_params "MySqlPassword Password123!"


title: Register events

summary: Register RedisGears functions to run when certain events occur in a Redis database.

title: Overview

text: You can register RedisGears functions to run when certain events occur in a Redis database.

title: Register on events

text:
To register RedisGears functions to run on an event, your code needs to:

1. Pass `KeysReader` to a `GearsBuilder` object.

1. Call the `GearsBuilder.register()` function.

1. Pass the `eventTypes` parameter to either:

    - The `register` function for Python.

    - The `KeysReader` object for Java.

For more information and examples of event registration, see:

- Java references:

    - [`KeysReader`]()

    - [`GearsBuilder.register()`]()

title: Event types

text: For the list of event types you can register on, see the [Redis keyspace notification documentation](#events-generated-by-different-commands).

title: Active-Active event types

text:
In addition to standard Redis [events](#events-generated-by-different-commands), [Redis Software Active-Active databases]() also support the registration of RedisGears functions for the following event types:

- `change`: This event occurs when a key changes on another replica of the Active-Active database.


title: Redis Open Source and Redis Software

title: Redis Open Source quick start

title: Install and upgrade modules

title: Module lifecycle

summary: Describes the Redis Software features supported by each Redis Open Source feature.

title: Redis Software and Redis Open Source feature compatibility

title: Search and query

summary: Redis Open Source adds support for JSON to Redis databases.

title: JSON

title: Time series

summary: Probabilistic data structures

title: Probabilistic data structures

summary: RedisGears supports batch and event-driven processing for Redis data.

title: Gears

summary: Features (modules) that have been deprecated or replaced with a new version.

title: Deprecated Redis Open Source features and modules

summary: Redis Open Source and modules release notes

title: Content

text: [Redis Software]() and [Redis Cloud]() support all [Redis Open Source]() features. Some of the capabilities Redis Open Source provides include querying, indexing, and full-text search and support for JSON, time series, and probabilistic data structures.


title: Install a module on a cluster

title: Overview

text:
[Redis Software]() comes packaged with several modules that provide additional Redis capabilities such as [search and query](), [JSON](), [time series](), and [probabilistic data structures](). As of version 8.0, Redis Software includes multiple feature sets, compatible with different Redis database versions. You can view the installed modules, their versions, and their minimum compatible Redis database versions from **Cluster > Modules** in the Cluster Manager UI.

To use other modules or upgrade an existing module to a more recent version, you need to install the new module package on your cluster.

Some module versions are not supported or recommended for use with Redis Software.

title: Module package requirements

text:
The module must be packaged as a `.zip` file containing:

- **module.json**: A metadata file with module information including:
  - `module_name`: The actual module name
  - `version`: Numeric version
  - `semantic_version`: Semantic version string (for example, "1.0.0")
  - `min_redis_version`: Minimum compatible Redis version
  - `commands`: List of commands the module provides
  - `capabilities`: List of module capabilities

- **Module binary**: The compiled `.so` file for the target platform

title: Get packaged modules

text:
To install or upgrade a module on a [Redis Software]() cluster, you need a module package.

- For versions of official Redis modules that are not available from the [Redis download center](https://redis.io/downloads/), [contact support](https://redis.io/support/).

- For custom-packaged modules, download a [custom-packaged module](https://redislabs.com/community/redis-modules-hub/) from the developer.

- User-defined modules are downloaded automatically if you [add them during bootstrapping](#bootstrap-user-defined-module).

title: Add user-defined modules during bootstrapping (Redis Software v8.0.6 and later) {#bootstrap-user-defined-module}

text:
As of Redis Software version 8.0.6, you can include `user_defined_modules` in REST API requests to [initiate boostrap operations]() such as `create_cluster`, `join_cluster`, or `recover_cluster`. Each node in the cluster independently downloads and installs the specified modules during its bootstrap process.

`user_defined_modules` has the following JSON schema:

[code example]

title: Best practices

text:
- Use `https` instead of `http` for secure module downloads.

- Include version numbers in module URLs.

- Use the same `user_defined_modules` configuration for all nodes in a cluster.

- If using authenticated downloads, ensure credentials are properly secured.

- Ensure modules are compatible with the Redis database version running on your cluster.

- Verify modules work correctly before deploying to production environments.

title: Example requests

text:
**Create cluster:**

The following example creates a cluster with multiple modules:

[code example]

**Join cluster:**

The following example joins a node to a cluster with multiple modules:

[code example]

**Recover cluster:**

The following example recovers a cluster with multiple modules:

[code example]

title: Troubleshooting

role: errors

text:
#### Error handling

Download failures do not fail the bootstrap process. If a module fails to download or install, a warning is logged and the bootstrap process continues with the remaining modules.

Warnings are recorded in the bootstrap status with:
- `warning_type`: `"module_download_failed"`
- `message`: Error description
- `details`: `{"module_name": "<name>"}`

#### Module download failed

Check the bootstrap logs for detailed error messages:

[code example]

Common causes:
- Invalid URL
- Network connectivity issues
- Authentication failures
- Module package format issues

#### Module compatibility errors

After processing user-defined modules, the system validates that all custom modules are compatible with existing databases in the cluster. This validation:

1. Checks which custom modules are used by existing databases.

1. Verifies that compatible module versions are available on the node.

1. Fails the bootstrap process if incompatible modules are detected.

If the bootstrap process fails with an `incompatible_modules` error:

1. Verify the module version is compatible with existing databases.

1. Ensure the module binary exists and is accessible.

#### Missing module.json

If you see `"module.json missing"` errors:

1. Verify the zip file contains a valid `module.json` at the root level.

1. Verify the JSON is properly formatted.

title: Add a user-defined module to a cluster (Redis Software v8.0.x and later) {#add-user-defined-module-to-cluster}

text:
To add a custom module to a cluster running Redis Software version 8.0.x or later, use the following REST API requests:

1. [Upload the custom module configuration](). Replace the values in the following example with your own.

    [code example]

1. For each node in the cluster, [upload the custom module artifact]():

    [code example]

    The *module* parameter specifies the full path of the module artifact and must be submitted as form-data. In addition, the module artifact must be available and accessible to the server processing the request.

title: Add a module to a cluster (Redis Software v7.22.x and earlier) {#add-a-module-to-a-cluster}

text:
Use one of the following methods to add a module to a cluster running Redis Software version 7.22.x or earlier:

**Cluster Manager UI:**

To add a module to the cluster using the Cluster Manager UI:

1. Go to **Cluster > Modules**.

1. Select **Upload module**.

1. Use the file browser to add the packaged module.

**REST API:**

To add a module to the cluster using the REST API:

1. Copy the module package to a node in the cluster.

1. Add the module to the cluster with a [`POST` request to the `/v2/modules`]() endpoint:

    [code example]

    Here, the *module* parameter specifies the full path of the module package and must be submitted as form-data. In addition, the package must be available and accessible to the server processing the request.

1. If the module installation succeeds, the `POST` request returns a [JSON object]() that represents the new module. If it fails, it may return a JSON object with an `error_code` and `description` with more details.

For RedisGears, follow these [installation instructions]() instead.

We recommend consulting [Redis support](https://redis.io/support/) before you upgrade a module on the cluster, especially if the cluster is used in production.

title: Next steps

text:
- Create a database and [enable the new module]().
- [Upgrade a module]() to the new version.

string (required)

location_type: http | https (required)

url: string (required)

username: string (optional)

password: string (optional)

code:
POST /v1/bootstrap/create_cluster
{
  "action": "create_cluster",
  "credentials": {
    "username": "admin@example.com",
    "password": "your-secure-password"
  },
  "cluster": {
    "name": "my-cluster.example.com"
  },
  "user_defined_modules": [
    {
      "name": "ModuleA",
      "location": {
        "location_type": "https",
        "url": "https://private-repo.example.com/enterprise-module-2.0.0.zip",
        "credentials": {
          "username": "download-user",
          "password": "download-password"
        }
      }
    },
    {
      "name": "ModuleB",
      "location": {
        "location_type": "https",
        "url": "https://modules.example.com/module-b-2.5.0.zip"
      }
    },
    {
      "name": "ModuleC",
      "location": {
        "location_type": "http",
        "url": "http://internal-server.local/module-c-1.2.0.zip"
      }
    }
  ]
}

section_id: example-requests

code:
POST /v1/bootstrap/join_cluster
{
  "action": "join_cluster",
  "credentials": {
    "username": "admin@example.com",
    "password": "your-secure-password"
  },
  "cluster": {
    "name": "my-cluster.example.com",
    "nodes": ["192.168.1.10", "192.168.1.11"]
  },
  "user_defined_modules": [
    {
      "name": "ModuleA",
      "location": {
        "location_type": "https",
        "url": "https://private-repo.example.com/enterprise-module-2.0.0.zip",
        "credentials": {
          "username": "download-user",
          "password": "download-password"
        }
      }
    },
    {
      "name": "ModuleB",
      "location": {
        "location_type": "https",
        "url": "https://modules.example.com/module-b-2.5.0.zip"
      }
    },
    {
      "name": "ModuleC",
      "location": {
        "location_type": "http",
        "url": "http://internal-server.local/module-c-1.2.0.zip"
      }
    }
  ]
}

code:
POST /v1/bootstrap/recover_cluster
{
  "action": "recover_cluster",
  "recovery_filename": "/path/to/backup.rdb",
  "credentials": {
    "username": "admin@example.com",
    "password": "your-secure-password"
  },
  "user_defined_modules": [
    {
      "name": "ModuleA",
      "location": {
        "location_type": "https",
        "url": "https://private-repo.example.com/enterprise-module-2.0.0.zip",
        "credentials": {
          "username": "download-user",
          "password": "download-password"
        }
      }
    },
    {
      "name": "ModuleB",
      "location": {
        "location_type": "https",
        "url": "https://modules.example.com/module-b-2.5.0.zip"
      }
    },
    {
      "name": "ModuleC",
      "location": {
        "location_type": "http",
        "url": "http://internal-server.local/module-c-1.2.0.zip"
      }
    }
  ]
}

code: Failed to download and install custom module '<name>': <error details>

section_id: troubleshooting

code:
POST https://<host>:<port>/v2/modules/user-defined
    {
      "module_name": "TestModule",
      "version": 1,
      "semantic_version": "0.0.1",
      "display_name": "test module",
      "commands": [
        {
          "command_arity": -1,
          "command_name": "module.command",
          "first_key": 1,
          "flags": ["write"],
          "last_key": 1,
          "step": 1
        }
      ],
      "command_line_args": "",
      "capabilities": ["list", "of", "capabilities"],
      "min_redis_version": "2.1"
    }

code:
POST https://<host>:<port>/v2/local/modules/user-defined/artifacts
    "module=@/tmp/custom-module.zip"

code:
POST https://<host>:<port>/v2/modules
    "module=@/tmp/redisearch.Linux-ubuntu16.04-x86_64.2.2.6.zip"


title: Enable a module for a database

title: Overview

text: Modules add additional functionality to Redis databases for specific use cases. You can enable modules when you create a database.

title: Automatically enabled capabilities in Redis 8

text: Databases created with or upgraded to Redis version 8 or later automatically enable the capabilities (modules) bundled with Redis Software as follows:

title: Create a database with a module

text: You can only add modules to a database when you first create it. You cannot add modules to an existing database.

title: Prerequisites

text:
- [Installed the module on the cluster]()
- [Upgraded the module]() to the latest version

title: Create a database and enable capabilities

text:
In the Redis Software Cluster Manager UI, follow these steps to add modules to a database:

1. From the **Databases** screen, select **Quick database** or **Create database**.

1. In the **Capabilities** section, select one or more capabilities:

You cannot use RediSearch 1.x and RediSearch 2.x in the same database.

1. To use custom configuration for capabilities:

    1. Select **Parameters**.

    1. Enter the [configuration options](#module-configuration-options).

    1. Select **Done**.

1. Configure additional database settings.

Depending on the [features supported by an enabled module](), certain database configuration fields may not be available.

1. Select **Create**.

title: Module configuration options

text:
- [RediSearch configuration options]()

- [RedisTimeSeries configuration options]()

- [RedisBloom configuration options]()


title: Install and upgrade modules

title: Install a module on a cluster

title: Enable a module for a database

title: Upgrade modules

title: Overview

text:
Several modules that provide additional Redis capabilities, such as search and query, JSON, time series, and probabilistic data structures, come packaged with [Redis Software](). As of version 8.0, Redis Software includes multiple feature sets, compatible with different Redis database versions.

However, if you want to use additional modules or upgrade a module to a more recent version, you need to:

1. [Install a module package]() on the cluster.
1. [Enable a module]() for a new database or [upgrade a module]() in an existing database.


title: Upgrade modules

title: Overview

text:
Upgrade a module in Redis Software to get the latest features and fixes.

- If you upgrade a single-node cluster, it does not load the new modules that are bundled with the new cluster version.

- Before you upgrade a database with the RediSearch module enabled to Redis 5.0, you must upgrade the RediSearch module to version 1.4.2 or later.

title: Prerequisites

text: Before you upgrade a module enabled in a database, [install the new version of the module on the cluster]().

title: Upgrade a module for a database

text:
After you install an updated module on the cluster, you can upgrade databases to use the new module version.

After you upgrade the module for a database, the database shards restart. This causes a short interruption in the availability of this database across the cluster.

To upgrade a module enabled for a database:

1. Connect to the terminal of a node in the cluster.
1. Run `rladmin status` to list the databases on the node.
1. Copy the name of the database that uses the module that you want to upgrade.

1. Find the exact module name and version:

    1. Extract the module archive (zip) file.
    1. Open the JSON file.
    1. Find the module name and version number in the file.

    Here's an example of the JSON file for the RediSearch module:

1. To see the versions of the modules on the cluster, run either:

    - `rladmin status modules` - Shows the latest modules available on the cluster and the modules used by databases.
    - `rladmin status modules all` - Shows all of the modules available on the cluster and the modules used by databases.

1. To upgrade a database to the latest version of Redis and its modules to the latest version without changing the module arguments:

    - For clusters with Redis Software versions 7.8.2 or later, run:

        [code example]

    - For clusters with versions earlier than 7.8.2, include the `latest_with_modules` option:

        [code example]

The upgrade process does not validate the module upgrade arguments, and incorrect arguments can cause unexpected downtime. Test module upgrade commands in a test environment before you upgrade modules in production.

    - Use `keep_redis_version` to upgrade the modules without upgrading the database to the latest Redis version.

        `keep_redis_version` is deprecated as of Redis Software version 7.8.2. To upgrade modules without upgrading the Redis database version, set `redis_version` to the current Redis database version instead.

    - To specify the modules to upgrade, add the following for each module:

        [code example]

        For the module arguments, use one of the following:

        - `module_args "<module_arguments>"` to replace the existing module arguments.

        - `module_args ""` without arguments to remove the existing module arguments.

        - `module_args keep_args` to use the existing module arguments.

title: Examples for Redis Software v7.8.2 and later

text:
The following module upgrade examples are supported for Redis Software versions 7.8.2 and later:

- Keep the current Redis database version, which is 7.2 in this example, and upgrade to the latest version of the enabled modules:

    [code example]

- Upgrade the database to use the latest version of Redis and the latest versions of the enabled modules:

    [code example]

title: Deprecated examples

text:
As of Redis Software version 7.8.2, the following module upgrade examples are deprecated but still supported.

- Keep the current version of Redis and upgrade to the latest version of the enabled modules:

    [code example]

- Upgrade the database to the latest Redis version and upgrade RediSearch to 1.6.7 with the specified arguments:

    [code example]

- Upgrade the database to the latest Redis version and upgrade RedisBloom to version 2.2.1 without arguments:

    [code example]

- Upgrade RedisJSON to 1.0.4 with the existing arguments and RedisBloom to version 2.2.1 without arguments:

    [code example]

- Upgrade the database to use the latest version of Redis and use the latest version of the enabled modules:

    [code example]

section_id: upgrade-a-module-for-a-database

code: rladmin upgrade db < database-name | database-ID > latest_with_modules

code: and module module_name <module_name> version <new_module_version_number> module_args "<module arguments>"

code: rladmin upgrade db shopping-cart redis_version 7.2

code: rladmin upgrade db shopping-cart

code: rladmin upgrade db shopping-cart keep_redis_version latest_with_modules

section_id: deprecated-examples

code: rladmin upgrade db shopping-cart and module db_name shopping-cart module_name ft version 10607 module_args "PARTITIONS AUTO"

code: rladmin upgrade db db:3 and module db_name shopping-cart module_name bf version 20201 module_args ""

code: rladmin upgrade module db_name MyDB module_name ReJSON version 10004 module_args keep_args and module db_name MyDB module_name bf version 20201 module_args ""

code: rladmin upgrade db shopping-cart latest_with_modules


title: Store JSON in Active-Active databases

summary: JSON support and conflict resolution rules for Active-Active databases.

title: Overview

text:
RedisJSON v2.2 adds support for JSON in [Active-Active Redis Software databases]().

The design is based on [A Conflict-Free Replicated JSON Datatype](https://arxiv.org/abs/1608.03960) by Kleppmann and Beresford, but the implementation includes some changes. Several [conflict resolution rule](#conflict-resolution-rules) examples were adapted from this paper as well.

title: Create an Active-Active JSON database

text:
To use JSON in an Active-Active database, you must enable JSON during database creation.

Active-Active Redis Cloud databases add JSON by default. See [Create an Active-Active database]() in the Redis Cloud documentation for details.

In Redis Software, Active-Active databases created with or upgraded to Redis version 8 or later automatically enable JSON. For earlier Redis versions, you can enable JSON during database creation:

1. See [Create an Active-Active geo-replicated database]() in the Redis Software documentation for prerequisites and detailed steps.

1. In the **Capabilities** section of the **Create Active-Active database** screen, select **JSON**:

When you select **JSON**, **Search and Query** is also selected by default to allow you to index and query JSON documents. If you do not want to use these additional features, you can clear the **Search and Query** check box.

1. Configure additional database settings.

1. Select **Create**.


title: JSON commands

summary: Lists JSON commands and provides links to the command reference pages.

title: Content

text:
The following table lists JSON commands. See the command links for more information about each command's syntax, arguments, and examples.

| Command | Redis
Software | Redis
Cloud | Description |
|:--------|:----------------------|:-----------------|:------|
| [JSON.ARRAPPEND]() | ✅ Supported

 | <nobr>✅ Flexible & Annual</nobr>
✅ Free & Fixed</nobr> | Appends an element to a JSON array. |
| [JSON.ARRINDEX]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns the index of a value's first occurrence in a JSON array. |
| [JSON.ARRINSERT]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Inserts JSON values into a JSON array before the given index. |
| [JSON.ARRLEN]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns the length of a JSON array. |
| [JSON.ARRPOP]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Removes and returns an element located at the index in the JSON array. |
| [JSON.ARRTRIM]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Trims a JSON array so that it contains only the specified inclusive range of elements. |
| [JSON.CLEAR]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Clears container values (arrays/objects) and sets numeric values to 0. |
| [JSON.DEBUG]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Debugging container command. |
| [JSON.DEBUG HELP]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns helpful information about the [JSON.DEBUG]() command. |
| [JSON.DEBUG MEMORY]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Reports a JSON element's memory usage in bytes. |
| [JSON.DEL]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Removes a JSON element. |
| [JSON.FORGET]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Removes a JSON element, the same as [JSON.DEL](). |
| [JSON.GET]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns the value of an element in JSON-serialized form. |
| [JSON.MGET]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns the values of multiple elements. |
| [JSON.MERGE]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Merge a given JSON value into matching paths. |
| [JSON.NUMINCRBY]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Increments the number stored at path by the specified number. |
| [JSON.NUMMULTBY]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Multiplies the number stored at path by the specified number. (deprecated as of RedisJSON v2.0) |
| [JSON.OBJKEYS]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns the keys contained in the specified JSON object. |
| [JSON.OBJLEN]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns the number of keys in the specified JSON object. |
| [JSON.RESP]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns a JSON element in [Redis Serialization Protocol (RESP)]() format. |
| [JSON.SET]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Sets the value of a JSON element. |
| [JSON.STRAPPEND]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Appends the given string to the specified key's existing strings. |
| [JSON.STRLEN]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns the length of a string. |
| [JSON.TOGGLE]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | If the boolean is `true`, changes it to `false`. If the boolean is `false`, changes it to `true`. |
| [JSON.TYPE]() | ✅ Supported

 | ✅ Flexible & Annual
✅ Free & Fixed</nobr> | Returns a JSON element's type. |


title: JSON

title: JSON commands

title: Store JSON in Active-Active databases

title: Overview

text:
Redis Open Source adds support for the [JSON data structure](http://www.json.org/) to Redis databases.

Applications developed with the [source available version of RedisJSON](https://github.com/RedisJSON/RedisJSON) are 100%
compatible with Redis Software databases with JSON enabled.

title: JSON paths

text:
[Paths]() let you traverse the structure of a JSON document, starting from the root, and interact only with the data you want. You can also use paths to perform operations on specific JSON elements.

Since there is no standard for JSON path syntax, Redis Open Source implements its own.

title: JSONPath syntax

text:
RedisJSON v2.0 and later support the [JSONPath syntax](), which resembles [Goessner's design](https://goessner.net/articles/JsonPath/):

  - Paths start with a dollar sign (`$`), which represents the root of the JSON document.

  - See the [JSONPath syntax table]() to learn how to access various elements within a JSON document.

The following path refers to `headphones`, which is a child of `inventory` under the root:

`$.inventory.headphones`

See [JSONPath examples]() for examples with more complex syntax.

title: Legacy path syntax

text:
The [legacy path syntax](#legacy-path-syntax) refers to the path implementation in RedisJSON v1. RedisJSON v2 still supports this legacy path syntax in addition to JSONPath syntax.

The legacy path syntax works as follows:

  - A period character represents the root.

  - For paths to the root's children, it is optional to prefix the path with a period.

  - Supports both dot notation and bracket notation for JSON object key access.

The following paths refer to `headphones`, which is a child of `inventory` under the root:

`.inventory.headphones`

`inventory["headphones"]`

`['inventory']["headphones"]`

title: Key name rules

text:
You can only use the legacy path syntax to access JSON keys if they follow these name syntax rules:

- Key names must begin with a letter, a dollar sign (`$`), or an underscore (`_`).
- Key names can contain letters, digits, dollar signs, and underscores.
- Key names are case-sensitive.

title: Index and search JSON documents

text:
You can index, search, and query stored JSON documents.

For more information about how to search and query JSON documents, see the [quick start]().

title: JSON in Active-Active databases

text:
RedisJSON v2.2 and later support the JSON data structure as a conflict-free replicated data type [(CRDT)](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type) in [Active-Active Redis Software databases]().

For details about how Redis Software resolves JSON operation conflicts that can arise when replicas attempt to sync, see the JSON [conflict resolution rules]().

title: More info

text:
- [JSON quick start](#use-redisjson)
- [JSON commands]()
- [RedisJSON source](https://github.com/RedisJSON/RedisJSON)


title: Module lifecycle

title: Overview

text:
Redis Software follows the [Redis Software lifecycle]().  (For complete details, see the Redis Software [subscription agreement](https://redis.com/software-subscription-agreement).)

The modules included in Redis Stack also follow a release lifecycle and schedule. Here, you'll find the "end-of-life" dates for each module and release.

title: Module release numbering

text:
Redis modules use a three-place numbering scheme to identify released versions.

The format is "Major1.Major2.Minor".

- Major sections of the version number represent fundamental changes to functionality and feature capabilities. The _Major1_ and _Major2_ part of the version number are incremented according to the size and scale of the changes in each release.

- The _Minor_ section of the version number represents quality improvements and fixes to existing capabilities.  The minor release number is increased when release quality improves.

title: Module end-of-life schedule {#modules-endoflife-schedule}

text:
End-of-Life for a given Major version is 18 months after the formal release of
that version or 12 months after the release of the next subsequent (following) version, whichever comes last.

title: RediSearch

title: RedisJSON

title: RedisGraph

title: RedisTimeSeries

title: RedisBloom

title: RedisGears


title: Redis Open Source and modules release notes

summary: Redis Open Source release notes for version 7.4 and later

title: Redis Open Source release notes

summary: Redis Stack release notes for version 7.2 and later

title: Redis Stack release notes

summary: RediSearch release notes for version 2.10 and earlier

title: RediSearch release notes

summary: RedisJSON release notes for version 2.8 and earlier

title: RedisJSON release notes

summary: RedisBloom release notes for version 2.8 and earlier

title: RedisBloom release notes

summary: RedisTimeSeries release notes for version 1.12 and earlier

title: RedisTimeSeries release notes

summary: RedisGraph release notes for version 2.10 and earlier

title: RedisGraph release notes

summary: RedisGears release notes for version 1.2 and earlier

title: RedisGears release notes

title: Content

text: See the following release notes for new features, enhancements, and other changes to Redis Open Source capabilities.


title: RedisBloom release notes

summary: RedisBloom 2.8 introduces improvements and bug fixes.

title: RedisBloom 2.8 release notes

summary: RESP3 support. Bug fixes.

title: RedisBloom 2.6 release notes

summary: Added t-digest - a probabilistic data structure for estimating quantiles based on a data stream or a large dataset of floating-point values.

title: RedisBloom 2.4 release notes

summary: BF.INFO returns bloom filter details. CF.INFO returns cuckoo filter details. Scalable bloom and cuckoo filters. Configurable bucket size for cuckoo filters. CMS.INCRBY returns count.

title: RedisBloom 2.2 release notes

summary: Added more probabilistic data structures, including top-K and count-min sketch.

title: RedisBloom 2.0 release notes

summary: Updated version.

title: RedisBloom 1.1 release notes

summary: First GA release of RedisBloom.

title: RedisBloom 1.0 release notes

title: Content


title: RedisBloom 1.0 release notes

summary: First GA release of RedisBloom.

title: Requirements

text:
RedisBloom v1.0.3 requires:

- Minimum Redis compatibility version (database): 4.0
- Minimum Redis Enterprise Software version (cluster): 5.0

title: v1.0.3 (December 2017)

text:
This contains a single fix, issue #[19](https://github.com/RedisBloom/RedisBloom/issues/19).

From this version onwards, `EXISTS`/`MEXISTS` returns 0 if the (Redis) key does not exist in the database.  Earlier versions returned an error.

title: v1.0.2 (November 2017)

text: This fixes a build issue (fixed s3 config in circle yaml).

title: v1.0.0 (September 2017)

text: This is the first GA release of ReBloom.


title: RedisBloom 1.1 release notes

summary: Updated version.

title: Requirements

text:
RedisBloom v1.1.1 requires:

- Minimum Redis compatibility version (database): 4.0.0
- Minimum Redis Enterprise Software version (cluster): 5.0.0

title: RedisBloom 1.1.1 (21 February 2019)

text: Update version to 1.1.1

title: RedisBloom 1.1.0 (12 February 2019)

text: Release of 1.1.0


title: RedisBloom 2.0 release notes

title: Requirements

text:
RedisBloom v2.0.3 requires:

- Minimum Redis compatibility version (database): 4.0.0
- Minimum Redis Enterprise Software version (cluster): 5.0.0

title: v2.0.3 (July 2019)

text:
- Performance improvements:
    - #[95](https://github.com/RedisBloom/RedisBloom/issues/95) Top-K - Reduce checks on heap, now only checks if item count is larger than minimum in heap.
    - #[95](https://github.com/RedisBloom/RedisBloom/issues/95) Top-K - The power of decay was calculated every time. Changed to use a lookup table.
- Major bug fix:
    - #[88](https://github.com/RedisBloom/RedisBloom/issues/88) Replication available for newly added Top-K and Count-min sketch
- Minor bug fixes:
    - #[89](https://github.com/RedisBloom/RedisBloom/issues/89) Module update broke rdb files
    - #[98](https://github.com/RedisBloom/RedisBloom/issues/98) Compilation for macOS

title: v2.0.0 (June 2019)

text:
We are proud to announce that we doubled the number of probabilistic data structures that are generally available in RedisBloom.  Full documentation is available on [redisbloom.io](https://redisbloom.io)

- #[70](https://github.com/RedisBloom/RedisBloom/issues/70) Top-K
    - [Commands]()
    - [Algorithm](https://www.usenix.org/conference/atc18/presentation/gong)

- #[65](https://github.com/RedisBloom/RedisBloom/issues/65) Count-min Sketch
    - [Commands]()
    - [Algorithm](https://en.wikipedia.org/wiki/Count%E2%80%93min_sketch)


title: RedisBloom 2.2 release notes

title: Requirements

text:
RedisBloom v2.2.18 requires:

- Minimum Redis compatibility version (database): 4.0.0
- Minimum Redis Enterprise Software version (cluster): 5.0.0

title: v2.2.18 (July 2022)

text:
This is a maintenance release for RedisBloom 2.2.

Update urgency: `MODERATE`: Program an upgrade of the server, but it's not urgent.

Details:

- Bug fixes:

    - [#481](https://github.com/RedisBloom/RedisBloom/issues/481) CF crashes on expansion 0
    - [#478](https://github.com/RedisBloom/RedisBloom/pull/478) [`BF.INFO`]() reports an inaccurate result about the memory footprint

title: v2.2.17 (June 2022)

text:
This is a maintenance release for RedisBloom 2.2.

Update urgency: `HIGH`: There is a critical bug that may affect a subset of users. Upgrade!

Details:

- Bug fixes:

    - [#451](https://github.com/RedisBloom/RedisBloom/pull/451) [`TOPK.INCRBY`](): fix calculation when old fingerprint count is reduced to 0
    - [#462](https://github.com/RedisBloom/RedisBloom/pull/462) [`CF.RESERVE`](): fix potential crash on incorrect number of parameters
    - [#434](https://github.com/RedisBloom/RedisBloom/pull/434) [`CF.INSERT`](): fix crash when capacity is negative
    - [#450](https://github.com/RedisBloom/RedisBloom/pull/450) [`TOPK.INCRBY`](): limit increment to 100,000 to prevent potential long freezes

title: v2.2.15 (May 2022)

text:
This is a maintenance release for RedisBloom 2.2.

Update urgency: None.

Details:

- Enhancements:

    - Added support for RedisBloom for AArch64 Linux

title: v2.2.14 (March 2022)

text:
This is a maintenance release for RedisBloom 2.2.

Update urgency: `MODERATE`: Program an upgrade of the server, but it's not urgent.

Details:

- Bug fixes:

    - [#419](https://github.com/RedisBloom/RedisBloom/pull/419) `CMS.INCRBY` now returns an error on overflow
    - [#412](https://github.com/RedisBloom/RedisBloom/pull/412) Fixed macOS build

title: v2.2.12 (February 2022)

text:
This is a maintenance release for RedisBloom 2.2.

Update urgency: `MODERATE`: Program an upgrade of the server, but it's not urgent.

Details:

- Bug fixes:

    - [#392](https://github.com/RedisBloom/RedisBloom/pull/392) Fixed a potential crash on Bloom filter expansion when loading from [AOF]()
    - [#404](https://github.com/RedisBloom/RedisBloom/pull/404) Fixed a potential crash on Cuckoo filter when calling `CF.LOADCHUNK` on a filter with `EXPANSION` greater than 1

title: v2.2.9 (November 2021)

text:
This is a maintenance release for RedisBloom 2.2.

Update urgency: `MODERATE`: Program an upgrade of the server, but it's not urgent.

Details:

- Bug fixes:

    - [#281](https://github.com/RedisBloom/RedisBloom/pull/281) Memory leak when loading TOPK keys from RDB.

    - [#303](https://github.com/RedisBloom/RedisBloom/pull/303) `CMS.INCRBY` for negative or non-number

    - [#354](https://github.com/RedisBloom/RedisBloom/pull/354) TOPK List Duplicate entries

    - [#369](https://github.com/RedisBloom/RedisBloom/pull/369) Free only latest filter on `CF.COMPACT`

    - [#371](https://github.com/RedisBloom/RedisBloom/pull/371) `CF.SCANDUMP` for cuckoo filter

    - [#374](https://github.com/RedisBloom/RedisBloom/pull/374) ReplicateVerbatim for `CF.LOADCHUNK` (similar to [#309](https://github.com/RedisBloom/RedisBloom/pull/309))

- Improvements:

    - [#331](https://github.com/RedisBloom/RedisBloom/pull/331) Add `WITHCOUNT` flag to `TOPK.LIST`

title: v2.2.6 (August 2021)

text:
This is a maintenance release for version 1.0.

Update urgency: `LOW` -  No need to upgrade unless there are new features you want to use.

Details:
- Enhancements:
    - #[333](https://github.com/redisbloom/redisbloom/issues/333) Support inter shards TLS capability

title: v2.2.4 (July 2020)

text:
Headlines:

- This release improves overall stability and provides fixes for found issues.

Details:

- Bug fixes:
    - #[215](https://github.com/redisbloom/redisbloom/issues/215) Count-Min-Sketch CMS.INCRBY command to reply with correct min result.
    - #[219](https://github.com/redisbloom/redisbloom/issues/219) Cuckoo Filter CF.DEBUG correct response formatting.

title: v2.2.3 (July 2020)

text:
Headlines:

- This release improves overall stability and provides fixes for found issues.

Details:

- Bug fixes:
    - [#217](https://github.com/RedisBloom/RedisBloom/pull/217) Client hung on `BF.INSERT` with multiple new items when a non-scaling filter is full.

title: v2.2.2 (March 2020)

text:
Headlines:

- This release improves overall stability and provides fixes for issues found after the previous release.

Details:

- Minor enhancements:
    - Bloom
        - #[180](https://github.com/redisbloom/redisbloom/issues/180) Removed the upper limit on Bloom Filter capacity.

title: v2.2.1 (January 2020)

text:
Headlines:

- This release improves overall stability and provides fixes for issues found after the previous release.

Details:

- Minor Enhancements:
    - Bloom
        - #[179](https://github.com/RedisBloom/RedisBloom/issues/179) Allow storing filters with size over 4294967295 (UINT32_MAX). Now 64 bits.
        - #[177](https://github.com/RedisBloom/RedisBloom/issues/177) Prevent passing both `EXPANSION` *and* `NONSCALING` parameters to `BF.RESERVE`.

title: v2.2.0 (December 2019)

text:
- Added functionality
    - Bloom
        - #[149](https://github.com/RedisBloom/RedisBloom/issues/149) `BF.INFO` returns details about a specific bloom filter
        - Scalable
            - #[153](https://github.com/RedisBloom/RedisBloom/issues/153) Ability to change the `EXPANSION` rate. This means each subsequent sub-filter will be `expansion` times larger as the previous one.
            - #[160](https://github.com/RedisBloom/RedisBloom/issues/160) Optimise the scaling up of filter according to the [Scalable Bloom Filter](https://core.ac.uk/download/pdf/55607643.pdf) paper
            - #[161](https://github.com/RedisBloom/RedisBloom/issues/161) Optional `NONSCALING` argument to disable scaling.  (This saves space since less hash functions are used)
        - #[155](https://github.com/RedisBloom/RedisBloom/issues/155) Disabling rounding up functionality
    - Cuckoo
        - #[149](https://github.com/RedisBloom/RedisBloom/issues/149) `CF.INFO` returns details about a specific cuckoo filter
        - Scalable
            - #[138](https://github.com/RedisBloom/RedisBloom/issues/138) Configurable `EXPANSION`. When an additional filter is created, its size will be the size of the current filter multiplied by the `expansion`.  Higher expansion rates will result in lower error rates.
            - #[142](https://github.com/RedisBloom/RedisBloom/issues/142) The maximum number of expansions limited to 32.
            - #[131](https://github.com/RedisBloom/RedisBloom/issues/131) Configurable `MAXITERATIONS`. Number of attempts to swap buckets before declaring filter as full and creating an additional filter.
        - #[135](https://github.com/RedisBloom/RedisBloom/issues/135) Configurable  `BUCKETSIZE`. Number of items in each bucket. Higher bucket size value improves the fill rate but result in a higher error rate and slightly slower operation speed.
        - #[142](https://github.com/RedisBloom/RedisBloom/issues/142) use of 64bit hash function
        - #[136](https://github.com/RedisBloom/RedisBloom/issues/136) expose compaction of filters in the API
    - CMS
        - #[97](https://github.com/RedisBloom/RedisBloom/issues/97) `CMS.INCRBY` returns count instead of ‘OK’

- Minor bug fixes
    - Bloom
        - #[154](https://github.com/RedisBloom/RedisBloom/issues/154) Check error rate is 1< (cannot be equal)
    - Cuckoo
        - #[134](https://github.com/RedisBloom/RedisBloom/issues/134) Added `CuckooInsert_MemAllocFailed` exception
        - #[130](https://github.com/RedisBloom/RedisBloom/issues/130) Number of deletes wasn’t saved to RDB
    - General
        - #[117](https://github.com/RedisBloom/RedisBloom/issues/117) Using `RMUtil_RegisterWriteDenyOOMCmd`
        - #[121](https://github.com/RedisBloom/RedisBloom/issues/121) Moved `ReplicaVerbatim` to end of functions


title: RedisBloom 2.4 release notes

title: Requirements

text:
RedisBloom v2.4.15 requires:

- Minimum Redis compatibility version (database): 6.0.16
- Minimum Redis Enterprise Software version (cluster): 6.2.8

title: v2.4.15 (April 2025)

text:
This is a maintenance release for RedisBloom 2.4.

Update urgency: `LOW`: No need to upgrade unless there are new features you want to use.

Details:

Improvements:
- [#865](https://github.com/redisbloom/redisbloom/pull/865) Added support for Azure Linux 3 (MOD-9171)

title: v2.4.14 (February 2025)

text:
This is a maintenance release for RedisBloom 2.4.

Update urgency: `MODERATE`: Program an upgrade of the server, but it's not urgent.

Details:

- Bug fixes:
  - [#828](https://github.com/redisbloom/redisbloom/pull/828) `CMS.INCRBY` does not notify `WATCH`ers or client-side caching (MOD-8193)
  - [#827](https://github.com/redisbloom/redisbloom/pull/827) Top-K - suboptimal results after RDB load due to missing initialization (MOD-8194)

title: v2.4.13 (January 2025)

text:
This is a maintenance release for RedisBloom 2.4.

Update urgency: `SECURITY`: There are security fixes in the release.

Details:

- **Security and privacy:**
  - [#845](https://github.com/redisbloom/redisbloom/pull/845) (CVE-2024-53993) CMS: potential out-of-bounds write (MOD-6970)

- Bug fixes:
  - [#845](https://github.com/redisbloom/redisbloom/pull/845) `CMS.MERGE` crashes or hangs on negative number of keys (MOD-6964)

- Improvements:
  - [#831](https://github.com/redisbloom/redisbloom/pull/831) Added support for Ubuntu 22 and macOS 13 and 14

title: v2.4.9 (March 2024)

text:
This is a maintenance release for RedisBloom 2.4.

Update urgency: `MODERATE`: Program an upgrade of the server, but it's not urgent.

Details:

- Bug fixes:

  - [#753](https://github.com/RedisBloom/RedisBloom/issues/753) Potential crash on `CMS.MERGE` when using invalid arguments

title: v2.4.8 (January 2024)

text:
This is a maintenance release for RedisBloom 2.4.

Update urgency: `HIGH`: There is a critical bug that may affect a subset of users. Upgrade!

Details:

- Bug fixes:

  - [#727](https://github.com/RedisBloom/RedisBloom/pull/727) Additional fixes for potential crash on `CF.LOADCHUNK` (MOD-6344)

title: v2.4.7 (January 2024)

text:
This is a maintenance release for RedisBloom 2.4.

Update urgency: `HIGH`: There is a critical bug that may affect a subset of users. Upgrade!

Details:

- Bug fixes:

  - [#735](https://github.com/RedisBloom/RedisBloom/pull/735) Potential crash on `CF.RESERVE` (MOD-6343)
  - [#727](https://github.com/RedisBloom/RedisBloom/pull/727) Potential crash on `CF.LOADCHUNK` (MOD-6344)

title: v2.4.6 (December 2023)

text:
This is a maintenance release for RedisBloom 2.4.

Update urgency: `LOW`: No need to upgrade unless there are new features you want to use.

Details:

- Bug fixes:

  - [#707](https://github.com/RedisBloom/RedisBloom/pull/707) Top-K: `TOPK.ADD` and `TOPK.QUERY` crash when an item name is an empty string (RED-114676)

- Improvements:

  - [#706](https://github.com/RedisBloom/RedisBloom/pull/706) Added support for CBL-Mariner 2 (MOD-6200)

title: v2.4.5 (April 2023)

text:
This is a maintenance release for RedisBloom 2.4.

Update urgency: `LOW`: No need to upgrade unless there are new features you want to use.

Details:

- Improvements:

  - Internal changes for supporting future Redis Enterprise releases

title: v2.4.4 (February 2023)

text:
This is a maintenance release for RedisBloom 2.4.

Update urgency: `MODERATE`: Program an upgrade of the server, but it's not urgent.

Details:

- Bug fixes:

  - [#609](https://github.com/RedisBloom/RedisBloom/issues/609) [CF.INFO]() - incorrect information for large filters

- Improvements:

  - [#389](https://github.com/RedisBloom/RedisBloom/issues/389) Introduce [BF.CARD]() to retrieve the cardinality of a Bloom filter or 0 when such key does not exist

title: v2.4 GA (v2.4.3) (November 2022)

text: This is the General Availability release of RedisBloom 2.4.

title: Highlights

text: RedisBloom 2.4 introduces a new sketch data structure: **t-digest**.

title: What's new in 2.4

text:
[t-digest](https://www.sciencedirect.com/science/article/pii/S2665963820300403) is a probabilistic data structure for estimating quantiles based on a data stream or a large dataset of floating-point values. It can be used to answer the following questions:

- What fraction of the values in the data stream are smaller than a given value?
- How many values in the data stream are smaller than a given value?
- Which value is smaller than _p_ percent of the values in the data stream? (what is the _p_-percentile value)?
- What is the mean value between the _p1_-percentile value and the _p2_-percentile value?
- What is the value of the _n_-th smallest / largest value in the data stream? (what is the value with [reverse] rank _n_)?

As for any other probabilistic data structures, t-digest requires sublinear space and has controllable space-accuracy tradeoffs.

Using t-digest is simple and straightforward:

* **Creating a sketch and adding observations**

  `TDIGEST.CREATE key [COMPRESSION compression]` initializes a new t-digest sketch (and errors if the key already exists). The `COMPRESSION` argument is used to specify the tradeoff between accuracy and memory consumption. The default is 100. Higher values mean more accuracy.

  `TDIGEST.ADD key value...` adds new observations (floating-point values) to the sketch. You can repeat calling [TDIGEST.ADD]() whenever new observations are available.

* **Estimating fractions or ranks by values**

  Use `TDIGEST.CDF key value...` to retrieve, for each input **value**, an estimation of the **fraction** of (observations **smaller** than the given value + half the observations equal to the given value).

  `TDIGEST.RANK key value...` is similar to [TDIGEST.CDF](), but used for estimating the number of observations instead of the fraction of observations. More accurately it returns, for each input **value**, an estimation of the **number** of (observations **smaller** than a given value + half the observations equal to the given value).

  And lastly, `TDIGEST.REVRANK key value...` is similar to [TDIGEST.RANK](), but returns, for each input **value**, an estimation of the **number** of (observations **larger** than a given value + half the observations equal to the given value).

* **Estimating values by fractions or ranks**

  `TDIGEST.QUANTILE key fraction...` returns, for each input **fraction**, an estimation of the **value** (floating point) that is **smaller** than the given fraction of observations.

  `TDIGEST.BYRANK key rank...` returns, for each input **rank**, an estimation of the **value** (floating point) with that rank.

  `TDIGEST.BYREVRANK key rank...` returns, for each input **reverse rank**, an estimation of the **value** (floating point) with that reverse rank.

* **Estimating trimmed mean**

  Use `TDIGEST.TRIMMED_MEAN key lowFraction highFraction` to retrieve an estimation of the mean value between the specified fractions.

  This is especially useful for calculating the average value ignoring outliers. For example, calculating the average value between the 20th percentile and the 80th percentile.

* **Merging sketches**

  Sometimes it is useful to merge sketches. For example, suppose we measure latencies for 3 servers, and we want to calculate the 90%, 95%, and 99% latencies for all the servers combined.

  `TDIGEST.MERGE destKey numKeys sourceKey... [COMPRESSION compression] [OVERRIDE]` merges multiple sketches into a single sketch.

  If `destKey` does not exist, a new sketch is created.

  If `destKey` is an existing sketch, its values are merged with the values of the source keys. To override the destination key contents, use `OVERRIDE`.

* **Retrieving sketch information**

  Use `TDIGEST.MIN` key and `TDIGEST.MAX key` to retrieve the minimal and maximal values in the sketch, respectively.

  Both return NaN (Not a Number) when the sketch is empty.

  Both commands return accurate results and are equivalent to `TDIGEST.BYRANK key 0` and `TDIGEST.BYREVRANK key 0` respectively.

  Use `TDIGEST.INFO key` to retrieve some additional information about the sketch.

* **Resetting a sketch**

  `TDIGEST.RESET key` empties the sketch and reinitializes it.