How to migrate from Amazon ElastiCache to Redis Enterprise or Redis Cloud?

Question

How to migrate from Amazon ElastiCache to Redis Enterprise or Redis Cloud?

Answer

Before migrating from ElastiCache to Redis Enterprise or Redis Cloud, you should evaluate the migration and consider the best option. We will describe three scenarios.

Database replacement

If you can tolerate a flush of all your Redis data, there’s no need to migrate the data at all. You can point your application to a new Redis Cloud database by configuring your client libraries and start working with a fresh, empty database. This can be a valid option if you use Redis as a cache and can restart without data. However, slow performance may impact your service while data is rehydrated, especially if Redis is used as a cache.

Offline data migration

If you use Redis for more than caching, such as session storage, you can't afford data loss or performance issues from a full data flush. For this, a durable persistence option is available. Next, consider if you can handle downtime, which might only last a few minutes, depending on the time needed to export and import your Redis data. Test this in your environment to estimate the downtime accurately, but watch out for memory usage issues common with ElastiCache. If the downtime is acceptable, we recommend offline data migration. It's straightforward but will cause downtime during the data transfer from ElastiCache to Redis Cloud.

Live data migration

Live data migration can be done without downtime or data loss, but it is more complex and requires careful consideration. This method uses an external tool called RIOT, which was developed by Redis experts. The documentation provides guidance, and expert support is available if needed.

Pros:

• Migrates live ElastiCache data to Redis Cloud
• No downtime

Cons:

• More complex with potential issues
• It may need tuning during testing
• It may not work for large key sizes (100MB+)
• Data consistency is not guaranteed
• High CPU usage
• It may require changes to ElastiCache
• RIOT support is the best effort

Let's consider an example to perform live data migration. You will require a few resources to use the migration tool:

• A Redis Cloud subscription
• Amazon ElastiCache (a primary endpoint in case of a single-master EC and a configuration endpoint in case of a clustered EC: Refer to Finding Connection Endpoints on the ElastiCache documentation to learn more)
• An Amazon EC2 instance based on Linux

Use SSH to connect to the new EC2 instance from your computer as shown here:

ssh -i “public key” AWS_EC2_INSTANCE

Now install the redis-cli tool:

$ sudo apt update
$ sudo apt install -y redis-tools

Verify the connectivity with the ElastiCache database

$ redis-cli -h ELASTICACHE_ENDPOINT -p 6379

Using the RIOT migration tool. For that, we recommended using OpenJDK 11 or later:

sudo add-apt-repository ppa:openjdk-r/ppa && sudo apt-get update -q && sudo apt install -y openjdk-11-jdk

Finally, complete the setup installing RIOT. Download the desired release. Then, unzip the package and make sure the RIOT binaries are in place, as shown here:

unzip riot-redis-2.0.8.zip
cd riot-redis-2.0.8/bin/

You can check the version of RIOT by running the command below:

./riot-redis --version
RIOT version "2.0.8"

Once Java and RIOT are installed, we are all set to begin the migration process with the command below, which replicates data directly from the source (ElastiCache) to the target (Redis Cloud). You can replicate the data from ElastiCache to Redis Cloud by running the below command:

sudo ./riot-redis -r redis://ELASTICACHE_ENDPOINT:6379 replicate -r redis://password@REDIS_CLOUD_ENDPOINT:port --live

ElastiCache can be configured in two ways: clustered and non-clustered. In a clustered ElastiCache, you must include the –cluster option before the source ElastiCache endpoint.

Decide on and execute your migration method. After migrating to Redis Cloud, explore new features like triggers, functions, and auto-tiering to manage large datasets.

References

RIOT