Become a Redis expert.
See what's new in Redis University
share
Hazelcast has published two benchmarks comparing its system against the Redis open source database:
Recently, multiple prospects and customers have asked us to run a similar benchmark (over the network) between Redis Enterprise and Hazelcast. Given that Redis Enterprise clusters are based on a different architecture than open source clusters (as further explained here), we wanted to see if there would be any differences in these results.
At first, we aimed to reproduce the exact same benchmark setup used by Hazelcast, but they apparently used proprietary hardware. As a company born in the cloud, here at Redis we do not host a single server in-house, and thus decided to look for a similar server configuration on AWS. Here is the setup we used:
| Parameter | Value |
| Server & client machines |
|
| Number of server machines | 3 |
| Number of client machines | 3 |
| Redis Enterprise | Hazelcast | |
| OS version | Ubuntu 16.04 | |
| Cluster version | 5.2.0 | 3.9 |
| Cluster configuration |
| 3 members |
In its benchmarks, Hazelcast used RadarGun to orchestrate the load generated against both Hazelcast and Redis. In the Redis case, RadarGun launched a Jedis cluster and used a configuration file similar to this (although we couldn’t find the exact setup in this fork that Hazelcast listed).
Using the right tool to test a product is crucial for running a successful benchmark. For instance, most experts would prefer to test Redis with a pipelining technique, as this speeds up Redis and is used by a large portion of Redis users. However, we didn’t find a pipeline configuration in the Hazelcast benchmarks.
We therefore opted for the following approach:
| Redis Enterprise | Hazelcast | |
| Version | memtier_benchmark version 1.2.8 | RadarGun version 3.0.0 |
| Number of threads per client | 64 | |
| Read:Write ratio | 80:20 | |
| Number of objects | 4 million | |
| Object size | Strings with a random distribution of lengths between 100 bytes and 10,000 bytes | |
| Total dataset size | 42GB | |
| Command line used | memtier_benchmark –cluster-mode -s <ip_address> -p <port_number> –key-maximum=4000000 –test-time=900 –data-size-range=100-10000 -c 1 -t 64 –pipeline=5 –key-pattern=P:P –ratio=1:4 –distinct-client-seed | dist.sh -c benchmark-hazelcast-server.xml -t -u ubuntu -o `pwd` -m ip-radargun:2103 ip-server-1 ip-server-2 ip-server-3 ip-client-1 ip-client-2 ip-client |
The results of our benchmark are presented below:
We re-ran the benchmark comparing Hazelcast against Redis over the network with two major changes:
We found the results for Hazelcast’s performance to be very similar to (or better than) what the company published here. Therefore, we tend to believe the Hazelcast cluster and the RadarGun orchestrator were configured correctly.
On the other hand, our findings reached a much better throughput (over 3.5X) and latency (~3X) for the Redis Enterprise cluster than those from Hazelcast. We think these differences were related to:
If you have questions related to this benchmark, please feel free to email me: keren at redis.com. If you would like to find out more about Redis Enterprise, visit here or email product at redis.com.
