Time series
Ingest and query time series data with Redis
The Redis time series data type lets you store real-valued data points along with the time they were collected. You can combine the values from a selection of time series and query them by time or value range. You can also compute aggregate functions of the data over periods of time and create new time series from the results. When you create a time series, you can specify a maximum retention period for the data, relative to the last reported timestamp, to prevent the time series from growing indefinitely.
Time series support very fast reads and writes, making them ideal for applications such as:
- Instrument data logging
- System performance metrics
- Financial market data
- Internet of Things (IoT) sensor data
- Smart metering
- Quality of service (QoS) monitoring
Redis time series are available in Redis Open Source, Redis Software, and Redis Cloud. See Install Redis Open Source or Install Redis Enterprise for full installation instructions.
Create a time series
You can create a new empty time series with the TS.CREATE
command, specifying a key name. Alternatively, if you use TS.ADD
to add data to a time series key that does not exist, it is automatically created (see
Adding data points below for more information about TS.ADD
).
> TS.CREATE thermometer:1
OK
> TYPE thermometer:1
TSDB-TYPE
> TS.INFO thermometer:1
1) totalSamples
2) (integer) 0
.
.
The timestamp for each data point is a 64-bit integer value. The value represents a Unix timestamp, measured in milliseconds since the Unix epoch. When you create a time series, you can specify a maximum retention period for the data, relative to the last reported timestamp. A retention period of zero means the data does not expire.
# Create a new time series with a first value of 10.8 (Celsius),
# recorded at time 1, with a retention period of 100ms.
> TS.ADD thermometer:2 1 10.8 RETENTION 100
(integer) 1
> TS.INFO thermometer:2
.
.
9) retentionTime
10) (integer) 100
.
.
You can also add one or more labels to a time series when you create it. Labels are name-value pairs where both the name and value are strings. You can use the names and values to select subsets of all the available time series for queries and aggregations.
> TS.ADD thermometer:3 1 10.4 LABELS location UK type Mercury
(integer) 1
> TS.INFO thermometer:3
1) totalSamples
2) (integer) 1
3) memoryUsage
4) (integer) 5000
.
.
19) labels
20) 1) 1) "location"
2) "UK"
2) 1) "type"
2) "Mercury"
.
.
Add data points
You can add individual data points with TS.ADD
,
but you can also use TS.MADD
to add multiple data
points to one or more time series in a single command. (Note that unlike TS.ADD
, TS.MADD
doesn't create any new time series if you specify keys that don't exist.) The return value
is an array containing the number of samples in each time series after the operation.
If you use the *
character as the timestamp, Redis will record the current
Unix time, as reported by the server's clock.
> TS.MADD thermometer:1 1 9.2 thermometer:1 2 9.9 thermometer:2 2 10.3
1) (integer) 1
2) (integer) 2
3) (integer) 2
Query data points
Use TS.GET
to retrieve the data point
with the highest timestamp in a time series. This returns both the timestamp and the value.
# The last recorded temperature for thermometer:2
# was 10.3 at time 2ms.
> TS.GET thermometer:2
1) (integer) 2
2) 10.3
Use TS.RANGE
to retrieve data points
from a time series that fall within a given timestamp range. The range is inclusive,
meaning that samples whose timestamp equals the start or end of the range are included.
You can use -
and +
as the start and end of the range, respectively, to
indicate the minimum and maximum timestamps in the series. The response is
an array of timestamp-value pairs returned in ascending order by timestamp.
If you want the results in descending order, use TS.REVRANGE
with the same parameters.
# Add 5 data points to a time series named "rg:1".
> TS.CREATE rg:1
OK
> TS.MADD rg:1 0 18 rg:1 1 14 rg:1 2 22 rg:1 3 18 rg:1 4 24
1) (integer) 0
2) (integer) 1
3) (integer) 2
4) (integer) 3
5) (integer) 4
# Retrieve all the data points in ascending order.
> TS.RANGE rg:1 - +
1) 1) (integer) 0
2) 18
2) 1) (integer) 1
2) 14
3) 1) (integer) 2
2) 22
4) 1) (integer) 3
2) 18
5) 1) (integer) 4
2) 24
# Retrieve data points up to time 1 (inclusive).
> TS.RANGE rg:1 - 1
1) 1) (integer) 0
2) 18
2) 1) (integer) 1
2) 14
# Retrieve data points from time 3 onwards.
> TS.RANGE rg:1 3 +
1) 1) (integer) 3
2) 18
2) 1) (integer) 4
2) 24
# Retrieve all the data points in descending order.
> TS.REVRANGE rg:1 - +
1) 1) (integer) 4
2) 24
2) 1) (integer) 3
2) 18
3) 1) (integer) 2
2) 22
4) 1) (integer) 1
2) 14
5) 1) (integer) 0
2) 18
# Retrieve data points up to time 1 (inclusive), but
# return them in descending order.
> TS.REVRANGE rg:1 - 1
1) 1) (integer) 1
2) 14
2) 1) (integer) 0
2) 18
Both TS.RANGE
and TS.REVRANGE
also let you filter results. Specify
a list of timestamps to include only samples with those exact timestamps
in the results (you must still specify timestamp range parameters if you
use this option). Specify a minimum and maximum value to include only
samples within that range. The value range is inclusive and you can
use the same value for the minimum and maximum to filter for a single value.
> TS.RANGE rg:1 - + FILTER_BY_TS 0 2 4
1) 1) (integer) 0
2) 18
2) 1) (integer) 2
2) 22
3) 1) (integer) 4
2) 24
> TS.REVRANGE rg:1 - + FILTER_BY_TS 0 2 4 FILTER_BY_VALUE 20 25
1) 1) (integer) 4
2) 24
2) 1) (integer) 2
2) 22
> TS.REVRANGE rg:1 - + FILTER_BY_TS 0 2 4 FILTER_BY_VALUE 22 22
1) 1) (integer) 2
2) 22
Query multiple time series
The TS.GET
, TS.RANGE
, and TS.REVRANGE
commands also have
corresponding
TS.MGET
,
TS.MRANGE
, and
TS.MREVRANGE
versions that
operate on multiple time series. TS.MGET
returns the data point with the highest
timestamp from each time series, while TS.MRANGE
and TS.MREVRANGE
return data points from a range of timestamps in each time series.
The parameters are mostly the same except that the multiple time series
commands don't take a key name as the first parameter. Instead, you
specify a filter expression to include only time series with
specific labels. (See Creating a time series
above to learn how to add labels to a time series.) The filter expressions
use a simple syntax that lets you include or exclude time series based on
the presence or value of a label. See the description in the
TS.MGET
command reference
for details of the filter syntax. You can also request that
data points be returned with all their labels or with a selected subset of them.
# Create three new "rg:" time series (two in the US
# and one in the UK, with different units) and add some
# data points.
> TS.CREATE rg:2 LABELS location us unit cm
OK
> TS.CREATE rg:3 LABELS location us unit in
OK
> TS.CREATE rg:4 LABELS location uk unit mm
OK
> TS.MADD rg:2 0 1.8 rg:3 0 0.9 rg:4 0 25
1) (integer) 0
2) (integer) 0
3) (integer) 0
> TS.MADD rg:2 1 2.1 rg:3 1 0.77 rg:4 1 18
1) (integer) 1
2) (integer) 1
3) (integer) 1
> TS.MADD rg:2 2 2.3 rg:3 2 1.1 rg:4 2 21
1) (integer) 2
2) (integer) 2
3) (integer) 2
> TS.MADD rg:2 3 1.9 rg:3 3 0.81 rg:4 3 19
1) (integer) 3
2) (integer) 3
3) (integer) 3
> TS.MADD rg:2 4 1.78 rg:3 4 0.74 rg:4 4 23
1) (integer) 4
2) (integer) 4
3) (integer) 4
# Retrieve the last data point from each US time series. If
# you don't specify any labels, an empty array is returned
# for the labels.
> TS.MGET FILTER location=us
1) 1) "rg:2"
2) (empty array)
3) 1) (integer) 4
2) 1.78
2) 1) "rg:3"
2) (empty array)
3) 1) (integer) 4
2) 7.4E-1
# Retrieve the same data points, but include the `unit`
# label in the results.
> TS.MGET SELECTED_LABELS unit FILTER location=us
1) 1) "rg:2"
2) 1) 1) "unit"
2) "cm"
3) 1) (integer) 4
2) 1.78
2) 1) "rg:3"
2) 1) 1) "unit"
2) "in"
3) 1) (integer) 4
2) 7.4E-1
# Retrieve data points up to time 2 (inclusive) from all
# time series that use millimeters as the unit. Include all
# labels in the results.
> TS.MRANGE - 2 WITHLABELS FILTER unit=mm
1) 1) "rg:4"
2) 1) 1) "location"
2) "uk"
2) 1) "unit"
2) "mm"
3) 1) 1) (integer) 0
2) 25
2) 1) (integer) 1
2) 18
3) 1) (integer) 2
2) 21
# Retrieve data points from time 1 to time 3 (inclusive) from
# all time series that use centimeters or millimeters as the unit,
# but only return the `location` label. Return the results
# in descending order of timestamp.
> TS.MREVRANGE 1 3 SELECTED_LABELS location FILTER unit=(cm,mm)
1) 1) "rg:2"
2) 1) 1) "location"
2) "us"
3) 1) 1) (integer) 3
2) 1.9
2) 1) (integer) 2
2) 2.3
3) 1) (integer) 1
2) 2.1
2) 1) "rg:4"
2) 1) 1) "location"
2) "uk"
3) 1) 1) (integer) 3
2) 19
2) 1) (integer) 2
2) 21
3) 1) (integer) 1
2) 18
Aggregation
A time series can become large if samples are added very frequently. Instead of dealing with individual samples, it is sometimes useful to split the full time range of the series into equal-sized "buckets" and represent each bucket by an aggregate value, such as the average or maximum value.
For example, if you expect to collect more than one billion data points in a day, you could aggregate the data using buckets of one minute. Since each bucket is represented by a single value, this reduces the dataset size to 1,440 data points (24 hours x 60 minutes = 1,440 minutes).
The range query commands let you specify an aggregation function and bucket size. The available aggregation functions are:
avg
: Arithmetic mean of all valuessum
: Sum of all valuesmin
: Minimum valuemax
: Maximum valuerange
: Difference between the highest and the lowest valuecount
: Number of valuesfirst
: Value with lowest timestamp in the bucketlast
: Value with highest timestamp in the bucketstd.p
: Population standard deviation of the valuesstd.s
: Sample standard deviation of the valuesvar.p
: Population variance of the valuesvar.s
: Sample variance of the valuestwa
: Time-weighted average over the bucket's timeframe (since RedisTimeSeries v1.8)
For example, the example below shows an aggregation with the avg
function over all
five data points in the rg:2
time series. The bucket size is 2ms, so there are three
aggregated values with only one value used to calculate the average for the last bucket.
> TS.RANGE rg:2 - + AGGREGATION avg 2
1) 1) (integer) 0
2) 1.9500000000000002
2) 1) (integer) 2
2) 2.0999999999999996
3) 1) (integer) 4
2) 1.78
Bucket alignment
The sequence of buckets has a reference timestamp, which is the timestamp where
the first bucket in the sequence starts. By default, the reference timestamp is zero.
For example, the following commands create a time series and apply a min
aggregation
with a bucket size of 25 milliseconds at the default zero alignment.
> TS.CREATE sensor3
OK
> TS.MADD sensor3 10 1000 sensor3 20 2000 sensor3 30 3000 sensor3 40 4000 sensor3 50 5000 sensor3 60 6000 sensor3 70 7000
1) (integer) 10
2) (integer) 20
3) (integer) 30
4) (integer) 40
5) (integer) 50
6) (integer) 60
7) (integer) 70
> TS.RANGE sensor3 10 70 AGGREGATION min 25
1) 1) (integer) 0
2) 1000
2) 1) (integer) 25
2) 3000
3) 1) (integer) 50
2) 5000
The diagram below shows the aggregation buckets and their alignment to the reference timestamp at time zero.
Value: | (1000) (2000) (3000) (4000) (5000) (6000) (7000)
Timestamp: |-------|10|-------|20|-------|30|-------|40|-------|50|-------|60|-------|70|--->
Bucket(25ms): |_________________________||_________________________||___________________________|
V V V
min(1000, 2000)=1000 min(3000, 4000)=3000 min(5000, 6000, 7000)=5000
You can also align the buckets to the start or end of the query range. For example, the following command aligns the buckets to the start of the query range at time 10.
> TS.RANGE sensor3 10 70 AGGREGATION min 25 ALIGN start
1) 1) (integer) 10
2) 1000
2) 1) (integer) 35
2) 4000
3) 1) (integer) 60
2) 6000
The diagram below shows this arrangement of buckets.
Value: | (1000) (2000) (3000) (4000) (5000) (6000) (7000)
Timestamp: |-------|10|-------|20|-------|30|-------|40|-------|50|-------|60|-------|70|--->
Bucket(25ms): |__________________________||_________________________||___________________________|
V V V
min(1000, 2000, 3000)=1000 min(4000, 5000)=4000 min(6000, 7000)=6000
Aggregation across timeseries
By default, the results from
TS.MRANGE
and
TS.MREVRANGE
are grouped by time series. However, you can use the GROUPBY
and REDUCE
options to group them by label and apply an aggregation over elements
that have the same timestamp and the same label value (this feature is available from RedisTimeSeries v1.6 onwards).
For example, the following commands create four time series, two for the UK and two for the US, and add some data points. The first TS.MRANGE
command groups the results by country and applies a max
aggregation to find the maximum sample value in each country at each timestamp. The second TS.MRANGE
command uses the same grouping, but applies an avg
aggregation.
> TS.CREATE wind:1 LABELS country uk
OK
> TS.CREATE wind:2 LABELS country uk
OK
> TS.CREATE wind:3 LABELS country us
OK
> TS.CREATE wind:4 LABELS country us
OK
> TS.MADD wind:1 1 12 wind:2 1 18 wind:3 1 5 wind:4 1 20
1) (integer) 1
2) (integer) 1
3) (integer) 1
4) (integer) 1
> TS.MADD wind:1 2 14 wind:2 2 21 wind:3 2 4 wind:4 2 25
1) (integer) 2
2) (integer) 2
3) (integer) 2
4) (integer) 2
> TS.MADD wind:1 3 10 wind:2 3 24 wind:3 3 8 wind:4 3 18
1) (integer) 3
2) (integer) 3
3) (integer) 3
4) (integer) 3
# The result pairs contain the timestamp and the maximum sample value
# for the country at that timestamp.
> TS.MRANGE - + FILTER country=(us,uk) GROUPBY country REDUCE max
1) 1) "country=uk"
2) (empty array)
3) 1) 1) (integer) 1
2) 18
2) 1) (integer) 2
2) 21
3) 1) (integer) 3
2) 24
2) 1) "country=us"
2) (empty array)
3) 1) 1) (integer) 1
2) 20
2) 1) (integer) 2
2) 25
3) 1) (integer) 3
2) 18
# The result pairs contain the timestamp and the average sample value
# for the country at that timestamp.
> TS.MRANGE - + FILTER country=(us,uk) GROUPBY country REDUCE avg
1) 1) "country=uk"
2) (empty array)
3) 1) 1) (integer) 1
2) 15
2) 1) (integer) 2
2) 17.5
3) 1) (integer) 3
2) 17
2) 1) "country=us"
2) (empty array)
3) 1) 1) (integer) 1
2) 12.5
2) 1) (integer) 2
2) 14.5
3) 1) (integer) 3
2) 13
Compaction
Aggregation queries let you extract the important information from a large data set into a smaller, more manageable set. If you are continually adding new data to a time series as it is generated, you may need to run the same aggregation regularly on the latest data. Instead of running the query manually each time, you can add a compaction rule to a time series to compute an aggregation incrementally on data as it arrives. The values from the aggregation buckets are stored in a separate time series, leaving the original series unchanged.
Use TS.CREATERULE
to create a
compaction rule, specifying the source and destination time series keys, the
aggregation function, and the bucket duration. Note that the destination time
series must already exist when you create the rule and also that the compaction will
only process data that is added to the source series after you create the rule.
For example, you could use the commands below to create a time series along with a compaction rule to find the minimum reading in each period of 3ms.
# The source time series.
> TS.CREATE hyg:1
OK
# The destination time series for the compacted data.
> TS.CREATE hyg:compacted
OK
# The compaction rule.
> TS.CREATERULE hyg:1 hyg:compacted AGGREGATION min 3
OK
> TS.INFO hyg:1
.
.
23) rules
24) 1) 1) "hyg:compacted"
2) (integer) 3
3) MIN
4) (integer) 0
.
.
> TS.INFO hyg:compacted
.
.
21) sourceKey
22) "hyg:1"
.
.
Adding data points within the first 3ms (the first bucket) doesn't produce any data in the compacted series. However, when you add data for time 4 (in the second bucket), the compaction rule computes the minimum value for the first bucket and adds it to the compacted series.
> TS.MADD hyg:1 0 75 hyg:1 1 77 hyg:1 2 78
1) (integer) 0
2) (integer) 1
3) (integer) 2
> ts.range hyg:compacted - +
(empty array)
> TS.ADD hyg:1 3 79
(integer) 3
> ts.range hyg:compacted - +
1) 1) (integer) 0
2) 75
The general strategy is that the rule does not add data to the compaction for the latest bucket in the source series, but will add and update the compacted data for any previous buckets. This reflects the typical usage pattern of adding data samples sequentially in real time (an aggregate value typically isn't correct until its bucket period is over). But note that earlier buckets are not "closed" when you add data to a later bucket. If you add or delete data in a bucket before the latest one, the compaction rule will still update the compacted data for that bucket.
Delete data points
Use TS.DEL
to delete data points
that fall within a given timestamp range. The range is inclusive, meaning that
samples whose timestamp equals the start or end of the range are deleted.
If you want to delete a single timestamp, use it as both the start and end of the range.
> TS.INFO thermometer:1
1) totalSamples
2) (integer) 2
3) memoryUsage
4) (integer) 4856
5) firstTimestamp
6) (integer) 1
7) lastTimestamp
8) (integer) 2
.
.
> TS.ADD thermometer:1 3 9.7
(integer) 3
> TS.INFO thermometer:1
1) totalSamples
2) (integer) 3
3) memoryUsage
4) (integer) 4856
5) firstTimestamp
6) (integer) 1
7) lastTimestamp
8) (integer) 3
.
.
> TS.DEL thermometer:1 1 2
(integer) 2
> TS.INFO thermometer:1
1) totalSamples
2) (integer) 1
3) memoryUsage
4) (integer) 4856
5) firstTimestamp
6) (integer) 3
7) lastTimestamp
8) (integer) 3
.
.
> TS.DEL thermometer:1 3 3
(integer) 1
> TS.INFO thermometer:1
1) totalSamples
2) (integer) 0
.
.
Use time series with other metrics tools
In the RedisTimeSeries GitHub organization, you can find projects that help you integrate RedisTimeSeries with other tools, including:
- Prometheus, a read/write adapter to use RedisTimeSeries as the backend database.
- Grafana 7.1+, using the Redis Data Source.
- Telegraf. Download the plugin from InfluxData.
- StatsD, Graphite exports using graphite protocol.
More information
The other pages in this section describe RedisTimeSeries concepts in more detail. See also the time series command reference.