Configuration Parameters
Redis time series support multiple configuration parameters.
As of Redis Community Edition 8.0, configuration parameters for the time series data structure are now set in the following ways:
- At load time via your
redis.conf
file. - At run time (where applicable) using the
CONFIG SET
command.
Also, Redis CE 8.0 persists probabilistic configuration parameters just like any other configuration parameters (e.g., using the CONFIG REWRITE
command).
Time series configuration parameters
Parameter name (version < 8.0) |
Parameter name (version ≥ 8.0) |
Run-time | Redis Software |
Redis Cloud |
---|---|---|---|---|
CHUNK_SIZE_BYTES | ts-chunk-size-bytes | ✅ | ✅ Supported |
✅ Flexible & Annual |
COMPACTION_POLICY | ts-compaction-policy | ✅ | ✅ Supported |
✅ Flexible & Annual |
DUPLICATE_POLICY | ts-duplicate-policy | ✅ | ✅ Supported |
✅ Flexible & Annual |
ENCODING | ts-encoding | ✅ | ✅ Supported |
✅ Flexible & Annual |
IGNORE_MAX_TIME_DIFF | ts-ignore-max-time-diff | ✅ | ||
IGNORE_MAX_VAL_DIFF | ts-ignore-max-val-diff | ✅ | ||
NUM_THREADS | ts-num-threads | ⬜ | ✅ Supported |
✅ Flexible & Annual |
RETENTION_POLICY | ts-retention-policy | ✅ | ✅ Supported |
✅ Flexible & Annual |
OSS_GLOBAL_PASSWORD | Deprecated in v8.0.0. Replace with the masterauth password. |
✅ |
ts-chunk-size-bytes
Default initial allocation size, in bytes, for the data part of each new chunk. This default value is applied to each new time series upon its creation. Actual chunks may consume more memory.
Type: integer
Valid range: [48 .. 1048576]
; must be a multiple of 8
Default: 4096
ts-compaction-policy
Default compaction rules for newly created keys with TS.ADD
, TS.INCRBY
, and TS.DECRBY
.
Type: string
Default: No compaction rules.
Discussion
Example
$ redis-server --loadmodule ./redistimeseries.so NUM_THREADS 3
COMPACTION_POLICY
Default compaction rules for newly created key with TS.ADD
, TS.INCRBY
, and TS.DECRBY
.
Note that COMPACTION_POLICY
has no effect on keys created with TS.CREATE
. To understand the motivation for this behavior, consider the following scenario: Suppose a COMPACTION_POLICY
is defined, but then one wants to manually create an additional compaction rule (using TS.CREATERULE
) which requires first creating an empty destination key (using TS.CREATE
). But now there is a problem: due to the COMPACTION_POLICY
, automatic compactions would be undesirably created for that destination key.
Each rule is separated by a semicolon (;
), the rule consists of multiple fields that are separated by a colon (:
):
-
Aggregation type: One of the following:
Aggregator Description avg
Arithmetic mean of all values sum
Sum of all values min
Minimum value max
Maximum value range
Difference between the highest and the lowest value count
Number of values first
The value with the lowest timestamp in the bucket last
The value with the highest timestamp in the bucket std.p
Population standard deviation of the values std.s
Sample standard deviation of the values var.p
Population variance of the values var.s
Sample variance of the values twa
Time-weighted average of all values (since v1.8) -
Duration of each time bucket - number and the time representation (Example for one minute:
1M
,60s
, or60000m
)- m - millisecond
- s - seconds
- M - minute
- h - hour
- d - day
-
Retention time - number and the time representation (Example for one minute:
1M
,60s
, or60000m
)- m - millisecond
- s - seconds
- M - minute
- h - hour
- d - day
0m
,0s
,0M
,0h
, or0d
means no expiration. -
(Since v1.8):
Optional: Time bucket alignment - number and the time representation (Example for one minute:
1M
,60s
, or60000m
)- m - millisecond
- s - seconds
- M - minute
- h - hour
- d - day
Assure that there is a bucket that starts at exactly alignTimestamp after the epoch and align all other buckets accordingly. Default value: 0 (aligned with the epoch). Example: if bucketDuration is 24 hours, setting alignTimestamp to
6h
(6 hours after the Epoch) will ensure that each bucket’s timeframe is [06:00 .. 06:00).
COMPACTION_POLICY
, you must use hash tags for all time series key names. This ensures that Redis will create each compaction in the same hash slot as its source key. If you don't, the system may fail to compact the data without displaying any error messages.When a compaction policy is defined, compaction rules will be created automatically for newly created time series, and their key would be set to:
-
If the time bucket alignment is 0:
key_agg_dur where key is the key of the source time series, agg is the aggregator (in uppercase), and dur is the bucket duration in milliseconds. Example:
key_SUM_60000
. -
If the time bucket alignment is not 0:
key_agg_dur_aln where key is the key of the source time series, agg is the aggregator (in uppercase), dur is the bucket duration in milliseconds, and aln is the time bucket alignment in milliseconds. Example:
key_SUM_60000_1000
.
Examples
max:1M:1h
- Aggregate usingmax
over one-minute windows and retain the last hourtwa:1d:0m:360M
- Aggregate daily [06:00 .. 06:00) usingtwa
; no expiration
ts-duplicate-policy
The default policy for handling insertion (TS.ADD
and TS.MADD
) of multiple samples with identical timestamps, with one of the following values:
Example
$ redis-server --loadmodule ./redistimeseries.so COMPACTION_POLICY max:1m:1h;min:10s:5d:10d;last:5M:10m;avg:2h:10d;avg:3d:100d
RETENTION_POLICY
Default retention period, in milliseconds, for newly created keys.
Retention period is the maximum age of samples compared to highest reported timestamp, per key. Samples are expired based solely on the difference between their timestamp and the timestamps passed to subsequent TS.ADD
, TS.MADD
, TS.INCRBY
, and TS.DECRBY
calls.
The value 0
means no expiration.
When both COMPACTION_POLICY
and RETENTION_POLICY
are specified, the retention of newly created compactions is according to the retention time specified in COMPACTION_POLICY
.
Default
0
Example
Setting the default retention to 300 days:
$ redis-server --loadmodule ./redistimeseries.so RETENTION_POLICY 25920000000
DUPLICATE_POLICY
Is policy for handling insertion (TS.ADD
and TS.MADD
) of multiple samples with identical timestamps, with one of the following values:
policy | description |
---|---|
BLOCK |
Ignore any newly reported value and reply with an error |
FIRST |
Ignore any newly reported value |
LAST |
Override with the newly reported value |
MIN |
Only override if the value is lower than the existing value |
MAX |
Only override if the value is higher than the existing value |
SUM |
If a previous sample exists, add the new sample to it so that the updated value is equal to (previous + new). If no previous sample exists, set the updated value equal to the new value. |
The default value is applied to each new time series upon its creation.
Type: string
Default: BLOCK
Precedence order
Since the duplication policy can be provided at different levels, the actual precedence of the used policy will be:
TS.ADD
'sON_DUPLICATE_POLICY
optional argument.- Key-level policy, as set with
TS.CREATE
's andTS.ALTER
'sDUPLICATE_POLICY
optional argument. - The
ts-duplicate-policy
configuration parameter. - The default policy.
ts-encoding
Note: Before v1.6 this configuration parameter was named CHUNK_TYPE
.
Default chunk encoding for automatically created time series keys when ts-compaction-policy is configured.
Type: string
Valid values: COMPRESSED
, UNCOMPRESSED
Default: COMPRESSED
ts-ignore-max-time-diff and ts-ignore-max-val-diff
Default values for newly created keys.
Types:
ts-ignore-max-time-diff
: integerts-ignore-max-val-diff
: double
Valid ranges:
ts-ignore-max-time-diff
:[0 .. 9,223,372,036,854,775,807]
ts-ignore-max-val-diff
:[0 .. 1.7976931348623157e+308]
Defaults:
ts-ignore-max-time-diff
: 0ts-ignore-max-val-diff
: 0.0
Discussion
Many sensors report data periodically. Often, the difference between the measured value and the previous measured value is negligible and related to random noise or to measurement accuracy limitations. In such situations it may be preferable not to add the new measurement to the time series.
A new sample is considered a duplicate and is ignored if the following conditions are met:
- The time series is not a compaction.
- The time series'
ts-duplicate-policy
isLAST
. - The sample is added in-order (
timestamp ≥ max_timestamp
). - The difference of the current timestamp from the previous timestamp (
timestamp - max_timestamp
) is less than or equal tots-ignore-max-time-diff
. - The absolute value difference of the current value from the value at the previous maximum timestamp (
abs(value - value_at_max_timestamp
) is less than or equal tots-ignore-max-val-diff
.
where max_timestamp
is the timestamp of the sample with the largest timestamp in the time series, and value_at_max_timestamp
is the value at max_timestamp
.
ts-num-threads
The maximum number of per-shard threads for cross-key queries when using cluster mode (TS.MRANGE
, TS.MREVRANGE
, TS.MGET
, and TS.QUERYINDEX
). The value must be equal to or greater than 1
. Note that increasing this value may either increase or decrease the performance!
Type: integer
Valid range: [1..16]
Redis CE default: 3
Redis Software default: Set by plan, and automatically updates when you change your plan.
Redis Cloud defaults:
- Flexible & Annual: Set by plan
- Free & Fixed:
1
ts-retention-policy
Default retention period, in milliseconds, for newly created keys.
Type: integer
Valid range: [0 .. 9,223,372,036,854,775,807]
Default: 0
Retention period is the maximum age of samples compared to highest reported timestamp, per key. Samples are expired based solely on the difference between their timestamps and the timestamps passed to subsequent TS.ADD
, TS.MADD
, TS.INCRBY
, and TS.DECRBY
calls.
The value 0
means no expiration.
When both ts-compaction-policy
and ts-retention-policy
are specified, the retention of newly created compactions is according to the retention time specified in ts-compaction-policy
.
Setting configuration parameters on module load (deprecated)
These methods are deprecated beginning with Redis CE v8.0.0.
Setting configuration parameters at load-time is done by appending arguments after the --loadmodule
argument when starting a server from the command line or after the loadmodule
directive in a Redis config file. For example:
In redis.conf:
loadmodule ./redistimeseries.so [OPT VAL]...
From the Redis CLI, using the MODULE LOAD command:
127.0.0.6379> MODULE LOAD redistimeseries.so [OPT VAL]...
From the command line:
$ redis-server --loadmodule ./redistimeseries.so [OPT VAL]...