{
  "schema_version": 2,
  "id": "integrate/redis-data-integration/observability",
  "title": "Observability",
  "url": "https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/observability/",
  "summary": "Learn how to monitor RDI",
  "content": "\nRDI reports metrics about its operation using\n[Prometheus exporter endpoints](https://prometheus.io/docs/instrumenting/exporters/).\nYou can connect to the endpoints with\n[Prometheus](https://prometheus.io/docs/prometheus/latest/getting_started/)\nto query the metrics and plot simple graphs or with\n[Grafana](https://grafana.com/) to produce more complex visualizations and\ndashboards.\n\nRDI exposes the following endpoints:\n- **Collector metrics**: CDC collector performance and connectivity\n- **Stream processor metrics**: Data processing performance and throughput. The exposed metrics depend on the [stream processor implementation](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/architecture#stream-processor-implementations) used by the pipeline:\n  - The classic processor exposes the metrics described in [Stream processor metrics](#stream-processor-metrics) through the `rdi-metrics-exporter` service.\n  - The Flink processor exposes the metrics described in [Flink processor metrics](#flink-processor-metrics) directly from its JobManager and TaskManager pods. The `rdi-metrics-exporter` service is not deployed for Flink-based pipelines.\n- **Operator metrics**: Kubernetes operator health and Pipeline resource states\n\nThe sections below explain these sets of metrics in more detail.\nSee the\n[architecture overview](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/architecture#overview)\nfor an introduction to these concepts.\n\n\u003e [!NOTE]\n\u003e If you don't use Prometheus or Grafana, you can still see\n\u003e RDI metrics with the RDI monitoring screen in Redis Insight or with the\n\u003e [`redis-di describe`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-describe)\n\u003e command from the CLI.\n\n## Accessing the metrics\n\nThe way you access the metrics endpoints depends on whether you are using a VM installation or a Helm installation for RDI. The sections below describe the correct approach for each installation type.\n\n### VM Installation\n\nFor VM installations, the metrics are available by default on the following endpoints:\n- Collector metrics: `https://\u003cRDI_HOST\u003e/collector-source/metrics`\n- Stream processor metrics: `https://\u003cRDI_HOST\u003e/processor/metrics`\n- Operator metrics: `https://\u003cRDI_HOST\u003e/operator/metrics`\n\nPlease note that for RDI versions prior to 1.16.0 the collector metrics are not accessible.\n\n### Helm installation\n\nFor Helm installations, the metrics are available via autodiscovery in the K8s cluster. Follow the steps below to use them:\n1. Make sure you have the Prometheus Operator installed in your K8s cluster (see the\n   [Prometheus Operator installation guide](https://prometheus-operator.dev/docs/getting-started/installation/) for more information about this).\n\n2. Update your values.yaml file to enable metrics for the operator, collector and stream processor components.\n\n    - For the collector, update the `collector` section, under the `dataPlane` section:\n        ```yaml\n        dataPlane:\n          collector:\n            # Enable service monitor\n            serviceMonitor:\n              enabled: true\n\n              # Make sure to label the ServiceMonitor so that Prometheus can discover it\n              labels:\n                release: prometheus\n        ```\n\n    - For the stream processor, update the `rdiMetricsExporter` section:\n        ```yaml\n        rdiMetricsExporter:\n          # Enable service monitor\n          serviceMonitor:\n            enabled: true\n\n            # Make sure to label the ServiceMonitor so that Prometheus can discover it\n            labels:\n              release: prometheus\n        ```\n\n    - For the operator, update the `operator` section:\n        ```yaml\n        operator:\n          prometheus:\n            enabled: true\n            labels:\n              release: prometheus\n          metrics:\n            enabled: true\n        ```\n\n    - For the Flink processor, enable the JobManager and TaskManager `ServiceMonitor` resources under `operator.dataPlane.flinkProcessor`:\n        ```yaml\n        operator:\n          dataPlane:\n            flinkProcessor:\n              jobManager:\n                serviceMonitor:\n                  enabled: true\n                  labels:\n                    release: prometheus\n              taskManager:\n                serviceMonitor:\n                  enabled: true\n                  labels:\n                    release: prometheus\n        ```\n\n\u003e [!NOTE]\n\u003e The Prometheus service discovery loop runs at regular intervals. This means that after deploying or updating RDI with the above configuration, it may take a few minutes for Prometheus to discover the new ServiceMonitors and start scraping metrics from the RDI components.\n\n## Collector metrics\n\nThese metrics are divided into three groups:\n\n- **Pipeline state**: metrics about the pipeline mode and connectivity\n- **Data flow counters**: counters for data breakdown per source table\n- **Processing performance**: processing speed of RDI micro batches\n\nThe following table lists all collector metrics and their descriptions:\n\n| Metric | Type | Description | Alerting Recommendations |\n|:--|:--|:--|:--|\n| **Schema History Metrics** | | | |\n| `ChangesApplied` | Counter | Total number of schema changes applied during recovery and runtime | Informational - monitor for trends |\n| `ChangesRecovered` | Counter | Number of changes that were read during the recovery phase | Informational - monitor for trends |\n| `MilliSecondsSinceLastAppliedChange` | Gauge | Number of milliseconds since the last change was applied | Informational - monitor for trends |\n| `MilliSecondsSinceLastRecoveredChange` | Gauge | Number of milliseconds since the last change was recovered from the history store | Informational - monitor for trends |\n| `RecoveryStartTime` | Gauge | Time in epoch milliseconds when recovery started (-1 if not applicable) | Informational - monitor for trends |\n| **Connection and State Metrics** | | | |\n| `Connected` | Gauge | Whether the collector is currently connected to the database (1=connected, 0=disconnected) | **Critical Alert**: Alert if value = 0 (disconnected) |\n| **Queue Metrics** | | | |\n| `CurrentQueueSizeInBytes` | Gauge | Current size of the collector's internal queue in bytes | Informational - monitor for trends |\n| `MaxQueueSizeInBytes` | Gauge | Maximum configured size of the collector's internal queue in bytes | Informational - use for capacity planning |\n| `QueueRemainingCapacity` | Gauge | Remaining capacity of the collector's internal queue | Informational - monitor for trends |\n| `QueueTotalCapacity` | Gauge | Total capacity of the collector's internal queue | Informational - use for capacity planning |\n| **Streaming Performance Metrics** | | | |\n| `MilliSecondsBehindSource` | Gauge | Number of milliseconds the collector is behind the source database (-1 if not applicable) | Informational - monitor for trends and business SLA requirements |\n| `MilliSecondsSinceLastEvent` | Gauge | Number of milliseconds since the collector processed the most recent event (-1 if not applicable) | Informational - monitor for trends in active systems |\n| `NumberOfCommittedTransactions` | Counter | Number of committed transactions processed by the collector | Informational - monitor for trends |\n| `NumberOfEventsFiltered` | Counter | Number of events filtered by include/exclude list rules | Informational - monitor for trends |\n| **Event Counters** | | | |\n| `TotalNumberOfCreateEventsSeen` | Counter | Total number of CREATE (INSERT) events seen by the collector | Informational - monitor for trends |\n| `TotalNumberOfDeleteEventsSeen` | Counter | Total number of DELETE events seen by the collector | Informational - monitor for trends |\n| `TotalNumberOfEventsSeen` | Counter | Total number of events seen by the collector | Informational - monitor for trends |\n| `TotalNumberOfUpdateEventsSeen` | Counter | Total number of UPDATE events seen by the collector | Informational - monitor for trends |\n| `NumberOfErroneousEvents` | Counter | Number of events that caused errors during processing | **Critical Alert**: Alert if \u003e 0 (indicates processing failures) |\n| **Snapshot Metrics** | | | |\n| `RemainingTableCount` | Gauge | Number of tables remaining to be processed during snapshot | Informational - monitor snapshot progress |\n| `RowsScanned` | Counter | Number of rows scanned per table during snapshot (reported per table) | Informational - monitor snapshot progress |\n| `SnapshotAborted` | Gauge | Whether the snapshot was aborted (1=aborted, 0=not aborted) | **Critical Alert**: Alert if value = 1 (snapshot failed) |\n| `SnapshotCompleted` | Gauge | Whether the snapshot completed successfully (1=completed, 0=not completed) | Informational - monitor snapshot completion |\n| `SnapshotDurationInSeconds` | Gauge | Total duration of the snapshot process in seconds | Informational - monitor for performance trends |\n| `SnapshotPaused` | Gauge | Whether the snapshot is currently paused (1=paused, 0=not paused) | Informational - monitor snapshot state |\n| `SnapshotPausedDurationInSeconds` | Gauge | Total time the snapshot was paused in seconds | Informational - monitor snapshot state |\n| `SnapshotRunning` | Gauge | Whether a snapshot is currently running (1=running, 0=not running) | Informational - monitor snapshot state |\n| `TotalTableCount` | Gauge | Total number of tables included in the snapshot | Informational - use for progress calculation |\n\n\u003e [!NOTE]\n\u003e Many metrics include context labels that specify the phase (`snapshot` or `streaming`), database name, and other contextual information. Metrics with a value of `-1` typically indicate that the measurement is not applicable in the current state.\n\n## Stream processor metrics\n\nThe metrics in this section are reported by the *classic* stream processor and\nexposed through the `rdi-metrics-exporter` service. For pipelines that use\nthe [Flink processor](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/architecture#stream-processor-implementations),\nsee [Flink processor metrics](#flink-processor-metrics) instead.\n\nRDI reports metrics during the two main phases of the ingest pipeline, the *snapshot*\nphase and the *change data capture (CDC)* phase. (See the\n[pipeline lifecycle](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/data-pipelines)\ndocs for more information). The table below shows the full set of metrics that\nRDI reports with their descriptions.\n\n| Metric Name | Metric Type | Metric Description | Alerting Recommendations |\n|-------------|-------------|--------------------|-----------------------|\n| `incoming_records_total` | Counter | Total number of incoming records processed by the system | Informational - monitor for trends |\n| `incoming_records_created` | Gauge | Timestamp when the incoming records counter was created | Informational - no alerting needed |\n| `processed_records_total` | Counter | Total number of records that have been successfully processed | Informational - monitor for trends |\n| `rejected_records_total` | Counter | Total number of records that were rejected during processing | **Critical Alert**: Alert if \u003e 0 (indicates processing failures) |\n| `filtered_records_total` | Counter | Total number of records that were filtered out during processing | Informational - monitor for trends |\n| `rdi_engine_state` | Gauge | Current state of the RDI engine with labels for `state` (e.g., STARTED, RUNNING) and `sync_mode` (e.g., SNAPSHOT, STREAMING) | **Critical Alert**: Alert if state indicates failure or error condition |\n| `rdi_version_info` | Gauge | Version information for RDI components with labels for `cli` and `engine` versions | Informational - use for version tracking |\n| `monitor_time_elapsed_total` | Counter | Total time elapsed (in seconds) since monitoring started | Informational - use for uptime tracking |\n| `monitor_time_elapsed_created` | Gauge | Timestamp when the monitor time elapsed counter was created | Informational - no alerting needed |\n| `rdi_incoming_entries` | Gauge | Count of incoming events by `data_source` and `operation` type (pending, inserted, updated, deleted, filtered, rejected) | Informational - monitor for trends, alert only on \"rejected\" \u003e 0 |\n| `rdi_stream_event_latency_ms` | Gauge | Latency in milliseconds of the oldest event in each data stream, labeled by `data_source` | Informational - monitor based on business SLA requirements |\n| **Processor Performance Total Metrics** | | | |\n| `rdi_processed_batches_total` | Counter | Total number of processed batches | Informational - use for data ingestion and load tracking |\n| `rdi_processor_batch_size_total` | Counter | Total batch size across all processed batches | Informational - use for throughput analysis |\n| `rdi_processor_read_time_ms_total` | Counter | Total read time in milliseconds across all batches | Informational - use for performance analysis |\n| `rdi_processor_transform_time_ms_total` | Counter | Total transform time in milliseconds across all batches | Informational - use for performance analysis |\n| `rdi_processor_write_time_ms_total` | Counter | Total write time in milliseconds across all batches | Informational - use for performance analysis |\n| `rdi_processor_process_time_ms_total` | Counter | Total process time in milliseconds across all batches | Informational - use for performance analysis |\n| `rdi_processor_ack_time_ms_total` | Counter | Total acknowledgment time in milliseconds across all batches | Informational - use for performance analysis |\n| `rdi_processor_total_time_ms_total` | Counter | Sum of the total `read_time`, `process_time` and `ack_time` values in milliseconds across all batches | Informational - use for performance analysis |\n| `rdi_processor_rec_per_sec_total` | Gauge | Total records per second across all batches | Informational - use for throughput analysis |\n| **Processor Performance Last Batch Metrics** | | | |\n| `rdi_processor_batch_size_last` | Gauge | Last batch size processed | Informational - use for real-time monitoring |\n| `rdi_processor_read_time_ms_last` | Gauge | Last batch read time in milliseconds | Informational - use for real-time performance monitoring |\n| `rdi_processor_transform_time_ms_last` | Gauge | Last batch transform time in milliseconds | Informational - use for real-time performance monitoring |\n| `rdi_processor_write_time_ms_last` | Gauge | Last batch write time in milliseconds | Informational - use for real-time performance monitoring |\n| `rdi_processor_process_time_ms_last` | Gauge | Last batch process time in milliseconds | Informational - use for real-time performance monitoring |\n| `rdi_processor_ack_time_ms_last` | Gauge | Last batch acknowledgment time in milliseconds | Informational - use for real-time performance monitoring |\n| `rdi_processor_total_time_ms_last` | Gauge | Last batch total time in milliseconds | Informational - use for real-time performance monitoring |\n| `rdi_processor_rec_per_sec_last` | Gauge | Last batch records per second | Informational - use for real-time throughput monitoring |\n\n\u003e [!NOTE]\n\u003e **Additional information about stream processor metrics:**\n\u003e\n\u003e - Where the metric name has the `rdi_` prefix, this will be replaced by the Kubernetes namespace name if you supplied a custom name during installation. The prefix is always `rdi_` for VM installations.\n\u003e - Metrics with the `_created` suffix are automatically generated by Prometheus for counters and gauges to track when they were first created.\n\u003e - The `rdi_incoming_entries` metric provides a detailed breakdown for each data source by operation type.\n\u003e - The `rdi_stream_event_latency_ms` metric helps monitor data freshness and processing delays.\n\u003e - The processor performance metrics are divided into two categories:\n\u003e   - **Total metrics**: Accumulate values across all processed batches for historical analysis\n\u003e   - **Last batch metrics**: Show real-time performance data for the most recently processed batch\n\n## Flink processor metrics\n\nThe Flink processor exposes Prometheus metrics directly from its JobManager\nand TaskManager pods. The `rdi-metrics-exporter` service is not deployed for\nFlink-based pipelines, and the metrics described in\n[Stream processor metrics](#stream-processor-metrics) are not available.\n\nThe full set of metrics returned by the Flink processor is large and includes\nevery metric emitted by the underlying Flink runtime (job, task, operator,\nJVM, network, and connector metrics). See the\n[Flink metrics documentation](https://nightlies.apache.org/flink/flink-docs-release-2.0/docs/ops/metrics/)\nfor the full reference of Flink-emitted metrics, and the\n[Flink Prometheus reporter](https://nightlies.apache.org/flink/flink-docs-release-2.0/docs/deployment/metric_reporters/#prometheus)\ndocs for the naming scheme.\n\nConfigure Prometheus to scrape these metrics by enabling the JobManager and\nTaskManager `ServiceMonitor` resources under `operator.dataPlane.flinkProcessor`,\nas shown in [Helm installation](#helm-installation) above.\n\n### Useful metrics\n\nIn addition to the standard Flink metrics, the Flink processor emits a small\nset of RDI-specific metrics that cover record counters, source/target\nconnectivity, and stream backlog. These metrics, together with a curated\nsubset of native Flink metrics, are surfaced through the\n[RDI API v2 metric collections endpoint](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/api-reference)\nand are the recommended starting point for dashboards and alerts.\n\n**RDI-emitted metrics** (per pipeline):\n\n| Metric | Description |\n|---|---|\n| `flink_jobmanager_job_operator_coordinator_stream_type_rdiRecords` | Per-stream record counters. Labels: `stream`, `type` (one of `incoming`, `inserted`, `updated`, `deleted`, `filtered`, `rejected`). |\n| `flink_jobmanager_job_operator_coordinator_enumerator_stream_type_rdiRecords` | Per-stream backlog and freshness. Labels: `stream`, `type` (`pending` for stream length, `lastArrival` for the epoch-millisecond timestamp of the last entry). |\n| `flink_taskmanager_job_task_operator_rdi_connected` | Source or target connection status (`1` = connected, `0` = disconnected). Filter by `operator_name` equal to `Source:_source` for the source and matching the regex `.*:target:_Writer$` for target writers; treat the source or target as connected if any subtask reports `1`. |\n| `flink_taskmanager_job_task_operator_rdi_lastModified` | Epoch-millisecond timestamp of the last successful write to the target Redis database. Filter by `operator_name` matching `.*:target:_Writer$` and take the maximum across subtasks. |\n| `flink_taskmanager_job_task_operator_pendingAck` | Number of records emitted by the source but awaiting checkpoint completion before being acknowledged. Sum across subtasks. |\n\n**Native Flink metrics** used by the API:\n\n| Metric | Description |\n|---|---|\n| `flink_taskmanager_job_task_operator_numRecordsInPerSecond` | Per-operator throughput. For source throughput, filter by `operator_name` equal to `Source:_source` and sum across subtasks. For sink throughput, filter by `operator_name` matching `.*:target:_Writer$` and sum across subtasks and across all target writers. |\n| `flink_taskmanager_job_task_busyTimeMsPerSecond` | Time the task spends actively processing records (ms/s). Average across subtasks of the main chained task; exclude the `dlq:_Writer` task. |\n| `flink_taskmanager_job_task_idleTimeMsPerSecond` | Time the task spends waiting for input (ms/s). Average across subtasks of the main chained task; exclude the `dlq:_Writer` task. |\n| `flink_taskmanager_job_task_backPressuredTimeMsPerSecond` | Time the task spends back-pressured because the downstream cannot keep up (ms/s). Average across subtasks of the main chained task; exclude the `dlq:_Writer` task. |\n| `flink_jobmanager_job_lastCheckpointDuration` | Duration of the most recent checkpoint (ms). |\n| `flink_jobmanager_job_lastCheckpointSize` | Persisted size of the most recent checkpoint (bytes). |\n| `flink_jobmanager_job_numberOfCompletedCheckpoints` | Total number of completed checkpoints. |\n| `flink_jobmanager_job_numberOfFailedCheckpoints` | Total number of failed checkpoints. |\n| `flink_jobmanager_job_\u003cstatus\u003eTime` | Time spent in each job state (ms), where `\u003cstatus\u003e` is one of `running`, `restarting`, `failing`, `cancelling`, `initializing`, `created`, or `deploying`. The metric for the current state is non-zero; all others are zero. Use this to derive both the current job status and the time spent in it. |\n| `flink_jobmanager_job_numRestarts` | Total number of job restarts since submission. |\n\n\u003e [!NOTE]\n\u003e Flink runtime metric names follow Flink's own naming scheme rather\n\u003e than the `rdi_` prefix used by the classic processor. When you build\n\u003e dashboards that should work for both processors, query the two metric sets\n\u003e separately.\n\n## Operator metrics\n\nThe RDI operator exposes Prometheus metrics at the `/metrics` endpoint to monitor the health and state of the operator itself and the Pipeline resources it manages.\n\nThe endpoint for operator metrics is `https://\u003cRDI_HOST\u003e/operator/metrics` (or the operator service endpoint in Kubernetes environments).\n\n### Operator metric types\n\nMost of the metrics exposed by the RDI operator are standard controller-runtime [metrics](https://book.kubebuilder.io/reference/metrics-reference).\nThe metrics that are relevant for RDI operations are listed in the table below:\n\n| Metric Name | Metric Type | Metric Description | Alerting Recommendations |\n|-------------|-------------|-------------------|-------------------------|\n| `rdi_operator_pipeline_phase` | Gauge | Current phase of each Pipeline resource with labels for `namespace`, `name`, and `phase` (Active, Inactive, Pending, Resetting, Error) | **Critical Alert**: Alert if the phase is \"Error\" for periods longer than 2 minutes |\n| `rdi_operator_is_leader` | Gauge | Leadership status of the operator instance (1 = leader, 0 = not leader) with label for `instance_id` | Informational - monitor to ensure that the correct RDI instance is the leader in HA or DR deployments |\n\n### Understanding operator metrics\n\n**Pipeline phase tracking**: The `rdi_operator_pipeline_phase` metric helps you monitor the lifecycle state of each RDI Pipeline resource. Each pipeline reports its current phase (Active, Inactive, Pending, Resetting, or Error) as a gauge value of `1`, while all other phases for that pipeline are set to `0`. This allows you to track phase transitions and identify pipelines that are stuck in error states.\n\n**Leader election**: In high availability (HA) or disaster recovery (DR) deployments with multiple RDI instances, the `rdi_operator_is_leader` metric indicates which RDI instance is actively managing Pipeline resources. Only one RDI instance should have a value of `1` at any time, while all other instances should report `0`. This metric is useful for troubleshooting leader election issues in HA or DR deployments.\n\n### Accessing operator metrics\n\nIn Kubernetes deployments, you can configure Prometheus to scrape operator metrics by enabling the Prometheus ServiceMonitor in your Helm values:\n\n```yaml\noperator:\n  prometheus:\n    enabled: true\n    labels:\n      release: prometheus\n```\n**Note:** The ServiceMonitor resources must be labelled correctly for metrics to be auto-scraped by Prometheus. The correct label is configured in Prometheus, by default it is `release: prometheus`.\nYou can also expose the metrics endpoint externally using an Ingress:\n\n```yaml\noperator:\n  ingress:\n    enabled: true\n    hosts:\n      - operator.example.com\n    pathPrefix: \"\"\n```\n\nThen access metrics at `https://operator.example.com/operator/metrics`.\n\n## Recommended alerting strategy\n\nThe alerting strategy described in the sections below focuses on system failures and data integrity issues that require immediate attention. Most other metrics are informational, so you should monitor them for trends rather than trigger alerts.\n\n### Critical alerts (immediate response required)\n\nThese are the only alerts that require immediate action:\n\n**Collector alerts:**\n- `Connected = 0`: Database connectivity has been lost. RDI cannot function without a database connection.\n- `NumberOfErroneousEvents \u003e 0`: Errors are occurring during data processing. This indicates data corruption or processing failures.\n- `SnapshotAborted = 1`: The snapshot process has failed, so the initial sync is incomplete.\n\n**Processor alerts:**\n- `rejected_records_total \u003e 0`: Records are being rejected. This indicates data quality issues or processing failures.\n- `rdi_engine_state`: Alert only if the state indicates a clear failure condition (not just \"not running\").\n\n**Operator alerts:**\n- `rdi_operator_pipeline_phase` with `phase=\"Error\"` for more than 2 minutes: A Pipeline resource has entered an error state and requires investigation.\n- No leader in HA or DR setups: If both RDI instances report `rdi_operator_is_leader = 0` for more than 2 minutes, the RDI pipeline is not active. \n- Multiple leaders in HA or DR setups: If both RDI instances report `rdi_operator_is_leader = 1`, RDI is in a \"split brain\" state.\n\n### Important monitoring (but not alerts)\n\nYou should monitor these metrics on dashboards and review them regularly, but they don't require automated alerts:\n\n- **Queue metrics**: Queue utilization can vary widely and hitting 0% or 100% capacity may be normal during certain operations.\n- **Latency metrics**: Lag and processing times depend heavily on business requirements and normal operational patterns.\n- **Event counters**: Event rates naturally vary based on application usage patterns.\n- **Snapshot progress**: Snapshot duration and progress depend on data size, so you should typically monitor them manually.\n- **Schema changes**: Schema change frequency is highly application-dependent.\n\n### Key principles for RDI alerting\n\n- **Alert on failures, not performance**: Focus alerts on system failures rather than performance degradation.\n- **Business context matters**: Latency and throughput requirements vary significantly between organizations.\n- **Establish baselines first**: Monitor metrics for weeks before you set any threshold-based alerts.\n- **Avoid alert fatigue**: If you see too many non-critical alerts, you are less likely to take truly critical issues seriously.\n- **Use dashboards for trends**: Most metrics are better suited for dashboard monitoring than alerting\n\n### Monitoring best practices\n\n- **Dashboard-first approach**: Use Grafana dashboards to visualize trends and patterns.\n- **Baseline establishment**: Monitor your specific workload for 2-4 weeks before you consider adding more alerts.\n- **Business SLA alignment**: Only create alerts for metrics that directly impact your business SLA requirements.\n- **Manual review**: Don't use automated alerts to review metric trends. Instead, schedule regular business reviews to check them manually.\n\n## RDI logs\n\nRDI uses [fluentd](https://www.fluentd.org/) and\n[logrotate](https://linux.die.net/man/8/logrotate) to ship and rotate logs\nfor its Kubernetes (K8s) components.\nSo whenever a containerized component is removed by the RDI operator process or by K8s,\nthe logs are available for you to inspect.\nBy default, RDI stores logs in the host VM file system at `/opt/rdi/logs`.\nThe logs are recorded at the minimum `INFO` level and get rotated when they reach a size of 100MB.\nRDI retains the last five log rotated files by default.\nLogs are in a straightforward text format, which lets you analyze them with several different observability tools.\nYou can change the default log settings using the\n[`redis-di configure-rdi`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-configure-rdi)\ncommand.\n\n## Dump support package\n\nIf you ever need to send a comprehensive set of forensics data to Redis support then you should\nrun the\n[`redis-di dump-support-package`](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/reference/cli/redis-di-dump-support-package)\ncommand from the CLI. See\n[Troubleshooting](https://redis.io/docs/latest/integrate/redis-data-integration/1.19.1/troubleshooting#dump-support-package)\nfor more information.\n",
  "tags": ["docs","integrate","rs","rdi"],
  "last_updated": "2026-09-19T17:55:58-07:00"
}