Monitoring guide

Monitor RDI engine and data processing jobs

RDI engine accumulates operating statistics that you can:

• Observe and analyze to discover various types of problems.
• Use for optimization purposes.

Console metrics

RDI can display its operating metrics in the console using the redis-di status command. The command provides information about the current RDI engine status, target database configuration, and processing statistics broken down by stream. This tool is intended to be used by Operator to get the current snapshot of the system as well as monitoring ongoing data processing (when used in live mode).

Prometheus integration

RDI allows collecting and exporting its metrics to Prometheus and visualizing them in Grafana. Operator can start the built-in exporter using the redis-di monitor command. The diagram describes this flow and the components involved:

Note: The host names and ports above are examples only and can be changed as needed.

Test RDI metrics exporter

Start the RDI metrics exporter using the command below:

redis-di monitor

Note: The default port for the exporter is 9121. If you need to change it, use the --exporter-port option. The default metrics collection interval is 5 seconds. If you need to change it, use the --collect-interval option.

Then navigate to http://localhost:9121/ to see the exported metrics. You should be able to see the following metric:

rdi_engine_state{state="RUNNING",sync_mode="UNKNOWN"} 1.0

Note: The actual value of the metric above can be 0 if you haven't started RDI engine yet (in which case, the state label should indicate that as well). You must have the RDI database created and configured before observing any metrics. If you are not seeing it or getting an error value instead, this indicates that the RDI database is not properly configured.

Configure Prometheus

Next, configure the Prometheus scraper. Edit the prometheus.yml file to add the following scraper config:

scrape_configs:
  # scrape RDI metrics exporter
  - job_name: rdi-exporter
    static_configs:
      - targets: ["redis-exporter:9121"]

Notes:

• Make sure the targets value above points to the host and port you configured to run the RDI metrics exporter.
• The scrape_interval setting in Prometheus should be the same or higher than the collect_interval setting for the exporter. For example, if the collect_interval is set to 5 seconds, the scrape_interval should also be set to 5 seconds or more. If the scrape_interval is set to less than the collect_interval, Prometheus will scrape the exporter before it has a chance to collect and refresh metrics, and you will see duplicated values in Prometheus.

Test the Prometheus scraper

After the scraper config is added to the Prometheus configuration, you should now be able to navigate to http://<HOSTNAME>:9090/graph (replace <HOSTNAME> with a valid Prometheus hostname or IP address).

Explore RDI metrics using the expression browser.

In the expression box, type in a metric name (for example, rdi_engine_state) and select Enter or the Execute button to see the following result:

rdi_engine_state{instance="redis-exporter:9121", job="rdi-exporter", status="RUNNING", sync_mode="UNKNOWN"} 1

Note: You may see more than just one RDI metric if RDI engine has already processed some data. If you do not see any metrics, check your scraper job configuration in Prometheus.

Use Grafana to analyze metrics

Optionally, you may deploy the sample Grafana dashboard to monitor the status of RDI engine and its individual jobs. Follow these steps to import the dashboard:

1. Download the dashboard file to your local machine.

2. Log into Grafana and navigate to the list of dashboards, then choose New > Import:

1. On the next screen, select Upload JSON file and upload the file you downloaded in step 1. Make sure you select the data source that is connected to the RDI metrics exporter:

1. Select Import and make sure you choose the jobs to monitor from the drop-down list (this will be empty if you don't have any jobs running yet):

RDI metrics

This list shows exported RDI metrics along with their descriptions: