{
  "schema_version": 2,
  "id": "develop/ai/redisvl/user_guide/how_to_guides/observability",
  "title": "Observe the Semantic Cache with OpenTelemetry",
  "url": "https://redis.io/docs/latest/develop/ai/redisvl/0.32.0/user_guide/how_to_guides/observability/",
  "summary": "",
  "content": "\n\nRedisVL can report what `SemanticCache` does through [OpenTelemetry](https://opentelemetry.io/): a trace for every lookup that separates embedding time from search time, a hit rate, and the distance of the nearest match on every lookup, hits and misses alike. That last signal is what you need to tune `distance_threshold`.\n\nRedisVL depends only on the OpenTelemetry API. Your application owns the SDK and the exporters, and RedisVL emits nothing until you configure them. With no SDK configured, the instrumentation adds about 11 µs to a lookup that embeds, measured on an Apple M4 Pro.\n\nSee [Cache LLM Responses](https://redis.io/docs/latest/../llmcache) for how to use the cache itself.\n\n## Install\n\nInstall RedisVL with the `otel` extra, plus the SDK and an exporter of your choice:\n\n```bash\npip install \"redisvl[otel]\" \"opentelemetry-sdk\u003e=1.32\" opentelemetry-exporter-otlp-proto-http\n```\n\nUse `opentelemetry-sdk` 1.32.0 or later. RedisVL’s histograms carry bucket advice, and earlier SDK releases drop it on OTLP export (fixed in 1.31.0) or apply only the first histogram’s advice (fixed in 1.32.0).\n\n## Configure an SDK\n\nSet the global tracer and meter providers once, at application start-up. RedisVL picks them up automatically, whether you create the cache before or after; there is nothing RedisVL-specific to call.\n\n```python\nfrom opentelemetry import metrics, trace\nfrom opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter\nfrom opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter\nfrom opentelemetry.sdk.metrics import MeterProvider\nfrom opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader\nfrom opentelemetry.sdk.trace import TracerProvider\nfrom opentelemetry.sdk.trace.export import BatchSpanProcessor\n\nfrom redisvl.extensions.cache.llm import SemanticCache\n\ntracer_provider = TracerProvider()\ntracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))\ntrace.set_tracer_provider(tracer_provider)\n\nmetrics.set_meter_provider(\n    MeterProvider(metric_readers=[PeriodicExportingMetricReader(OTLPMetricExporter())])\n)\n\ncache = SemanticCache(name=\"llmcache\", redis_url=\"redis://localhost:6379\")\ncache.store(prompt=\"What is the capital of France?\", response=\"Paris\")\ncache.check(prompt=\"What's the capital of France?\")\n```\n\nThe OTLP exporters send to `http://localhost:4318` unless you set `OTEL_EXPORTER_OTLP_ENDPOINT`, so run an OpenTelemetry Collector there, or point them at a backend that accepts OTLP. With nothing listening, the exporters log connection errors and retry, which can delay shutdown, but your application keeps working.\n\nAll RedisVL telemetry uses the instrumentation scope `redisvl`, with RedisVL’s version and the semantic-conventions schema URL it targets.\n\n## What RedisVL emits\n\n### Spans\n\nEach `check`, `store`, `update` and `drop` call produces a span tree. A lookup looks like this:\n\n```default\nsemantic_cache.check     INTERNAL\n├── embeddings {model}   CLIENT\n└── retrieval {index}    CLIENT\n```\n\n`store`, `update` and `drop` produce `semantic_cache.store`, `semantic_cache.update` and `semantic_cache.drop`, and async methods produce the same trees as their sync forms. There is an embeddings span only when the operation embeds a prompt: a `check` or `store` given a precomputed `vector` has none, and `update` and `drop` never embed. The embeddings span comes from the vectorizer, so `SemanticRouter`, `SemanticMessageHistory` and direct calls to `embed` and `embed_many` produce it too.\n\n| Span                 | Attribute                                     | Value                                                                                                                                                              |\n|----------------------|-----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `semantic_cache.*`   | `redisvl.cache.name`                          | The cache name                                                                                                                                                     |\n|                      | `redisvl.cache.result`                        | `hit` or `miss`, on `check` only                                                                                                                                   |\n| `embeddings {model}` | `gen_ai.operation.name`                       | `embeddings`                                                                                                                                                       |\n|                      | `gen_ai.provider.name`                        | For example `openai`, `azure.ai.openai`, `cohere`, `mistral_ai`, `gcp.vertex_ai`, `gcp.gemini` or `aws.bedrock`; other vectorizers report their type, such as `hf` |\n|                      | `gen_ai.request.model`                        | The vectorizer’s model                                                                                                                                             |\n|                      | `gen_ai.embeddings.dimension.count`           | The vectorizer’s dimensions, when known                                                                                                                            |\n|                      | `redisvl.embeddings_cache.result`             | `hit`, `miss`, or `partial` for a batch; only when the vectorizer has an `EmbeddingsCache` and the call doesn’t pass `skip_cache=True`                             |\n| `retrieval {index}`  | `gen_ai.operation.name`                       | `retrieval`                                                                                                                                                        |\n|                      | `gen_ai.data_source.id`, `db.collection.name` | The index name                                                                                                                                                     |\n|                      | `gen_ai.retrieval.top_k`                      | `num_results`                                                                                                                                                      |\n|                      | `db.system.name`                              | `redis`                                                                                                                                                            |\n|                      | `db.operation.name`                           | `FT.SEARCH`                                                                                                                                                        |\n|                      | `redisvl.cache.distance_threshold`            | The threshold the lookup used                                                                                                                                      |\n|                      | `redisvl.cache.distance`                      | The nearest candidate’s distance, when there was one                                                                                                               |\n\nA failed operation sets an error status and `error.type` on every RedisVL span it passes through. `error.type` is the exception’s class, prefixed by its module unless it is a builtin, such as `redis.exceptions.ConnectionError`.\n\n### Metrics\n\n| Metric                                      | Unit   | Attributes                                                                                                      | Recorded on                                                                                                                                        |\n|---------------------------------------------|--------|-----------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------|\n| `redisvl.semantic_cache.operation.duration` | `s`    | `redisvl.cache.name`, `redisvl.cache.operation`, `redisvl.cache.result` (on `check`), `error.type` (on failure) | Every `check`, `store`, `update` and `drop`, sync or async                                                                                         |\n| `redisvl.semantic_cache.distance`           | `1`    | `redisvl.cache.name`, `redisvl.cache.result`                                                                    | Every completed lookup that found at least one candidate                                                                                           |\n| `gen_ai.client.operation.duration`          | `s`    | `gen_ai.operation.name`, `gen_ai.provider.name`, `gen_ai.request.model`, `error.type` (on failure)              | Every embedding call that reaches the provider, but not an embeddings-cache hit. One `embed_many` call records one point, covering all its batches |\n\nThere is no counter: the duration histogram’s count is the number of operations. Cache-operation buckets start at 100 µs, because a lookup against local Redis takes about 250 µs before embedding, while embedding-duration buckets follow the GenAI conventions’ advice, from 10 ms. Distance buckets are 0.01 wide up to 0.1, then coarser up to 2, the maximum cosine distance. To change any of them, use an SDK [View](https://opentelemetry.io/docs/languages/python/instrumentation/#metrics).\n\nNames in the `gen_ai.*` namespace follow the OpenTelemetry GenAI conventions, which are still in Development status and may change in a future release.\n\n## Tune the distance threshold\n\nThe distance histogram, split by result, shows how close your lookups come to the threshold, which defaults to 0.1:\n\n| Pattern                                 | What it suggests                                  | What to do                                                          |\n|-----------------------------------------|---------------------------------------------------|---------------------------------------------------------------------|\n| Misses bunched just above the threshold | Some may be near-duplicates the cache is refusing | Consider loosening the threshold to turn the genuine ones into hits |\n| Hits bunched just below the threshold   | These are the matches most likely to be wrong     | Consider tightening the threshold, trading hit rate for accuracy    |\n| Misses spread far above the threshold   | Genuinely new prompts                             | Nothing: no threshold would serve them                              |\n\nThe distance of misses, as a heatmap:\n\n```promql\nsum by (le) (rate(redisvl_semantic_cache_distance_bucket{redisvl_cache_result=\"miss\"}[1h]))\n```\n\nThe share of misses within 0.05 of a 0.1 threshold:\n\n```promql\nsum(rate(redisvl_semantic_cache_distance_bucket{redisvl_cache_result=\"miss\", le=\"0.15\"}[1h]))\n/\nsum(rate(redisvl_semantic_cache_distance_count{redisvl_cache_result=\"miss\"}[1h]))\n```\n\nThe share counts only misses that found a candidate: a lookup whose filter excluded every entry, or that ran against an empty cache, records no distance. The `le` value must be one of the bucket boundaries: a value between them, such as `0.23`, returns nothing. The metric does not record which threshold a lookup used, so after you change it, compare windows from before and after rather than mixing them.\n\nA distance cannot tell you whether a match was right. Before you move the threshold, look at the prompts behind those distances with content capture on, as described under [Privacy]().\n\n## Starter queries\n\nThese PromQL queries use the names the OpenTelemetry Prometheus exporter produces. Other pipelines can translate names differently, so check your backend’s metric names first.\n\nHit rate, per cache:\n\n```promql\n(\n  sum by (redisvl_cache_name) (rate(redisvl_semantic_cache_operation_duration_seconds_count{redisvl_cache_operation=\"check\", redisvl_cache_result=\"hit\"}[5m]))\n  or\n  0 * sum by (redisvl_cache_name) (rate(redisvl_semantic_cache_operation_duration_seconds_count{redisvl_cache_operation=\"check\", redisvl_cache_result=\"miss\"}[5m]))\n)\n/\nsum by (redisvl_cache_name) (rate(redisvl_semantic_cache_operation_duration_seconds_count{redisvl_cache_operation=\"check\", redisvl_cache_result=~\"hit|miss\"}[5m]))\n```\n\nThe `or` term reports 0 for a cache that has never hit, which would otherwise drop out of the result because it has no `hit` series. Failed checks carry `error.type` and no result, so they count towards neither side.\n\n95th-percentile duration, per operation:\n\n```promql\nhistogram_quantile(0.95, sum by (le, redisvl_cache_operation) (rate(redisvl_semantic_cache_operation_duration_seconds_bucket[5m])))\n```\n\n95th-percentile embedding latency, per provider and model:\n\n```promql\nhistogram_quantile(0.95, sum by (le, gen_ai_provider_name, gen_ai_request_model) (rate(gen_ai_client_operation_duration_seconds_bucket{gen_ai_operation_name=\"embeddings\"}[5m])))\n```\n\n## Add redis-py’s own metrics\n\nredis-py 7.4 and later ship their own OpenTelemetry metrics for connections and commands. RedisVL never initialises them, so turn them on yourself after setting the meter provider. Command latency is not in the default groups:\n\n```python\nfrom redis.observability import MetricGroup, OTelConfig, get_observability_instance\n\nget_observability_instance().init(\n    OTelConfig(\n        metric_groups=[\n            MetricGroup.CONNECTION_BASIC,\n            MetricGroup.RESILIENCY,\n            MetricGroup.COMMAND,\n        ]\n    )\n)\n```\n\nredis-py emits metrics only, not spans, which is why RedisVL traces the search itself as the `retrieval` span.\n\nIf you also run a provider’s own GenAI instrumentation, such as one for the OpenAI SDK, both can report the same embedding call: expect a nested client span, and a second set of `gen_ai.client.operation.duration` points under that library’s instrumentation scope.\n\n## Privacy and content capture\n\nBy default RedisVL records no prompt, response, metadata or filter values, in any span, event or metric. It never records exception messages on its spans either, because they can contain prompt text: a failure records only the exception’s class. Cache names and model identifiers are exported, so keep tenant or user ids out of cache names.\n\nTo see what the cache matched, set:\n\n```bash\nexport OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY\n```\n\nWith `SPAN_ONLY` or `SPAN_AND_EVENT`, the `retrieval` span also records:\n\n- `gen_ai.retrieval.query.text`: the prompt, whenever the lookup was passed one, even alongside a vector.\n- `gen_ai.retrieval.documents`: a JSON list of the candidates’ entry ids and scores. Each `score` is the cosine similarity, `1 - distance`, so higher is closer, as the GenAI conventions define a relevance score; the nearest raw distance stays on `redisvl.cache.distance`. Entry ids are unsalted SHA-256 hashes of the stored prompt and filters, so treat them as content too.\n\nRedisVL never records response text, but the variable is process-wide: other GenAI instrumentations that read it may also start recording prompts and responses. Unlike some of them, RedisVL does not treat `true` as a span mode, and needs no stability opt-in. It reads the variable once, on the first cache lookup.\n\nPrompts can contain personal data. Capture content in place rather than copying production traffic elsewhere, send it only to a trace backend approved to hold prompt data, sample to limit the volume, and turn capture off again when you have what you need, which takes a restart.\n\n## Turn it off\n\nTo silence RedisVL’s telemetry while keeping the rest of your application’s, set:\n\n```bash\nexport REDISVL_OTEL_DISABLED=true\n```\n\nOnly `true` (case-insensitive) disables it. RedisVL reads the variable once, the first time it records telemetry, so set it before your application starts.\n",
  "tags": [],
  "last_updated": "2026-11-19T13:17:52-06:00"
}
