LangCache semantic cache calculator
See how much you can save by using semantic caching for LLM apps
Cache hit rate estimator
Query Source
Cost Calculator
Estimated percentage of cache hits
Enter the number of queries per day
Average number of input tokens per query
Average number of output tokens per query
Cost assumptions
• LLM costs: $2.5 input/$10 output per 1M tokens (GPT-4o pricing)
• LangCache service cost: $0.5 input/$0 output per 1M tokens (free in public preview) - illustrative pricing for demonstration, final GA pricing TBD
• Storage: $100/month (may vary depending on size)
Cost details
Annual cost without LangCache
$3,741,250.00
Cost Calculation:
• Annual Queries: 1,000,000/day × 365 = 365,000,000
• LLM Input Cost (GPT-4o): 365,000,000 × 100 × $2.5/1M = $91,250.00
• LLM Output Cost (GPT-4o): 365,000,000 × 1000 × $10/1M = $3,650,000.00
• Total Cost: $91,250.00 + $3,650,000.00 = $3,741,250.00
Annual cost with LangCache
$580,637.50
Cost Calculation:
• LLM Cost: $3,741,250.00 × (1 - 85%) = $561,187.50
• Embedding Cost: 365,000,000 × 100 × $0.5/1M = $18,250.00
• Storage Cost (annual): $1,200.00
• Total Cost: $561,187.50 + $18,250.00 + $1,200.00 = $580,637.50
Annual savings
$3,160,612.50