Prompt Caching at 3 Billion Tokens: What It Saves

Series: Benchmarks — Prompt Caching
Published: 2026-08-23
Dataset: github.com/ClockLobsterLabs/LLM-Cost-Comparison
Author: Clock Lobster Labs

The short answer

If a workload consumes 3 billion input tokens per month — about 100 million per day — and behaves like our small test, prompt caching would save roughly $30 per month on DeepSeek V4 Flash input charges.

If every one of those tokens belonged to a large, stable, cacheable prefix and the cache stayed warm, the theoretical saving would be closer to $124 per month. That is an upper-bound scenario, not what our short experiment demonstrated.

These figures cover input tokens only. Output tokens, provider fees, routing changes, and the cost of generating extra skill context are excluded. Price snapshot: DeepSeek V4 Flash through OpenRouter, $0.05166/M fresh input and $0.010332/M cached input.

What we tested

We sent the same four short tasks to deepseek/deepseek-v4-flash through OpenRouter in two separate sticky sessions:

• Regular condition: a short, stable system instruction.
• Skill condition: the repository's SKILL.md — a long, stable instruction and workflow document — followed by the changing task.

Response caching was disabled so that an identical whole-request replay could not be mistaken for provider prompt caching. We recorded usage.prompt_tokens_details.cached_tokens and kept the output cap small to make this a cache-telemetry experiment rather than a quality benchmark.

The important detail: warm-up

The skill did not hit immediately. Requests one through three warmed the provider cache. On request four, approximately 6,400 of 6,610 prompt tokens were reused — a 96.8% hit for that request. Across all four skill requests, including the warm-up traffic, the reuse rate was 24.2%.

This is why a production estimate should not multiply the advertised cache discount by every token in the workload. Cache lifetime, routing stickiness, prefix stability, and the ratio of warm-up to follow-up requests all matter.

Scaling the result to 3 billion tokens

The arithmetic is straightforward. At 3 billion monthly input tokens:

Fresh-input cost = 3,000 × $0.05166 = $154.98/month

Measured case:
  Cached tokens = 3,000M × 24.2% = 726M
  Fresh tokens  = 3,000M − 726M = 2,274M
  Cost          = (2,274 × $0.05166) + (726 × $0.010332)
                = $124.99/month
  Saving        = $154.98 − $124.99 = $29.99/month

At 100 million tokens per day, that is approximately $1 per day in measured savings. A fully warm 96.8% reuse pattern would save about $4.13 per day.

Where the estimate can go wrong

1. “3 billion tokens” may include output. Our measurement applies only to input tokens. Output pricing is different and caching does not reduce generated output.
2. The skill has a real prompt cost. A stable skill can make follow-up context cheap, but it still adds thousands of tokens to the first request and to any cache miss. For tiny tasks, the skill may cost more than the plain prompt it replaces.
3. Cache hits are not a quality result. This experiment did not test whether the skill improved answers, reduced retries, or completed work with fewer total turns.
4. Provider behavior can change. OpenRouter uses provider routing and sticky sessions to improve prompt-cache reuse, but a provider outage, changed session, expired cache, or altered prefix can reduce the rate.

Conclusion

For a 3-billion-input-token monthly workload, prompt caching is financially meaningful but not transformative at DeepSeek V4 Flash's already-low input price:

• Our measured short-run pattern: approximately $30/month saved on input, before output tokens.
• A mature, mostly warm workflow: up to approximately $124/month saved on input.
• The larger business case is likely operational: fewer repeated context-processing tokens, lower latency, and more predictable agent sessions.

The practical recommendation is to measure cached_tokens in production before promising savings. Keep the skill prefix byte-for-byte stable, use a stable session identifier, and report cold-start and warm-hit traffic separately.

Adding output tokens — and the Max variant

The input-only figures above understate a real work bill. For a deliberately simple planning assumption, suppose the workload uses 300 million visible output tokens per month — a 10:1 input-to-output ratio. That is not a universal workload shape, but it is a reasonable starting point for a prompt-heavy coding and agent workload.

Flash Max needs a second adjustment. The repository's July appraisal measured a 1.34× effort multiplier for DeepSeek V4 Flash 0731. Applying that multiplier to the same 300M visible output tokens produces approximately 402M billed output tokens. Those extra tokens represent reasoning/effort overhead, not 402M additional user-visible words.

For a roughly equivalent operating-tier comparison, I also include GPT-5.6 Luna Medium beside Flash and GPT-5.6 Terra High beside Flash Max. OpenAI positions Luna for cost-sensitive, high-volume workloads and Terra as the intelligence/cost balance. These are sensible alternatives to price, but not proven capability matches: Terra High may outperform Flash Max on some tasks, while Flash Max may be the better value on others.

Flash calculation uses $0.05166/M input, $0.010332/M cached input, and $0.10332/M output from the 2026-08-23 OpenRouter price snapshot. Flash Max uses the repository's 2026-08-13 snapshot: $0.072/M input, $0.003/M cached input, and $0.144/M output. OpenAI's current API pages list Luna at $0.20/M input, $0.02/M cached input, and $1.20/M output; Terra at $2.00/M input, $0.20/M cached input, and $12.00/M output. Prices can differ by gateway and change over time.

Interpretation: with output included, the measured Flash scenario is about $156/month, versus $186/month without caching. The fully warm Flash case is about $62/month. Luna Medium is approximately $919/month measured-cache and $527/month fully warm. Terra High is approximately $10,093/month measured-cache and $6,173/month fully warm. The OpenAI figures are dominated by the assumed reasoning/output burden, so they need direct task-level measurement before procurement.

Luna Medium uses a conservative 1.25× billed-output multiplier, matching the earlier Luna scenario in our model-appraisal work. Terra High uses a provisional 1.50× multiplier because this experiment did not measure its hidden reasoning tokens. If Terra High produces only 300M billed output tokens, subtract $1,800 from each Terra total; if it produces 600M, add $1,800.

Method and sources

Experiment date: 2026-08-23. Model: DeepSeek V4 Flash through OpenRouter. Four tasks per condition, temperature 0, maximum 32 completion tokens, separate sticky sessions, response caching disabled. All four responses reached the small output cap; no answer-quality conclusion is drawn. The 300M output, 1.34× Flash Max, 1.25× Luna Medium, and 1.50× Terra High effort figures are scenario assumptions for the total-cost estimate, not additional measurements from this cache test.

OpenAI model positioning and current token prices: GPT-5.6 Luna and GPT-5.6 Terra.

OpenRouter documents provider prompt caching, sticky routing, and the cached_tokens usage field in its prompt-caching guide. Response caching is a separate exact-request feature documented here.

The measured values are a small proof-of-behavior, not a production forecast. Replace them with your own cache-hit rate, token mix, cache TTL, and current provider price before using the estimate for procurement.