Firstpass v0.2.1
GitHub
DocsGuaranteeBenchmarksCLIGitHub

Docs / How Agents Talk to Models / Prompt caching

Part A · The core pathModule 5 of 13

Prompt caching

Caching attacks the one cost history, tools, and MCP all create: the repeated prefix. Mark a stable prefix with cache_control; the first call pays a one-time write (~1.25× input), every later reuse pays a read (~0.1× input) instead of full price. It cuts input, not output or call count — but on a long, tool-heavy session it takes most of the input bill off the table. It only helps if your prefix is actually stable.

history, tools, and MCP all inflate input tokens — the same long prefix (system prompt, tools, early history) re-read on every call. Prompt caching is the provider's fix. You mark a stable prefix with cache_control; the first call pays a one-time cache write (~1.25× the input rate) to store it, and every later call that reuses that prefix pays a cache read (~0.1× the input rate) instead of the full price. The response's usage splits the input into fresh vs cached so you can see it working:

json
// request: mark the stable prefix cacheable (system + tools + old turns)
"system": [ { "type": "text", "text": "You are…",
             "cache_control": { "type": "ephemeral" } } ]

// response usage on a later turn — most input is now a cheap cache read
"usage": {
  "input_tokens": 210,                    // fresh tokens, full price
  "cache_read_input_tokens": 24000,       // reused prefix, ~0.1× price
  "cache_creation_input_tokens": 0,       // nonzero only on the write
  "output_tokens": 420
}
Two input-cost curves across ten conversation turns. Without caching, the per-turn cost climbs steadily as the growing transcript is re-read at full price. With prompt caching, turn one pays a small write overhead at 1.25× the input rate; every later turn reads the stable prefix at roughly 0.1× — so the with-cache curve stays low and nearly flat while the no-cache curve keeps climbing, creating roughly 90% savings on the re-read portion. Input cost per turn — caching flattens the multi-turn climb $ turns 1 → 10 no cache re-read grows each turn write (1.25×, once) with cache · reads ≈ 0.1× ≈ 90% saved same 10-turn session history re-read ≈ 82 k tok no cache ≈ $0.25 input cached ≈ $0.03 input ≈ 90% off the re-read half illustrative · Anthropic-style ratios
Caching attacks exactly the cost multi-turn creates: the repeated prefix. It doesn't reduce output tokens or call count — but on a long, tool-heavy session it can take most of the input bill off the table. Whether it helps depends on how stable your prefix is.
Common mistake Assuming caching “just works” once you set cache_control. The cache keys on an exact prefix match and entries are short-lived (minutes). Change one byte early in the prompt — a rotating timestamp in the system prompt, a reordered tools array, a per-request id — and every downstream token becomes a fresh full-price read again. Caching rewards a stable prefix; a prompt that mutates near the top quietly pays full price while looking cached.
Prompt caching re-prices the repeated prefix from full rate to ~0.1×, flattening multi-turn's climb — but only for the bytes that stay byte-identical across calls.