Distil is the compressor that tells you when it is hurting you — and measures any other compressor, or your provider's own compaction, the same way.
On SWE-bench Lite the shipped path is cost-neutral and within about 2 points on task success — not statistically distinguishable at n=300. Two runs after the shell-search fix: 204/299 vs 210/299 plain (−2.0 pts, 95% CI −5.5..+1.5, cost $7.18 vs $7.18) and 213/298 vs 218/298 (−1.7 pts, 95% CI −5.1..+1.7, cost $8.44 vs $8.52). Non-inferiority at the pre-registered 5-point margin is not shown, and a 7-task long-session pilot is inconclusive. Reports and method → · every compressor, same harness →
Four things are proven rather than promised, and each has its own page:
a cache contract (compression may never rewrite bytes the provider already cached),
a COMA-class adversarial gate (with the two cases that don't come back clean published),
a degradation curve across every rung of the dial,
and an exact-quote guarantee so an Edit still matches the file the agent read.
Honest scope: the certificate covers decision-equivalence on a trajectory corpus, not end-to-end task success — on real SWE-bench Verified the proxy does not transfer once compression gets aggressive, and we publish that. Research configurations that did better there are not what ships — full statement, CIs and ablations →
Most compressors ask for trust.
Distil measures — and says when it hurts.
- Drops tokens and hopes the agent still decides the same
- “Looks fine” on a demo — no proof on your traffic
- You discover the regression in production
- Certifies decision-equivalence before it ever serves
- A shadow gate re-checks live decisions with paired replays — a self-agreement control absorbs sampling noise
- Gives back what it folds — byte-exact, from a content-addressed store, through a tool the agent can call mid-task
- Tested against hostile input, not just hard input — and the two cases that fail are on the page
- Fails closed — if quality would drop, it doesn't compress
How it works — in plain English#
No jargon. Here's the whole idea in three pictures.
Don't re-send what's already remembered
Every turn, your agent re-sends the entire conversation — and you pay for all of it, again. The model can cheaply "remember" the parts that haven't changed, so Distil keeps those parts perfectly stable and only touches the new bits. Like not re-reading someone the whole book to add one sentence.
Drop only what doesn't matter
Most of that context never changes what the agent decides to do next. Distil finds those dead-weight parts with a simple test — remove it, replay, did the decision change? If no, it was safe to cut. Keep the clues, drop the filler.
Prove it — don't just trust it
Here's the part nobody else does: Distil measures that your agent still makes the same choice on the shrunk context. If it can't prove that, it sends everything, untouched. A warranty on the compression, not a promise.
And nothing is truly thrown away — trimmed detail is filed nearby, and the agent can pull back the exact original the moment it needs it. That's why Distil can compress hard and stay safe. The gentle, longer walkthrough →
Under the hood — a cost-optimized cache hierarchy with a contract#
Most compressors optimize bytes. In an agent loop the money is somewhere else.
Two highest-leverage techniques — and why they win
Cache-aware compression
You re-send the growing context every turn. With prompt caching a cache read is ~10× cheaper than fresh input — so the dominant cost is cache misses, not context size. Distil keeps the prefix byte-stable and compresses only the volatile tail. Naive recompression sends fewer tokens yet costs more, because it rewrites the cached prefix every turn.
Since 1.45 the live adapter keeps the cached prefix byte-stable, and the
provider's own cache_read confirms the prefix survives every turn (a live A/B whose
raw output was not committed). Before 1.45 the live adapter had the failure it names above —
in that same A/B it cost 2× direct. See
CACHE.md.
Causal / counterfactual pruning
The eval engine isn't a ruler — it's a discovery engine. Remove a context block, replay, did any decision change? Blocks that never change a decision are provably free to drop: speculative retrievals, stale history. The measurement produces the compression policy.
Exact-quote provenance
A coding agent's Edit is a literal match against bytes it read
earlier. Digest that read and the decision is unchanged — the edit is simply
skipped, and the run ends with the agent reporting success and the file
untouched. Distil reads provenance from the shell command
(cat, head, sed -n), not just the tool name,
and refuses to digest a verbatim slice of a file.
Measured content-free over 2,489 local sessions: shell reads are 33.6% of tool-result mass, against 10.7% matched by a tool-name rule. Byte-exact quote loss falls 39.3% → 16.2%. It costs savings — that price is published rather than hidden.
An estimator allowed to say no
Shadow mode replays a sampled request three times — twice on the original context,
once on the compressed one — and reports the paired difference
1{A=B} − 1{A=A′} with a bootstrap 95% CI, unclipped. A ratio
clamped at zero can never report harm; a difference can. One reporting floor
(50 A/B + 30 A/A) gates every surface, and below it each one prints
below reporting floor instead of a number. It reports per compression mode,
because pooling hides the one that hurts: on the maintainer's 2026-09-15 sample, digest agreed
102/206 times against a self-agreement of 113/206 (−5.3 pp, over the budget) while
lossless-only read 109/192 against 109/193 (≈0). The drift guard now holds digest at
lossless-only wherever its recent harm bound is over budget.
A grader that is itself measured
A certificate is only as good as the model that graded it. The live certifier was changed under a pre-registered, paired migration eval on real traces, and the served path is graded separately from the certified one. Model migration →
The proof — a quality contract, not an estimate#
We didn't just benchmark ourselves. We ran the real LLMLingua-2 and Headroom packages through the same gate, graded live by the real model.
Dated: 2026-07-05, distil 1.10.1 vs llmlingua 0.2.2 and headroom-ai 0.27.0. Headroom has moved since: re-run 2026-09-04 against headroom-ai 0.37.0, distil-causal takes 52.9% tokens / 58.7% dollars at 100% decision-equivalence (PASS) against Headroom’s 1.7% / 2.0% / 81% (FAIL) on the warm corpus gate, and on the read→edit→re-read codebench workload Headroom takes 35.6% tokens for +4.9% dollars. The “2.1× less aggressive” ratio is from 0.27.0 and does not survive the re-run. Fresh head-to-head →
Proxy caveat: this is next-action equivalence on a (partly synthetically decision-determined) corpus — not end-to-end task success. On real SWE-bench Verified (E7) aggressive compression drops pass@1 52%→16%; the proxy certificate does not transfer. Research configurations (E8–E14) did better on SWE-bench Verified; they are not the shipped path, and the shipped path's task outcome is the SWE-bench Lite result at the top of this page.
Underneath it: a strategy ships only if a pre-registered non-inferiority test certifies it. Lossless passes; quality-degrading compression is rejected.
And after it ships, the same budget keeps watching. The certificate, the live drift alarm and the proof ledger read one pinned risk budget (decision change ≤5% at 95% confidence). When shadow mode's paired verdicts prove live traffic has crossed it, the proxy stops compressing lossily from the next request, across restarts, and writes the trip to the receipt chain. Every wrap exit says so in plain words until you recalibrate. An alarm that only prints at exit is a log line; this one acts.
$ distil savings --pricing claude-opus-4-8 strategy $ / run vs baseline cache hits ------------------------------------------------------------------------ baseline (no cache, no compress) 0.01524 0.0% 0 cache only 0.01115 26.8% 1,028 naive compress + cache 0.01696 -11.3% 0 ← busts cache distil (cache-aware lossless) 0.01023 32.8% 1,028 $ distil certify --strategy distil decision-equivalence match rate: 100.0% VERDICT: PASS (certified non-inferior) $ distil certify --strategy aggressive decision-equivalence match rate: 0.0% VERDICT: FAIL (would degrade quality — blocked)
Small numbers, shown anyway#
These sit here rather than beside the proof tiles above, because they answer a different question. The tiles above are what distil does, and they are large. These are how many people have opted in to being counted, and they are small — a project that publishes a calibrated decision-change rate does not get to hide its install count. Every figure is recomputed nightly by public code from a public git branch, and consenting is not contributing: an idle opt-in is never counted as a saver.
Measured across 9 domains — the same gate, not one example#
A strategy isn't trustworthy because it works once. distil bench certifies
every domain — ops, coding, support, research, data, devops, finance — or it doesn't ship.
Risk-graded tiers#
Apply the provably-safe ones everywhere; gate the rest on evidence.
Provably lossless
Reconstructable transforms — JSON minify, reversible run-length encoding. Always on.
Reversible digest
Decision-aware digest + a handle; the full original stays local and re-expands on demand.
Lossy, but gated
Pruning & summarization allowed only at ratios the non-inferiority gate certifies.
Capabilities — every layer of the cost stack#
| Capability | What it does | Loss profile |
|---|---|---|
| Priced cache-aware engine | Models the multi-turn loop and proves naive recompression busts the cache | — |
| Schema canonicalization | Recursively key-sorts JSON/tool payloads so the prefix is byte-stable | lossless |
| Volatile-field extraction | Lifts dates/UUIDs/JWTs out of the cached prefix so it stops churning | lossless · reversible |
| Reversible digest + handles | Originals stay local and re-expand on demand | reversible |
| Reject-if-bigger invariant | Never emits a block larger than its original | safety |
| Adversarial validation gate | distil validate — drives the real compressor against hostile inputs and asserts reversibility, recency-exactness, fail-open, and content-free telemetry hold | CI gate |
| Causal / counterfactual pruning | Ablation discovers context that never changes a decision | certified |
| TOST non-inferiority gate (DERC) | Per-step: a strategy ships only if it passes the quality contract | the moat |
| Trajectory-level certificate | Task-level: distil certify-trajectories bounds end-to-end degradation on matched full/compressed runs | the differentiator |
| Savings ledger + leaderboard | Local-first, privacy-preserving cumulative savings | opt-in |
| Content-type keep policy | Pins each kind's load-bearing lines — a log's pass/fail verdict, a traceback's frames, a diff's hunk headers | 1.15 |
| Query-aware salience | Sees the agent's intent in the same request and pins the line it's asking about — lexically (a grep hit, a SHA), and now semantically: a learned model keeps the line that answers the query with no shared word ("retry limit" → max_attempts = 5) | 1.16 |
| Columnar fold | Re-encodes JSON record arrays into a compact self-describing table — real token savings even on a flat-rate plan | lossless |
| Session dissect | distil dissect — per-session deep-dive with an anomaly list that catches silent failures automatically | 1.15 |
| Savings discovery | distil discover — what is still costing you across recent sessions, ranked, each with its derivation and the command to act on it; median and p10/p90 printed beside the best session, and silent where it cannot measure | report |
| Drift guard | When live paired verdicts prove decision change has crossed the one risk budget, every proxy on the machine serves lossless-only until distil reset --drift-guard — an alarm that acts, not a log line | safety · rc soak |
| Cold-point recompression | Shrinks older tool output only on a turn the provider's cache has certainly expired for, then holds those bytes stable — clause (h); not yet measured live | reversible · rc soak |
| Sealed receipt chain | A content-free, hash-chained receipt per request, sealed into segments with Merkle-root checkpoints: hand an auditor one receipt and its inclusion proof, not the whole log (distil receipts --prove) | audit |
| Opaque-block passthrough | Anthropic compaction/thinking blocks and OpenAI reasoning/compaction items reach the provider byte-identical, pinned by contract tests, and are censused rather than hidden | untouched |
| Tool definitions left alone | The tools array is never rewritten (measured: too little to gain, ADR 0012); distil wrap -- claude keeps Claude Code's own MCP tool search on instead — verified live on 1.54.0 (tools_deferred 4–5, tool payload 9,733 tokens) | untouched |
Works with every SDK — one proxy, no code change#
Point any base_url-honoring client at the proxy — Python, TypeScript, any language —
and get cache-aware lossless compression. See integrations →
Install — pick a format#
The stdlib-only core makes the packaging clean: zero runtime deps, corpus bundled.
uvx — zero install, recommended. Runs straight from PyPI, nothing to clean up.
uvx --from distil-llm distil bench
Run straight from PyPI — nothing installed. Needs uv.
pipx install distil-llm distil certify
Permanent, isolated venv — PEP 668-safe. brew install pipx if needed.
brew install dshakes/tap/distil distil bench
Isolated keg — doesn't touch your system Python.
docker build -t distil . docker run distil bench
Container image — reproducible, zero host deps.
make pyz python dist/distil.pyz bench
Single file — copy distil.pyz anywhere with Python 3.9+.
Already installed? Do this next → distil onboard detects your agent + billing, wires the savings status line, and gives you a guided tour tailored to your setup — then distil default makes it the default for every session (distil offboard reverses that).
pip install is blocked by PEP 668 on modern macOS/Linux, and that's fine). distil onboard wires your agent and the savings status line; distil default makes it the default and distil offboard fully reverses it — nothing is touched that isn't put back. Node/other stacks just point their SDK baseURL at distil proxy. Verify anytime: distil bench → expect GATE: PASS.
from distil.adapters.anthropic import wrap client = wrap(anthropic.Anthropic()) # compresses + keeps the cache warm
What we won't pretend#
--tokenizer anthropic and
--runner anthropic make it billing-grade against the real model.
Compress the context.
Keep the decisions.
One proxy in front of any agent. A signed decision-equivalence certificate on the way out — or it doesn't compress. No API key to try it.