Docs / Receipts & audit

Receipts & audit

Every decision becomes a SHA-256 hash-chained, append-only receipt. Export the sealed log and an external auditor re-derives the whole chain from genesis — with no access to the proxy or its database. Tamper or reorder, and verification breaks at that index and exits non-zero.

Every Firstpass call produces a Trace — a JSON receipt carrying the full routing decision: every rung attempted, each gate verdict, tokens and cost per attempt, and the model that served. Receipts are SHA-256 hash-chained from a genesis record, so the log is tamper-evident by construction. Export the sealed log and any auditor re-derives the whole chain from genesis — no proxy, no database, just math over a JSONL file.

receipt written per call, atomically

SHA-256

each record hashes its predecessor

JSONL

portable, self-contained export

external dependencies to verify

Why receipts#

A router that decides what your users see must be answerable — not just correct. Regulators, incident reviews, and customer disputes need proof of the decision, not just the outcome. A receipt names every rung tried, every gate verdict, the mode, the cost — and a cryptographic link to every prior decision — so accountability is provable, not asserted.

A router that decides what your users see needs to be answerable. "The model chose it" is not an answer a regulator, an incident review, or a customer dispute will accept. Firstpass makes every decision independently checkable after the fact: the rungs tried, each gate verdict, the mode, the cost, and a cryptographic link to the decision before it.

Key idea "The model chose it" is not an audit trail. A receipt names the routing decision: which rungs were tried, which gates fired, what each one returned, what the final choice cost — and a cryptographic chain that lets anyone confirm the record hasn't been altered. The math replaces trust in the vendor.

Receipts turn "the router did it" into a provable record — independently checkable, long after the call, without access to the running system.

The hash chain#

Each receipt stores the SHA-256 of the previous receipt in prev_hash, linking records from a genesis block to the head. Edit any record — its hash changes, and the next record's prev_hash no longer matches. firstpass verify re-derives the chain from genesis and exits non-zero at the first broken index. Everything before the break still verifies; the failure is localized and provable.

Each receipt stores the SHA-256 hash of the previous record in prev_hash, forming an append-only chain from a genesis record. Because each hash covers the record's contents and its predecessor's hash, editing any record changes its hash — which no longer matches the next record's prev_hash. The chain is tamper-evident by construction.

Edit one record and the break is localized and provable: everything before it still verifies, and the verifier exits non-zero at the exact index that was altered.

Under the hood The hash is SHA-256 over the canonicalized record JSON — deterministic serialization ensures any two parties computing the same record reach the same digest. The genesis record's prev_hash is all zeros, the fixed starting point every verifier agrees on. Edit, reorder, or delete a record in place and verification fails at the next record whose prev_hash no longer matches its predecessor's hash.

Common mistake Assuming the hash chain detects dropped receipts. It catches tampering with written records — any edit, reorder, or in-place deletion breaks a prev_hash link. But a receipt that was never written leaves the chain over the surviving records perfectly intact: there is no mismatch to find, because the missing record was never a link. The chain proves the integrity of what's there, not the completeness of what should be. Use durable mode to close that write gap first; the chain handles everything else.

Each receipt's SHA-256 hash covers its content and its predecessor's hash — edit, reorder, or delete any record and the break is localized and provable at the exact index.

A real receipt#

A receipt (internally a Trace) carries: an attempts[] array — one entry per rung tried, with model, provider, in/out tokens, cost, latency, each gate's verdict, and the overall verdict — and a final{} block with the served rung and model, total cost, gate overhead, total latency, escalation count, the counterfactual baseline, and savings. The two hashes at the root are the chain links that make every record auditable.

A receipt is a JSON object. The load-bearing fields are the trace id, the attempts (each with its gate verdicts and cost), the mode, the total cost, and the two hashes.

The attempts array — one entry per rung tried

Each entry in attempts records one model call end-to-end: which rung, which model and provider, how many tokens were consumed and at what cost, the latency, each gate's name and verdict, and the overall verdict ("serve" or "escalate"). If there are two attempts, the first rung failed a gate and the router escalated to the next rung automatically.

The final block — the decision summary

The final block summarizes what the caller received. served_rung and served_from name the winning attempt. counterfactual_baseline_usd is what the same call would have cost routed straight to the configured strong model; savings_usd is the difference. These two fields are how you audit the router's cost claim — per call, in the log, not in a dashboard you have to trust.

json

{
  "trace_id": "tr_9f2c8a41d3e7",
  "ts": "2026-07-20T18:42:07Z",
  "tenant_id": "acme-prod",
  "session_id": "sess_8821b",
  "mode": "balanced",
  "policy": "default",
  "request": { "api": "anthropic.messages", "prompt_hash": "sha256:1a2b3c…",
               "features": { "tool_count": 0, "has_images": false } },
  "attempts": [
    {
      "rung": 0, "model": "anthropic/claude-haiku-4-5", "provider": "anthropic",
      "in_tokens": 312, "out_tokens": 84, "cost_usd": 0.0003, "latency_ms": 418,
      "gates": [
        { "name": "json-valid", "verdict": "pass" },
        { "name": "schema",     "verdict": "fail", "reason": "missing 'total'" }
      ],
      "verdict": "escalate"
    },
    {
      "rung": 1, "model": "openai/gpt-4.1-mini", "provider": "openai",
      "in_tokens": 398, "out_tokens": 91, "cost_usd": 0.0018, "latency_ms": 631,
      "gates": [
        { "name": "json-valid", "verdict": "pass" },
        { "name": "schema",     "verdict": "pass", "score": 0.83 }
      ],
      "verdict": "serve"
    }
  ],
  "final": {
    "served_rung": 1,
    "served_from": "openai/gpt-4.1-mini",
    "total_cost_usd": 0.0021,
    "gate_cost_usd": 0.0000,
    "total_latency_ms": 1049,
    "escalations": 1,
    "counterfactual_baseline_usd": 0.0089,
    "savings_usd": 0.0068
  },
  "prev_hash": "sha256:a91f7c…",
  "hash": "sha256:5c02d9…"
}

Key idea The receipt tells you what would have happened alongside what did. counterfactual_baseline_usd is the cost if every call had gone straight to the strong model; savings_usd is the difference. This is how you audit the router's savings claim — per call, in the log, without trusting a dashboard.

Under the hood The savings counterfactual counts cache-read tokens at full input price — a known modeling gap if your workload is heavily cached. total_cost_usd is what Firstpass actually spent; what you would have spent unrouted depends on your provider's cache pricing, which Firstpass does not model.

Common mistake Reading only total_cost_usd and ignoring gate_cost_usd. Gate cost is the overhead Firstpass added to verify the output; it is already folded into total_cost_usd, but isolating it lets you measure the verification tax against the savings shown in savings_usd.

A receipt is a Trace — an attempts[] array with per-rung costs and gate verdicts, and a final{} block with savings accounting — plus the two hash links that make it chain-verifiable.

Export & verify#

firstpass export seals the live log to a portable JSONL file; firstpass verify re-derives every hash from genesis and exits non-zero at the first broken link. The verifier needs only the file — no proxy, no database, no Firstpass running. Hand it to any third party and the math delivers the same verdict.

Two commands, and the guarantee is independent of the running system:

bash

# 1. seal the log to portable JSONL
firstpass export --out receipts.jsonl

# 2. an auditor re-derives the chain from genesis — no proxy, no DB
firstpass verify --file receipts.jsonl

Seal → export → independent re-derivation. The check is just SHA-256 over a documented record, so a third party reaches the same yes/no with the proxy offline, compromised, or gone.

verify reads only the file. It recomputes every hash from the genesis record forward and confirms each prev_hash link. If a record was edited, inserted, deleted, or reordered, the recomputed hash diverges and verify exits non-zero at the first broken index — a clean signal for CI or an external auditor. There is nothing to trust except the math and the file.

Independent by design The whole point is that verification needs nothing from Firstpass. Hand the JSONL to a third party; they run firstpass verify (or reimplement the check — it is just SHA-256 over a documented record) and get the same yes/no. The proxy could be offline, compromised, or gone.

Export to JSONL, hand the file to any auditor with firstpass verify, and the math alone delivers the verdict — no proxy, no database required.

Durable never-drop#

The default write mode is best_effort. Under load, a dropped write creates a gap in the chain — indistinguishable from a deletion if you need a complete audit trail. FIRSTPASS_RECEIPTS=durable spills receipts to disk under backpressure and drains on boot, so no decision is silently lost and the chain stays valid across restarts and load spikes.

Under load, a best-effort writer might drop a receipt — and a gap in the chain is as bad as a tamper. Durable mode closes that hole:

bash

FIRSTPASS_RECEIPTS=durable firstpass up

With FIRSTPASS_RECEIPTS=durable, receipts spill to disk under backpressure and drain on boot, so no decision is lost and the chain stays valid across restarts and load spikes. The default is best_effort; switch to durable when an unbroken audit trail is a requirement, not a nicety.

Common mistake Relying on the hash chain alone to detect dropped receipts. A receipt that was never written leaves no hash mismatch — the chain over the surviving records stays intact, so verify reports success even though a record is missing. The chain attests integrity, not completeness; use durable mode when a gapless audit trail is a hard requirement.

Durable mode closes the write gap: receipts spill to disk under backpressure and drain on restart — the hash chain then handles everything else.

Compliance#

The receipt chain is the foundation for compliance workflows: every served output traces to a full decision record, cost is itemized per call, tampering is provably detectable at a specific index, and the check runs without the vendor in the loop.

The receipt chain is the substrate for audit and compliance workflows:

Answerability — every served output traces to a decision with its full rationale.
Cost attribution — model spend and gate cost are itemized per decision.
Tamper evidence — an altered log is provably altered, at a specific index.
Independence — the check runs without the vendor in the loop.

Verify receipts programmatically over MCP with the verify_receipts tool — see API & MCP.

Key idea The MCP verify_receipts tool exposes the same chain-check over an API surface — useful for programmatic audits, CI pipelines, or regulatory exports where a human-run CLI is not the right integration point.

The receipt log is independently auditable, cost-attributable per decision, and tamper-evident — the substrate for any compliance workflow that needs more than "trust the vendor."