Firstpass v0.2.1
GitHub
DocsGuaranteeBenchmarksCLIGitHub

Docs / Every scenario, end to end

Every scenario, end to end

An agent doesn't make one model call — it makes hundreds, and the safe reflex of sending every one to the strongest model is why an afternoon of agent work bills like a research run. Firstpass runs the cheapest model first, proves its output against a real check, and escalates only when the proof fails. You pay for strength exactly when strength is needed — and every decision leaves a receipt that says why.

Every request Firstpass sees — a one-shot question, a tool-loop step, a subagent call, turn 17 of a long conversation — travels the same four steps: extract features → match route → prove the output → write a receipt. This page walks every outcome as a real payload, escalation diagram, and receipt. Scan the TL;DRs to see the shape; read the JSON to see the mechanics.

Why routing, and why proof#

How it works showed the four moves on one abstract request. Real agent traffic is never one request — it's hundreds, in a dozen shapes: tool-loop steps, subagent calls, agent teams, turn 17 of a conversation, a provider mid-outage. This page's job is to prove the same four moves hold for every one of them — each as a real payload, escalation, and receipt.

You've seen the idea: run the cheapest rung first, prove the actual output against a gate, escalate one rung only when the proof fails — cheap-and-proven for the easy majority, strong exactly when strength is earned. The reason that's worth a whole page of scenarios is that agent traffic is messy. Point Claude Code, the Agent SDK, or anything speaking the Anthropic or OpenAI wire at Firstpass, and it emits completion requests by the hundred — every reasoning step, every tool-result read, every subagent its own POST, the whole growing transcript riding along on each. A clean idea is easy to state on one request; the test is whether it stays clean when the shapes get real. It does — and the rest of this page is that idea holding its shape, in one picture first:

One request opens on the cheapest rung; its real output is proved against a gate. When the gate passes — measured at 82 percent of requests on a live coding run — the cheap answer is served and no strong model is ever called. When it fails, the cascade escalates one rung and proves again, serving only a proven answer. The routing decision is proof of the output, never a prediction from the prompt. Cheapest first · prove the real output · escalate only on failure the routing decision is proof of the answer, never a guess from the prompt one request a single POST cheapest rung takes the first pass PROVE test · schema · judge on the real output 82% · gate passes serve — cheap & proven the common case · no strong-model bill 18% · gate fails escalate +1 rung budget-capped strong rung serve proven
The gate is the pivot: it proves the cheap model's actual output before anything ships. Pass — most of the time — and the strong model is never called. Fail, and the cascade escalates and proves again. Nothing is served unproven.

Because the decision is proof and not a prompt-time guess, most of the cheap rung's answers turn out good enough to serve as-is — and the ones that don't escalate. That isn't a hope: on a live run of 974 real coding tasks, each scored by its real unit tests, the cheap rung's pass rate is measured:

On a live run of 974 real MBPP coding tasks, one model's outputs were scored by real unit-test gates: 82 percent cleared the conformal serve threshold and can be served on the cheap rung, while 18 percent fell below it and would escalate in a deployed cascade. The realized served-failure rate was 7.7 percent, under the 10 percent target at 95 percent confidence. Cost savings depend on your models and workload and are not measured here. The cheap rung on 974 live coding tasks single-model conformal run · real unit-test gates, live provider Cheap-rung outputs, by the serve gate 82% cleared the serve gate 18% good enough to ship on the cheap rung — no escalation needed below the threshold — would escalate on proof of need 82% cleared the gate on the cheap rung. That is the savings lever — those requests need no strong model. Your dollar figure is your workload's; measure it. Cost shape (illustrative) every call · top price Firstpass cheap-first, proven strong paid only on 18% Measured: 82% serve fraction, 7.7% served-fail (single-model conformal). Cost bars illustrative — no dollar figure is measured; Firstpass measures your real savings.
Measured on 974 live MBPP tasks (single-model conformal): 82% of cheap-rung outputs cleared the serve gate and 7.7% served-failure, under the 10% bound at 95% confidence. In a deployed cascade the other 18% escalate; your cost savings depend on your models — measure them. Full run in Benchmarks; the math in The guarantee.
New here? Read this after "How it works" That page builds the gate and the ladder — the machinery under the picture above. This page is the guided tour: what a request is, how your prompts become hundreds of them, and how the same loop carries you from a one-shot question to a dynamic multi-agent workflow. The rest of the page is that loop, one request at a time.
Route by proof of output, not prediction from prompt — the gate decides, and 82% of the time on a live coding run the cheap rung's output clears it.

What a request looks like#

A request is a stock Anthropic (or OpenAI) API call — nothing Firstpass-specific on the wire. The router reads exactly two optional headers and three structural body signals — tool count, image flag, a coarse prompt-size bucket — to choose a route. It never reads your prompt text. Once a route matches, the full body is forwarded unchanged to whichever rung the route picks.

A request is a standard provider call. Firstpass accepts two wire dialects and matches whichever the client speaks:

FieldWhat it is
POST /v1/messagesAnthropic dialect — what Claude Code and the Agent SDK send.
POST /v1/chat/completionsOpenAI dialect — for OpenAI-shaped clients.
"model" in the bodyA real model, or the sentinel "auto". On the enforce path this is overwritten by the rung the router picks — see the lifecycle.
messages / toolsThe full prompt and the entire conversation history so far. The client sends it every turn — the model is stateless, so history rides in the body.

Five optional headers are the only channel through which the client tells Firstpass anything beyond the raw call. They are how routing gets its signal — and how one deployment routes different traffic to different ladders:

HeaderPurpose
x-firstpass-agentWhich agent this call belongs to. Becomes a routing feature.
x-firstpass-subagentWhich subagent (e.g. architect, test-runner). Becomes a routing feature.
x-firstpass-modePer-request override of the route's cost/quality preset (cost, quality, …). Highest-precedence preset signal.
x-firstpass-sessionGroups requests into one session — for the audit trail and the per-session budget cap. The only conversation-aware knob.
x-firstpass-keyTenant key, when multi-tenant auth is on.
No headers still works Send none and Firstpass still routes every call — it falls to the wildcard route (match = {}). Headers don't enable routing; they let you split traffic across ladders. No tags means one ladder for everything, which is a fine place to start.
Key idea Routing is content-blind by design. Firstpass routes on structure — tool count, image presence, prompt-size bucket — not on your prompt text. That makes the routing signal privacy-preserving and the route match fast: no tokenization, no embedding, no semantic classifier runs before the first rung opens.

Here is a real one, exactly as Claude Code puts it on the wire when you ask it to fix a bug. Nothing about it is Firstpass-specific — it is a stock Anthropic call, plus two optional headers:

http
POST /v1/messages
x-firstpass-agent: claude-code          # optional — becomes a routing feature
x-firstpass-session: agent-run-4417     # optional — budget + audit grouping

{
  "model": "auto",                      // sentinel — the route's ladder overwrites it
  "max_tokens": 4096,
  "stream": true,
  "system": "You are Claude Code, …",   // the agent's system prompt
  "tools": [                            // the agent's toolbelt — 3 tools this call
    { "name": "Read", "input_schema": { … } },
    { "name": "Edit", "input_schema": { … } },
    { "name": "Bash", "input_schema": { … } }
  ],
  "messages": [
    { "role": "user", "content": "find the bug in handler.rs and fix it" }
  ]
}

The router splits this one request into two things it treats very differently. From the headers and a few structural signals — the tool count (3), whether any message carries an image (no), a coarse bucket of the prompt size — it builds the routing features. It never reads system or the message text to decide; the routing signal is deliberately content-blind. Then, once a route is matched, the entire body — system, messages, tools, unchanged — is forwarded to whichever rung answers. Features decide where the request goes; the body is what the model actually reads.

A request is read two ways. From the headers and a few cheap structural signals — the agent tag, the tool count, an image flag, a coarse size bucket — the router builds content-blind routing features; it never reads the prompt or system text. Once a route is matched, the full body — messages, tools, and all — is forwarded unchanged to whichever rung the route picks. One request, read two ways POST /v1/messages x-firstpass-agent: claude-code "model": "auto" "system": "You are Claude Code…" "tools": [ Read, Edit, Bash ] "messages": [ user: "find the bug…" ] structural signals feed features whole body is forwarded on route_for features → route Router reads → features agent · subagent · tool_count 3 size bucket · has_images false prompt + system text never read Forwarded unchanged → chosen rung system + messages + tools, byte-for-byte the model reads the whole body
The routing decision rides on the headers and a few structural signals — not the content. The prompt, history, and tools are forwarded whole to the rung that answers.
Route features come from structure, not text — Firstpass decides where a request goes without ever reading what it says.

The life of one request#

Every POST — one-shot question, tool-loop step, subagent call, turn 17 of a conversation — travels the same four steps: extract features → match route → enforce or observe → serve and record. No fast path, no special cases. Uniformity is what makes per-request proof safe to run on every call.

This is the concrete pipeline behind the four moves — the same route → prove → escalate → serve you met in How it works, drawn as the exact steps a request takes on the wire. Every scenario below is this one spine with different inputs: no fast path for "simple" prompts, no special case for subagents. The uniformity is the point — it's what lets one engine stay correct across every case that follows.

One request flows through four steps: extract features from headers and body, match a route by those features, then either enforce (cascade cheapest-first with a gate on each rung, escalating on failure) or observe (forward the original request unchanged). Every decision is written to a receipt. inbound POST served bytes + sealed receipt Same four steps for every request Request one POST Extract features agent · subagent · size Match route route_for(features) Enforce ladder + gate, escalate on fail Observe forward original, unchanged Serve + receipt caller's dialect out
Extract → match → enforce or observe → serve. The enforce branch is the cascade the rest of the docs describe; observe forwards the request untouched. Both write a receipt.
  1. Extract features. From the headers (agent, subagent) plus cheap body-derived signals — tool count, whether images are present, a coarse prompt-size bucket, an hour bucket. Firstpass never reads your prompt text to route; it buckets by token count, which keeps the routing signal privacy-preserving.
  2. Match a route. route_for(features) walks your [[route]] blocks top-to-bottom and returns the first whose match predicate holds. Every field you set in match is an AND-constraint; every field you omit is a wildcard; match = {} catches everything.
  3. Enforce or observe. The matched route's mode decides. Enforce runs the cascade: open on the cheapest rung, run the gate, serve if it passes, else climb one rung — overwriting the request's model with the rung's model as it goes. Observe forwards the original request byte-identically (also the safe fallback when a request can't be routed faithfully).
  4. Serve and record. The winning output is returned in the caller's own dialect. The full decision — every rung tried, every gate verdict, the cost — is written to a hash-chained receipt.
Where "which model answered" lives The response you get back is a normal completion in your dialect — no vendor cruft bolted on. The rung, the gate verdicts, and the cost live in the receipt. The split is deliberate: the answer is the product, the proof is the receipt. You'll see both, for a real escalation, in the complex case.
Four deterministic steps, every time — the gate verdict and the provider response decide which of the five endings you land on.

The scenarios, one by one#

Four steps, five outcomes. The `served_from` field on the receipt — `attempt`, `best_attempt`, `error` — is the machine-readable branch point that tells your pipeline which ending it got. Scan the map here; read the next five sections for the real payloads.

Those four steps produce exactly five outcomes — and every request Firstpass serves is one of them. Scan the map, then follow any row to its real request, response, and receipt. The served_from field on the receipt is how your pipeline tells the outcomes apart after the fact.

ScenarioWhat the router doesThe receipt says
Cheap model is rightRung 0 answers, the gate passes, it's served. No escalation, no waste.served_from: "attempt" · rung 0
Cheap model falls shortRung 0's output fails the gate; climb one rung; rung 1 passes and is served.served_from: "attempt" · rung 1
Nothing clears the gateLadder or budget runs out with no pass; the best attempt is served, flagged unproven.served_from: "best_attempt"
A provider is downA rung 5xxs → it abstains → fail over to the next rung, even on another provider.an abstain, then pass on a new provider
Observe modeForwarded byte-for-byte to the model you asked for; the cascade runs only in the shadow.mode: "observe"
One engine, five endings There is no branch in the code for "hard prompt" or "provider outage." The same extract → match → cascade → serve runs every time; the gate verdicts and the provider responses decide which of these five endings you land on. The next five sections are those endings, each a real request you can run.
One loop, five outcomes. served_from on the receipt is the branch point for downstream code.

A simple question#

Rung 0 runs, the gate passes, done. No escalation, no overhead beyond a single cheap model call. The receipt has one attempt, served_from: "attempt", served_rung: 0. This is the common case — 82% of outputs on a live coding run clear the gate at rung 0.

Start with the easy case. You ask a one-shot question that needs no tools. The agent sends one request; Firstpass opens it on rung 0 (the cheapest model), the gate passes, and that output is served. One request, one rung, done. Here is the actual call:

bash
# "model":"auto" hands the choice to the matched route's ladder
curl localhost:8080/v1/messages \
  -H 'content-type: application/json' \
  -d '{
    "model": "auto",
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "Capital of France?"}]
  }'

The response comes back as an ordinary Anthropic completion — the same shape the provider would have returned, because from the client's side nothing about the wire contract changed:

json
{
  "id": "msg_01F3k…",
  "type": "message",
  "role": "assistant",
  "model": "claude-haiku-4-5",          // the rung that actually answered
  "content": [{ "type": "text", "text": "Paris." }],
  "stop_reason": "end_turn",
  "usage": { "input_tokens": 14, "output_tokens": 4 }
}

The receipt for that request has a single attempt on rung 0 with served_from: "attempt" and served_rung: 0. No escalation, no waste. The whole cascade for an easy request is the cheapest model — the gate just confirmed it was good enough to ship. The interesting case is when it isn't.

served_from: "attempt" · served_rung: 0 — the cascade ran, the gate confirmed, the cheapest answer shipped.

A complex question#

A complex task is a loop of independent requests — each routed and proven from scratch. When the cheap rung's output fails the gate, the cascade climbs exactly one rung. An easy turn later in the same loop re-enters at rung 0; the router never pins difficulty to a conversation.

Now a real task: "find the bug in this handler and fix it." The agent can't answer in one shot — it reads files, reasons, edits, re-reads, and concludes. That is a loop, and every iteration is its own request:

Zoom in: request 3, where escalation happens

Firstpass routes each one independently and proves each one independently. Zoom into request 3, where the edit is gated by the project's own test suite. The cheap rung takes the first pass, its patch fails cargo test, and the cascade climbs one rung — which passes. This is the whole product in one request:

Request 3 escalates. Rung 0 (haiku, $0.0007) produces a patch that fails the cargo-test gate. The cascade climbs to rung 1 (sonnet, $0.0121), whose patch passes the gate and is served. Total cost $0.0128 versus a $0.0630 always-strong baseline — the receipt records every rung, verdict, and dollar. Rung 0 fails its gate → climb → rung 1 passes → serve rung 0 · haiku-4-5 in 2000 · out 700 $0.0007 · 900ms cargo-test fail · 3100ms rung 1 · sonnet-5 in 2000 · out 800 $0.0121 · 1200ms cargo-test pass · 2950ms serve rung 1 +1 rung receipt · hash-chained served $0.0128 baseline $0.0630 saved $0.0502 escalations 1 latency 2100ms
The cheap rung's failed patch costs a real but tiny $0.0007; the escalation buys correctness for $0.0121 more. The alternative — running the strong model on every request — is the $0.0630 baseline the receipt counts against.

That request writes one receipt. It is a JSON object whose field names are the audit contract — every rung tried, every gate verdict, the cost, and the previous record's hash. This is the real serialized shape, straight from the trace record:

json
{
  "trace_id": "b7e1c2a0-…-4417",
  "prev_hash": "sha256:a91f7c…",
  "tenant_id": "acme",
  "session_id": "agent-run-4417",
  "ts": "2026-07-20T18:42:07Z",
  "mode": "enforce",
  "policy": { "id": "static@v0", "explore": false },
  "request": {
    "api": "anthropic.messages",
    "prompt_hash": "sha256:3d9b…",          // salted hash — never the prompt text
    "features": {
      "version": 1, "task_kind": "code_edit", "language": "rust",
      "agent": "claude-code", "prompt_token_bucket": 11,
      "tool_count": 4, "has_images": false, "hour_bucket": 18
    }
  },
  "attempts": [
    { "rung": 0, "model": "anthropic/claude-haiku-4-5", "provider": "anthropic",
      "in_tokens": 2000, "out_tokens": 700, "cost_usd": 0.0007, "latency_ms": 900,
      "gates": [{ "gate_id": "cargo-test", "verdict": "fail",
                 "cost_usd": 0.0, "ms": 3100, "reason": "2 tests failed" }],
      "verdict": "fail" },
    { "rung": 1, "model": "anthropic/claude-sonnet-5", "provider": "anthropic",
      "in_tokens": 2000, "out_tokens": 800, "cost_usd": 0.0121, "latency_ms": 1200,
      "gates": [{ "gate_id": "cargo-test", "verdict": "pass", "cost_usd": 0.0, "ms": 2950 }],
      "verdict": "pass" }
  ],
  "final": {
    "served_rung": 1,
    "served_from": "attempt",
    "total_cost_usd": 0.0128,
    "gate_cost_usd": 0.0,
    "total_latency_ms": 2100,
    "escalations": 1,
    "counterfactual_baseline_usd": 0.0630,
    "savings_usd": 0.0502
  },
  "hash": "sha256:5c02d9…"                    // SHA-256 of this record — the next record's prev_hash
}

Read that receipt and the routing is fully explained: the cheap rung was tried ($0.0007), its output failed a real test (not a guess — cargo-test ran and returned fail), the cascade escalated exactly once, and the served answer cost $0.0128 against a $0.0630 always-strong baseline. Request 4 in the same loop might pass on rung 0, because "did the edit apply cleanly" is easy. The router never assumes that because turn 2 was hard, turn 4 will be — it re-earns the answer every time. No routing decision is ever carried forward, because none is kept.

served_from: "attempt" · served_rung: 1 — a real gate ran, one escalation bought correctness; the receipt counts every dollar.

Nothing clears the gate#

When no rung clears the gate — or a budget cap stops the climb — Firstpass never returns nothing. It serves the strongest attempt it saw, stamped served_from: "best_attempt" so downstream code can detect and branch on it. Prefer a hard error? Set on_exhausted = "error".

Sometimes the task is hard enough that no rung passes the gate — or a per-request budget cap stops the climb first. Firstpass never returns nothing, and never pretends. It serves the strongest attempt it saw and stamps the receipt served_from: "best_attempt" — a flag your pipeline can branch on. Here a strict json-valid gate rejects every rung, and a $0.02 per-request cap halts the climb before the top model is ever tried:

Rung 0 (haiku) fails the json-valid gate, the cascade climbs to rung 1 (sonnet) which also fails, and the per-request budget cap stops the climb before the top rung is tried. With no rung passing, the best attempt seen — rung 1 — is served and the receipt is stamped served_from best_attempt, unproven, so downstream code can detect it. Every rung fails → serve the best attempt, flagged unproven rung 0 · haiku in 60 · out 900 $0.0012 json-valid fail rung 1 · sonnet in 60 · out 1100 $0.0185 json-valid fail budget cap $0.02 · stop opus untried serve best attempt · rung 1 unproven +1 rung receipt · served_from best_attempt UNPROVEN served $0.0197 served_rung 1 baseline $0.0630 no gate passed
No rung cleared the gate, so the best attempt is served — honestly flagged. served_from: "best_attempt" is the machine-readable signal that the answer shipped unproven; the receipt still counts every dollar spent.
bash
# a genuinely hard ask behind a strict json-valid gate
curl localhost:8080/v1/messages \
  -H 'content-type: application/json' \
  -d '{
    "model": "auto",
    "max_tokens": 512,
    "messages": [{"role": "user",
      "content": "Return ONLY valid JSON: every US state with its capital and 2020 population."}]
  }'

Both rungs run, both fail the gate, the cap stops the climb before opus, and rung 1's output — the best of a bad set — is served. The receipt tells the whole story, and the honesty is in one field:

json
{
  "trace_id": "c4a2e1f0-…-9f31",
  "mode": "enforce",
  "request": { "api": "anthropic.messages", "prompt_hash": "sha256:7b1e…" },
  "attempts": [
    { "rung": 0, "model": "anthropic/claude-haiku-4-5", "provider": "anthropic",
      "in_tokens": 60, "out_tokens": 900, "cost_usd": 0.0012, "latency_ms": 1400,
      "gates": [{ "gate_id": "json-valid", "verdict": "fail", "reason": "prose before JSON" }],
      "verdict": "fail" },
    { "rung": 1, "model": "anthropic/claude-sonnet-5", "provider": "anthropic",
      "in_tokens": 60, "out_tokens": 1100, "cost_usd": 0.0185, "latency_ms": 2100,
      "gates": [{ "gate_id": "json-valid", "verdict": "fail", "reason": "trailing comma, line 148" }],
      "verdict": "fail" }
  ],
  "final": {
    "served_rung": 1,
    "served_from": "best_attempt",          // ← no gate passed; best effort shipped, flagged
    "total_cost_usd": 0.0197,
    "total_latency_ms": 3500,
    "escalations": 1,
    "counterfactual_baseline_usd": 0.0630
  }
}
The served-failure guarantee, stated precisely You always get bytes back — the strongest answer the budget bought — and the receipt marks it unproven so you can retry, flag, or fall back. It is never a silent bad answer dressed up as a good one. Prefer a hard error over a best-effort serve? Set on_exhausted = "error" and an exhausted ladder returns a structured error with served_from: "error" instead of a body. Either way the outcome is explicit — see the guarantee.
served_from: "best_attempt" — unproven, flagged, and still real bytes. Never a silent bad answer, never a router error in place of a completion.

A provider is down#

A provider 5xx is an abstain, not a gate failure. The cascade tries the next rung — which can live on a completely different provider. The client sees one ordinary response; only the receipt reveals the failover. A hard 4xx stops immediately — the request itself was wrong, and another provider can't fix that.

Providers have bad minutes. When a rung returns a 5xx or drops the connection, Firstpass treats it as an abstain on that rung — not a gate failure, not a served answer — and moves to the next rung, which can live on a different provider entirely. The client sees one ordinary response; only the receipt reveals the failover. The ladder here crosses providers on purpose:

Rung 0 on Anthropic returns HTTP 503, so it is recorded as an abstain rather than a failure. The cascade fails over to rung 1 on OpenAI, which answers and passes its gate. The served response is re-rendered into the caller's Anthropic dialect, and the receipt shows the abstain, the provider switch, and served_from attempt on rung 1. 503 on one provider → abstain → the next provider serves rung 0 · anthropic claude-haiku-4-5 the ladder's first rung HTTP 503 abstain rung 1 · openai gpt-4o-mini · provider switch same request, sent elsewhere non-empty pass serve rung 1 fail over receipt · served_from attempt rung 0 abstain (503) rung 1 pass provider anthropic → openai served_rung 1 · stayed up
A provider-side fault costs nothing and serves nothing — it abstains, and the cascade tries the next rung. Because the ladder spans providers, an Anthropic outage is answered by OpenAI, in your original dialect, without the client noticing.

The response comes back in your dialect as always — the only tell is the model field, which names the rung that actually ran:

json
{
  "id": "msg_01Rb7…",
  "type": "message",
  "role": "assistant",
  "model": "openai/gpt-4o-mini",          // answered by the failover rung — in Anthropic shape
  "content": [{ "type": "text", "text": "Paris." }],
  "stop_reason": "end_turn",
  "usage": { "input_tokens": 14, "output_tokens": 4 }
}

The receipt records the abstain and the cross-provider serve — the failover is auditable after the fact:

json
{
  "trace_id": "e91c3d55-…-2a70",
  "mode": "enforce",
  "attempts": [
    { "rung": 0, "model": "anthropic/claude-haiku-4-5", "provider": "anthropic",
      "in_tokens": 0, "out_tokens": 0, "cost_usd": 0.0, "latency_ms": 240,
      "gates": [],                                // provider errored before any output could be gated
      "verdict": "abstain" },
    { "rung": 1, "model": "openai/gpt-4o-mini", "provider": "openai",
      "in_tokens": 14, "out_tokens": 4, "cost_usd": 0.0001, "latency_ms": 380,
      "gates": [{ "gate_id": "non-empty", "verdict": "pass" }],
      "verdict": "pass" }
  ],
  "final": {
    "served_rung": 1,
    "served_from": "attempt",
    "total_cost_usd": 0.0001,
    "total_latency_ms": 620,
    "escalations": 1
  }
}
A hard 4xx is not a failover A 4xx means the request itself was wrong — malformed body, bad auth — and a stronger model on another provider can't fix that. So the ladder stops immediately with served_from: "error" and a structured error to the caller. Failover is only for provider-side faults (5xx, transport), where the same request may well succeed elsewhere.
Provider fault = abstain = try the next rung. Cross-provider failover is automatic when your ladder spans providers; the client never sees the switch.

Observe mode: shadow, zero change#

Observe forwards every request byte-for-byte to the model you asked for and returns the provider's own response — no change. In the shadow, Firstpass runs the cascade and writes a receipt with the savings estimate — real measurement, zero behavior change. It is the default mode on a fresh install, and the safe on-ramp before flipping to enforce.

Before Firstpass changes a single response, run it in observe — the default on a fresh install. Every request is forwarded byte-for-byte to the exact model you asked for, and the response you get is precisely what the provider returns. Firstpass still scores the cascade in the shadow and writes a receipt, so you collect the savings estimate and the audit trail with zero behavior change. Set it per route, or per request with a header:

In observe mode the request is forwarded byte-for-byte to the model the client asked for and that provider's response is served unchanged — the top path. In parallel, in the shadow, Firstpass runs the cascade and gate for learning and writes a receipt marked mode observe, which carries the savings estimate. The served path never changes; only the receipt is produced. Served path unchanged · the cascade runs only in the shadow your request model: sonnet-5 Firstpass observe claude-sonnet-5 exactly what you asked for response served provider's own bytes shadow cascade + gate would rung 0 have passed? receipt · estimate mode: observe SERVED — unchanged SHADOW — write-only
Observe is the safe on-ramp: the client keeps getting the model it asked for, byte-for-byte, while Firstpass quietly measures what a cascade would have saved. When the shadow receipts convince you, flip the route to enforce.
bash
# x-firstpass-mode: observe forwards byte-for-byte and only shadows the cascade
curl localhost:8080/v1/messages \
  -H 'content-type: application/json' \
  -H 'x-firstpass-mode: observe' \
  -d '{
    "model": "claude-sonnet-5",
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "Summarize this PR in one line."}]
  }'

The response is the upstream model's, untouched — model is the one you requested, not a rung. The receipt records the mode and the shadow verdict that feeds the savings estimate over time:

json
{
  "trace_id": "7f0a19c2-…-1c88",
  "mode": "observe",                          // nothing was changed — this receipt is measurement only
  "request": { "api": "anthropic.messages", "prompt_hash": "sha256:11ac…" },
  "attempts": [
    { "rung": 0, "model": "anthropic/claude-sonnet-5", "provider": "anthropic",
      "in_tokens": 900, "out_tokens": 40, "cost_usd": 0.0126, "latency_ms": 1100,
      "gates": [{ "gate_id": "non-empty", "verdict": "pass" }],   // gated in the shadow, for learning
      "verdict": "pass" }
  ],
  "final": {
    "served_rung": 0,
    "served_from": "attempt",
    "total_cost_usd": 0.0126,
    "escalations": 0
  }
}
Observe → enforce, when the data convinces you Read a week of shadow receipts, see the savings and the gate pass-rate on your real traffic, then flip the route to an enforce mode — one line per route. See observe vs enforce and Configuration.
Observe = zero risk, real measurement. Read the shadow receipts on your traffic; flip one config line to enforce when you're ready.

Subagents#

A subagent call is a tagged POST — structurally identical to any other request. Set x-firstpass-subagent on the call, write a [[route]] block that matches it, and the subagent gets its own ladder with its own gate. A `match` value can be a list, so many mechanical subagents share one route. Context never leaks between agents.

When the main agent spawns a subagent, the subagent runs its own loop — and every call in that loop is, again, an independent POST to Firstpass. Nothing about a subagent call is structurally different from a main-loop call. So how does Firstpass route a planner subagent to a strong model and a mechanical one to a cheap model? You tag the request. The orchestrator sets x-firstpass-subagent on the subagent's calls, and a route matches on it:

toml
# the planner: enforce a tall ladder with a quality bias
[[route]]
match = { subagent = "architect" }
mode  = "enforce"
routing_mode = "quality"
ladder = ["anthropic/claude-haiku-4-5", "anthropic/claude-sonnet-5", "anthropic/claude-opus-4-8"]

# mechanical subagents: one cheap rung is plenty — and match takes a list
[[route]]
match = { subagent = ["test-runner", "explore"] }
mode  = "enforce"
routing_mode = "cost"
ladder = ["anthropic/claude-haiku-4-5"]

# everything else — the wildcard catch-all, always last
[[route]]
match = {}
mode  = "observe"

Two schema details do most of the work here. mode is enforce or observe — whether the route acts or just shadows. routing_mode is the cost/quality preset on top of that. And a match value can be a single string or a list (any-of), so a batch of mechanical subagents shares one route.

The main agent's calls and each subagent's calls arrive as independent requests. Firstpass matches each to a route by its subagent tag: the architect subagent to a tall quality ladder, the test-runner to a single cheap rung, and untagged main-loop calls to the wildcard route. The tag picks the ladder main loop no subagent tag x-firstpass-subagent: architect x-firstpass-subagent: test-runner route_for first match wins wildcard · observe match = {} quality · haiku → sonnet → opus match = { subagent = "architect" } cost · haiku only match = { subagent = ["test-runner", …] }
The subagent's requested model does not decide what runs. The matched route's ladder does, and the gate decides where on that ladder it stops.
A subagent has its own transcript Context does not leak between agents. A subagent carries its own message history — its own system prompt, its own growing list of messages — separate from the main agent's. It accumulates turn-over-turn exactly like the main loop (shown below), and every one of its calls is routed and proven on its own.
Tag the call, write the route, get the ladder. A subagent is just a request with a name — the router treats it identically.

Agent teams & dynamic workflows#

An agent team is concurrent POSTs. There is no team object and no coordinator inside the router — Firstpass matches each request to a route by its own features and routes it independently. The one team-level concept is the shared session budget: give every worker the same x-firstpass-session and they draw from one pool.

A team of agents — or a dynamic workflow that fans out dozens of workers in parallel — looks, from Firstpass's seat, like nothing new: more concurrent requests. Ten agents running at once is ten streams of independent POSTs. Each is matched on its own features (agent, subagent, plus task_kind and language if your client sets them) and routed to the ladder you configured for that kind of worker. There is no "team" object and no coordinator inside the router; concurrency is just volume.

What ties a team together is not routing — it's budget. Give every request in a workflow the same x-firstpass-session, and the per-session cap (per_session_usd) governs the whole team's spend as one pool. That is the one place a multi-agent workload is treated as a unit, and it is a spend guardrail, not a routing input.

Dynamic ≠ special A workflow that decides at runtime to spawn a new worker doesn't need Firstpass to know that in advance. The new worker's first call is just another request; it matches a route the instant it arrives. Dynamism lives in the orchestrator; Firstpass only ever sees the next POST.
Concurrency is just volume. Per-worker routes differentiate; a shared x-firstpass-session ties the team's spend to one budget cap.

Multi-turn & "previous context"#

The router keeps no memory between turns — and that is the feature. Context rides in the request body (the full history, resent every turn), so the model that answers turn 12 can be different from turn 1 and lose nothing. Every turn re-enters the cascade at the start rung and is proven from scratch.

This is the question that trips people up, so here is the blunt version: the router does not remember your conversation, and that is the feature. Statelessness is what makes per-request proof safe to run on every call — a router that pinned "this conversation needs Opus" would be trading a correctness guarantee for a cost guess.

Where does the context go, then? Into the request body. An LLM is stateless — every turn, the client (Claude Code, the SDK) resends the entire message history so the model can "remember." That history is part of the POST Firstpass forwards. So the context absolutely reaches the model — it just rides in the prompt, not in any router-side memory. You can watch it happen: turn 2's body is turn 1's body with the last exchange appended verbatim.

json
// TURN 1 — the POST body carries a single message
"messages": [
  { "role": "user", "content": "find the bug in handler.rs and fix it" }
]

// TURN 2 — same conversation. Turn 1 is kept verbatim; the model's
// reply and the tool's output are appended. Nothing is dropped.
"messages": [
  { "role": "user", "content": "find the bug in handler.rs and fix it" },
  { "role": "assistant", "content": [
    { "type": "text", "text": "Reading the handler." },
    { "type": "tool_use", "id": "toolu_01A", "name": "Read",
      "input": { "path": "handler.rs" } }
  ] },
  { "role": "user", "content": [
    { "type": "tool_result", "tool_use_id": "toolu_01A",
      "content": "pub fn handle(req: Req) -> Resp { /* … */ }" }
  ] }
]

What actually carries across turns

The body grew from one message to three, and it only ever grows — every future turn stacks on top of this one. Here is the payoff that surprises people: because the whole thread rides in the body, the rung that answers turn 12 can be a different model than answered turn 1 and lose nothing. Each call reads the transcript cold, so continuity lives in the bytes, not in any one model's memory. Turn 12 re-enters the cascade at the same start rung as turn 1 and is proven from scratch. Two things, and only two, thread across the calls of one conversation:

ThreadWhat it carries across turns
x-firstpass-session → budgetA spend cap and an audit grouping. The only conversation-aware knob. It never changes which model is picked — only whether the session has money left.
Learned start-rung (optional)An off-by-default bandit (ADR 0007) can learn a smarter entry rung for a class of traffic — keyed by request features, not by conversation. It moves where the climb begins; the gate still decides what ships, so it can never change correctness.
Common mistake Assuming that because turn 8 escalated to sonnet, turn 9 will start there too. It won't — every turn re-enters at rung 0 and is proven from scratch. An "easy" turn 9 answer from the cheap rung is exactly as cheap as it would have been in a brand-new conversation. The session budget is the one thing that connects turns; the routing logic treats each one as if no prior turns exist.
Each turn of a conversation re-enters the cascade independently at the start rung and is proven from scratch. The router keeps no memory between turns. The only line connecting the turns is the session budget, which caps their combined spend. Every turn starts fresh — only the budget carries over turn 1 start rung → gate turn 2 start rung → gate turn 3 start rung → gate turn N start rung → gate no state between turns x-firstpass-session · per_session_usd — one budget across all turns
Statelessness is the feature. A router that pinned "this conversation needs Opus" would trade a correctness guarantee for a cost guess — Firstpass re-proves every turn instead.
Stateless router, stateful body — context rides in the bytes, not in any router-side memory; every turn is proven from scratch.

Claude Code vs the Agent SDK#

Both are HTTP clients that speak the Anthropic dialect. Wire either to Firstpass with one env var or one constructor option and every call — reasoning, tools, subagents — routes through the cascade. Everything on this page applies identically to either client.

Both are just HTTP clients that speak the Anthropic dialect, so both wire up the same way: point the base URL at Firstpass. Everything on this page applies identically to either — the differences are only in how you set the base URL and headers.

bash
# Claude Code — send every call through Firstpass instead of the API
export ANTHROPIC_BASE_URL=http://localhost:8080
# now `claude` runs its whole loop — reasoning, tools, subagents — through the router
ts
// Claude Agent SDK — same idea, set the baseURL on the client
const client = new Anthropic({ baseURL: "http://localhost:8080" });
// optional: tag calls so subagent-aware routes can fire
// headers: { "x-firstpass-subagent": "architect" }
One base URL change, everything routes. Claude Code: ANTHROPIC_BASE_URL. Agent SDK: baseURL on the client constructor.

What you configure, and what's automatic#

Three things are automatic with zero config; three more require you to tag calls or flip a mode. The automatic path is safe and correct — every call routes, proves, and receipts. The optional parts add differentiation, spend controls, and enforcement when the shadow data convinces you.

Everything above is capability out of one seat. Three facts about that seat are worth stating plainly — not as caveats, but as the dividing line between what Firstpass does for free and what you switch on:

Automatic — zero configYou switch on — by tagging or setting a mode
Every call routes. With no headers, all traffic matches the wildcard route and runs one ladder — correct and safe, just undifferentiated.Differentiation. A subagent call and a main-loop call are byte-identical POSTs; Firstpass tells them apart only when the client sets x-firstpass-agent/-subagent. Tagging is a wrapper or SDK request-option away.
Observe mode forwards byte-identically and still writes receipts, so you get the savings estimate with zero behavior change.Enforce. Start in observe, read the shadow receipts, and flip to an enforce mode when the data convinces you. The switch is per-route.
The cascade proves and serves each request; the receipt counts its cost against an always-strong baseline.Long-context spend. History rides in every request, so a long session inflates every rung's input tokens. The cascade doesn't summarize history — that's the client's job — so name it, measure it on the receipts, and cap it with per_session_usd.
The safe default when routing can't apply If a request can't be routed faithfully — an unknown dialect quirk, a translation Firstpass won't risk — it falls back to observe and forwards the original untouched. The served-failure guarantee holds: you never get a worse answer than the provider would have given, only a proven-cheaper one when the gate allows.
Zero config routes everything to one ladder in observe mode. Tags and mode switches add differentiation; per_session_usd caps total spend.

How Firstpass tackles all of it#

Every scenario — simple, complex, exhausted, failover, observe, subagents, teams, multi-turn — reduces to the same four steps. Proof over prediction, one request at a time.

Every scenario on this page reduces to the same handful of moves. There is not much to it:

You have…Firstpass sees…You steer it with…
A simple questionOne request → rung 0 + gateNothing — the default just works
A complex taskMany independent requests, each re-provenThe ladder + gate per route
SubagentsTagged requests matched to their own routex-firstpass-subagent + [[route]]
Agent teams / workflowsMore concurrent requests, matched individuallyPer-worker routes + shared session budget
Multi-turn conversationIndependent turns; history rides in the bodyx-firstpass-session → per_session_usd

The design holds because it is not clever about state. It does not predict what your conversation will need. It proves each request against a gate and serves the cheapest output that clears it. Statelessness is what makes that safe to run on every call. Differentiation comes from the tags you send. The one thing carried across a conversation — the budget — is a guardrail, not a guess. That is the whole model: proof over prediction, one request at a time.

Proof over prediction. One request, four steps, one receipt. Every scenario on this page is this loop.