Docs / Overview

Overview

Firstpass is an open-source adaptive LLM router. Every request opens on the cheapest model in a ladder; a gate checks the actual output; it serves the first output that passes and escalates one rung only on gate failure. Every decision gets a tamper-evident, hash-chained receipt.

Firstpass is an open-source verification-gated LLM cascade router, v0.2.1, pre-GA. Every request opens on the cheapest model; a gate proves the real output before serving it; on gate failure, it escalates exactly one rung. Four moves — Route → Prove → Escalate → Learn — on every request. It sits as an HTTP reverse proxy between your agent and any provider; your agent code doesn't change.

≤10%

served-failure, 95% conf.

82%

served from cheap tier

974

real tasks in the live bound

v0.2.1

pre-GA, open source

What Firstpass is#

Most routers predict quality before any output exists — and when the prediction is wrong, you find out in production with no artifact explaining why. Firstpass proves quality after: open the cheapest model, gate the actual output, serve on pass, escalate one rung on fail. The positioning in one line: proof over prediction.

Prediction-based routing and its limits

Most model gateways route by prediction: a classifier reads the prompt and guesses which model can handle it, before a single token of output exists. If the guess is wrong, you find out in production — and there's no artifact explaining why the request went where it did.

How proof routing works

Firstpass routes by proof. It opens on the cheapest model, then gates the real output — runs your tests, checks a schema, asks a judge, or measures self-consistency. If the output passes, it serves. If it fails, it escalates exactly one rung and gates again. The cheap model handles most traffic; the expensive model is spent only when the cheap one is provably not enough.

The positioning is one line: proof over prediction — verified cascade routing.

Key idea The "proof" is your definition of good — a test suite, a JSON schema, a pass/fail judge, or self-consistency across samples. Firstpass supplies the escalation machinery and the audit; you supply the criterion. That means routing is decided by the thing you already trust, not an opaque classifier — and every decision is explained by a gate result on the actual output, captured in a hash-chained receipt.

Honest novelty The cascade mechanism itself is prior art. Firstpass is not claiming to have invented cheap-model-first escalation. What is new is the combination in the section below — the audit, the calibration, and the guarantee wrapped around the cascade.

Firstpass routes by proof — the cheap model attempts first, the gate decides from the real output, and only a provably failed gate triggers escalation.

The four moves#

Route opens the cheapest rung — no per-prompt prediction. Prove runs your gate on the real output. Escalate goes one rung on gate failure, budget-capped. Learn feeds outcomes back to self-tune the serve threshold. All four run on every request, in order, and your agent sees a single round-trip.

Everything the router does reduces to four moves, in order, on every request.

What each move does

Route

Open the cheapest rung in the ladder. No per-prompt prediction — the cheap model takes the first pass.

Prove

Gate the real output, not the prompt. Built-in checks, your test suite, an LLM judge, or self-consistency.

Escalate

One rung up on gate failure — budget-capped, with cross-provider failover on transport or 5xx errors.

Learn

Deferred outcomes feed back through /v1/feedback; adaptive conformal moves the serve threshold under drift.

How it works follows a single request through all four in detail.

Under the hood Firstpass is an HTTP reverse proxy sitting below your agent. It accepts the same POST /v1/messages (Anthropic dialect) or POST /v1/chat/completions (OpenAI dialect) your agent would send to a provider, swaps only the model field when routing, and forwards everything else verbatim. SSE streaming is buffered so the gate can run, then re-emitted as SSE — your agent sees a normal response. The four moves all happen in that buffer window, transparently.

Every request follows Route → Prove → Escalate → Learn in a single round-trip; your agent points at Firstpass the same way it points at any provider.

What is actually new#

The cascade idea — cheap model first, escalate on failure — is prior art. What Firstpass adds: tamper-evident hash-chained audit, outcome-calibrated serve threshold, zero-retrain model onboarding, predict-to-start + verify-to-serve, and a distribution-free served-failure guarantee earned from a real run.

Five things, and Firstpass is precise about which are novel versus assembled from known parts:

The five novelties, precisely stated

Tamper-evident audit + independent verification. Every decision is a hash-chained receipt; an external auditor re-derives the whole chain from genesis with no access to the proxy or its database. See Receipts & audit.
Outcome-feedback calibration of the serve threshold. Real outcomes — not prompt features — move the threshold that decides serve-versus-escalate. See Adaptive conformal.
Zero-retrain model onboarding. Add a rung as provider/model; there is no classifier to retrain when models change. See Providers.
Predict-to-start, verify-to-serve. A learned start-rung bandit predicts where to begin; the gate still proves what to serve. Prediction sets the entry point; proof sets the exit. See Learned start-rung.
A served-failure guarantee. A distribution-free bound on how often a wrong answer is served, earned from a real run. See The guarantee.

Key idea "Prediction sets the entry point; proof sets the exit." A learned bandit can pick a smarter starting rung for efficiency — but the gate is still what decides whether that output is served. Speed and correctness are separate problems, solved by separate mechanisms, and neither undermines the other.

The five novelties are separable and stackable — audit, calibration, and the guarantee each add value independently and compound when combined.

Who it's for#

Three ideal-fit profiles: teams paying frontier prices on tasks a cheap model could handle; anyone who needs a tamper-evident "why" for every routing decision; builders with an existing definition of "good" — tests, schemas, rubrics — who want routing by their criterion instead of a vendor classifier.

The three ideal-fit profiles

Teams running agents at volume who pay for a frontier model on requests a cheap model could have handled — and want the savings without gambling on quality.
Anyone who has to answer "why did the model say that?" — regulated workflows, incident reviews, or a customer dispute. The receipt is the answer.
Builders who already have a notion of "good" — a test suite, a schema, a rubric — and want routing decided by that definition instead of a vendor's opaque classifier.

Key idea If you already have a test suite that runs in seconds, you already have a gate. Plug it in and every routing decision becomes: "did the cheap model pass your tests?" The receipt that explains the routing decision comes for free — the same artifact that routes also audits.

Firstpass fits best when you have a definition of "good" you can run, and a cost problem you can see in the bill.

Status#

v0.2.1 is shipped and running. The "Shipped & verified" column is available and on by default. The "Next / research" column is gated on further validation — nothing there ships as a default, and every feature is opt-in only.

Firstpass is v0.2.1, pre-GA, and honest about the line between shipped and researched.

Shipped vs. in progress

Shipped & verified	Next / research
Both wire dialects, structured enforce	Elastic verification (validated, phasing in)
All five gate kinds	Per-query difficulty prediction
Start-rung bandit, speculation, failover	Session-level routing
Conformal guarantee + Learn-then-Test	Hosted plane (gated on external audit)
Adaptive threshold, OPE, savings/evals	crates.io publish, 30-day soak
Receipts + export/verify + durable mode
Modes, provider-smoke CI

Under the hood Elastic verification (ADR 0008) is implemented and config-gated but off by default. Enable it by supplying a calibrated λ under escalation.elastic in your config. The conformal guarantee and the served-failure bound hold whether or not elastic mode is on.

The shipped column is available today; the next column is gated on further validation — see Benchmarks for the live data behind the guarantee.

Where to go next#

New here: start with How it works for the full request lifecycle. Evaluating routing quality: go to The guarantee for the live data. Ready to build: the Quickstart gets you routed in minutes.

How it works — the full request flow, observe vs enforce, and the anatomy of one decision.
Gates — the mechanism that makes routing proof-based.
The guarantee — how the served-failure bound is computed, with the real data.
CLI reference — every subcommand, from onboard to verify.