Firstpass v0.2.1
GitHub
DocsGuaranteeBenchmarksCLIGitHub

Docs / Quickstart

Quickstart

The fastest way to understand Firstpass is to watch it prove one decision. In about a minute, with no API keys, you'll run a real routing loop — cheap model first, gate the output, escalate on failure — and read the tamper-evident receipt it seals. Then you'll point your own agent at it, safely.

Run two commands with no API keys to watch Firstpass prove one decision end to end. Then point your own agent at it in observe mode — requests forward byte-identical to your provider while Firstpass writes shadow receipts off the hot path. Flip to enforce when the numbers convince you.
~60s
to your first receipt
0
API keys for the demo
1
env var to leave, ever

1 · See it prove one decision#

Two commands, no keys, no config. The demo stands up a mock upstream and drives a request through the full loop — cheap model first, gate the output, escalate on failure — then prints the hash-sealed receipt. You're watching the whole product in miniature.

The demo needs no keys and talks to no provider. It stands up a mock upstream, drives one request through the full loop — open the cheap rung, gate the output, escalate one rung when the gate fails, serve the passing answer — and prints the receipt to your terminal. All you need is git and a Rust toolchain (rustup.rs).

bash
git clone https://github.com/dshakes/firstpass && cd firstpass
cargo run -p firstpass-proxy --example demo

You don't have to install anything or read the config first. The point is to see the shape of a decision before you learn the parts — everything after this is that same loop, made real.

Clone and run — no keys, no config; the demo drives the full routing loop and prints the tamper-evident receipt in about 60 seconds.

2 · Read the receipt#

The receipt is Firstpass's proof artifact — a hash-chained record of every rung tried, each gate verdict, cost, and counterfactual. Not a log line: the complete, auditable account of one decision. Read it top to bottom and you've read the architecture.

The demo prints a receipt like the one below. It is the whole decision, in one record: which rungs were tried, how each gate voted, what it cost against the always-expensive counterfactual, and the hash link that makes the log tamper-evident. This artifact — not a prediction, not a log line — is what Firstpass produces for every request.

json · one receipt
{
  "seq": 1,
  "mode": "balanced",
  "attempts": [
    { "rung": 0, "model": "mock/cheap",  "gate": "json-valid", "verdict": "fail" },
    { "rung": 1, "model": "mock/strong", "gate": "json-valid", "verdict": "pass", "served": true }
  ],
  "served_rung": 1,
  "cost_usd": 0.0004,
  "counterfactual_usd": 0.0011,
  "prev_hash": "genesis",
  "hash": "b9c4…b1a7"
}

Read it top to bottom and you've read the product: rung 0 answered, the gate rejected it, so Firstpass escalated to rung 1, which passed and was served. seq 1 and prev_hash "genesis" mark this as the first link in the chain; the next receipt will carry this hash as its prev_hash, and so on — which is what lets firstpass verify re-derive the entire history from scratch and catch any edit. Receipts & audit takes the record apart field by field.

Every request produces one receipt: attempts, verdicts, cost vs counterfactual, and a hash link that lets firstpass verify re-derive the full chain from scratch and catch any edit.

3 · What just happened#

Four moves in order: Route → Prove → Escalate → Learn. The demo ran all four on one mock request. Every real request goes through the same loop — the pages linked below take each move apart.

That one decision is the whole system in miniature — the four moves from the Overview, now watched running. Firstpass does exactly these four things, in order, on every request:

Route
Open the cheapest rung in your ladder. No classifier guesses from the prompt — the cheap model simply takes the first pass. Routing & escalation →
Prove
Gate the real output. In the demo that's json-valid; in production it's your schema, your test suite, an LLM judge, or self-consistency. Gates →
Escalate
One rung up on gate failure — budget-capped, with cross-provider failover on transport or 5xx errors. The strong model is spent only when the cheap one provably isn't enough. How escalation is bounded →
Learn
Outcomes feed back through /v1/feedback and move the serve threshold, so the served-failure rate stays bounded even as traffic drifts. The guarantee →

How it works follows a single request through all four in detail, and Every scenario, end to end traces a real one hop by hop.

Route → Prove → Escalate → Learn, every request, sealed in one tamper-evident receipt — that's the whole system.

4 · Point your own agent at it#

Install Firstpass, run firstpass onboard (dry-run first, then --apply), and start collecting receipts. Fresh installs start in observe mode — every request forwards byte-identical to your provider while Firstpass builds a shadow picture. Flip to enforce when you've seen enough.

Install the proxy

Now do it for real. Firstpass is agent-first by construction — its primary user is an agent, so the first tab is the one you hand to yours instead of running yourself. Every tab lands the same single static binary with no runtime dependencies, and every one is live today.

prompt · paste into Claude Code, Codex, Cursor…
Read https://dshakes.github.io/firstpass/llms.txt, then install Firstpass and
route this project through it. Run `firstpass onboard` first and show me the
plan before you apply anything. Keep it in observe mode; don't touch my keys.
bash
curl --proto '=https' --tlsv1.2 -LsSf \
  https://github.com/dshakes/firstpass/releases/latest/download/firstpass-proxy-installer.sh | sh
bash
uvx firstpass            # run without installing
pip install firstpass    # or install it
bash
brew install dshakes/tap/firstpass-proxy
bash
docker run -p 8080:8080 -e FIRSTPASS_BIND=0.0.0.0:8080 \
  ghcr.io/dshakes/firstpass:latest
bash
cargo install --git https://github.com/dshakes/firstpass firstpass-proxy
Why an agent tab The install surface is machine-readable on purpose: llms.txt states the whole setup in prose an agent can act on, firstpass onboard prints a plan before it touches anything, and firstpass doctor is a self-check the agent can read the exit code of. An agent can wire and unwire Firstpass with no human in the loop — and once it's running, firstpass mcp serves the receipts back over MCP so the agent can audit its own routing. See API & MCP.

Wire your agent in

Then let Firstpass wire itself in. onboard detects your provider config, starts the proxy, points your agent's base URL at it, and checks itself — --apply writes the changes, and bare onboard is a dry run that shows you exactly what it would do first:

bash
firstpass onboard          # dry run — shows every change
firstpass onboard --apply  # wire it in
firstpass doctor           # confirm config, keys, connectivity
Safe by default Fresh installs start in observe mode: every request is forwarded to your provider unchanged and a receipt is written off the hot path. Zero behavior change — you get the audit trail and the savings estimate before you let Firstpass route a single request. When the shadow numbers convince you, switch to enforce and it starts serving from the cheap rung. Observe vs enforce →
Key idea Observe mode is a byte-identical passthrough on the wire — Firstpass forwards your request body verbatim, swapping nothing, and writes the receipt asynchronously. Your agent sees the same response it would have gotten from the provider directly. There is no behavior difference until you explicitly switch to enforce.
Common mistake Jumping straight to enforce before reading the observe-mode receipts. Spend a session or two in observe first — the shadow data shows exactly how often the cheap rung would have cleared the gate on its own, so you know what to expect before any real traffic routes differently.

Configure your ladder

Enforce mode is driven by a small TOML ladder — which models to try, in order, and which gates must prove the output. You do not have to write one: firstpass onboard asks these same three questions on a terminal and writes the file itself, then hands it to the proxy it starts. The builder below is the same generator, so you can see the exact file first — or skip straight to onboard. Configuration covers matching, budgets, and per-route gates.

Non-interactive too Nothing blocks in a pipe: firstpass onboard --apply --provider anthropic --shape code --mode observe answers all three up front, and with no flags and no terminal it takes the do-nothing defaults and prints which ones it used. Re-running never overwrites an existing firstpass.toml.
toml · firstpass.toml

Or let your agent write it The builder covers the common shapes; your repo may not be a common shape. Hand this to your agent instead — it can read what your code actually does and pick gates the builder can’t infer: “Read https://dshakes.github.io/firstpass/llms.txt, look at how this repo tests itself, and write me a firstpass.toml whose gates are my real test command. Start it in observe mode.”
bash
FIRSTPASS_MODE=enforce FIRSTPASS_CONFIG=./firstpass.toml firstpass up

Your keys stay yours — Firstpass is bring-your-own-key, adds no markup, and redacts keys from every log and receipt. And leaving is always one line, by design:

bash
firstpass offboard   # or just: unset ANTHROPIC_BASE_URL
Install → onboard → observe first (byte-identical, receipts off the hot path) → enforce when ready. One env var to leave, ever.

Where to go next#