Docs / CLI reference

CLI reference

One binary, firstpass, covers the whole lifecycle — from agentic one-command setup to independent receipt verification. Every subcommand, grouped by what it's for.

One binary: firstpass. Three moves to get running — onboard --apply wires the proxy, doctor confirms it, up starts it. Every other subcommand (trace, explain, savings, evals, verify, calibrate, ope) is for understanding what the router already did and tuning it from evidence.

Install#

Four install paths — script installer (fastest for most), pip/uvx (works wherever Python does), Homebrew (macOS), Docker (isolation). All four land the same binary.

Pick one installer

bash

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/dshakes/firstpass/releases/latest/download/firstpass-proxy-installer.sh | sh   # or:
pip install firstpass          # or: uvx firstpass
brew install dshakes/tap/firstpass-proxy
docker run -p 8080:8080 ghcr.io/dshakes/firstpass
cargo install --git https://github.com/dshakes/firstpass

Common mistake Skipping firstpass doctor after install and going straight to firstpass up. Doctor validates your provider keys, config file, and network connectivity — problems that would cause a confusing crash or silent failure at startup show up cleanly there instead.

Install with one command; run firstpass doctor before anything else to confirm everything is wired.

Lifecycle#

onboard --apply is a one-time agentic setup — it detects your existing provider config and wires the proxy in front of it. After that: doctor to confirm health, up to start routing. offboard cleanly undoes all of it.

Command	Does
`firstpass onboard --apply`	One-command agentic setup — detects your provider config and wires the proxy in. Reversible.
`firstpass offboard`	Reverses onboarding cleanly — removes the proxy from the path.
`firstpass up`	Start the proxy.
`firstpass doctor`	Diagnose config, keys, connectivity, and sandbox readiness.

Key idea firstpass onboard without --apply is a dry run — it shows you exactly what it would change without writing anything. Add --apply to commit. This is the only command that modifies your environment; every other subcommand is read-only against your receipts and config.

bash

firstpass onboard --apply
firstpass doctor
firstpass up

Onboard once, doctor to confirm, up to start — offboard if you ever need to cleanly remove the proxy.

Inspect#

Two concerns: per-request visibility (trace, explain) and aggregate performance (savings, evals, modes). Reach for trace <id> when a specific decision looks wrong; reach for savings to see whether routing is paying off overall.

Command	Does
`firstpass trace <request_id>`	Show the full decision for one request — rungs, gate verdicts, cost, chain link.
`firstpass explain`	Explain why a request routed the way it did.
`firstpass savings`	Net savings vs always-top-rung, with gate cost already subtracted.
`firstpass evals`	Aggregate gate outcomes and served-failure across recorded traffic.
`firstpass modes`	Show the resolved preset for each mode.

Key idea trace <id> gives you the raw receipt — rungs attempted, gate verdicts, per-rung cost and latency, hash chain link. explain translates that same receipt into a sentence: why the cheap rung, why this gate triggered, why escalation fired. Start with explain for the human answer; use trace when you want to verify the numbers or export the receipt.

Use trace for the raw numbers, explain for the plain-English why, savings to see if it's paying off in aggregate.

Audit & calibrate#

export seals receipts to portable JSONL; verify re-derives the hash chain offline (no proxy or database needed); calibrate computes a statistically-sound serve threshold from your own traffic in two flavors: split-conformal or Learn-then-Test.

Command	Does
`firstpass export --out receipts.jsonl`	Seal the receipt log to portable JSONL.
`firstpass verify --file receipts.jsonl`	Re-derive the hash chain from genesis; exit non-zero on tamper. No proxy/DB needed.
`firstpass calibrate --method conformal`	Compute the split-conformal serve threshold + bound.
`firstpass calibrate --method ltt`	Learn-then-Test / RCPS — exact-binomial false-accept control.

Key idea firstpass verify --file receipts.jsonl needs no running proxy and no database — it re-derives the hash chain from genesis purely from the exported JSONL. A third party (auditor, regulator, customer) can independently confirm the receipt log was not tampered with using only the file and the binary.

Under the hood The receipt log is a hash chain: each receipt records a hash of the prior one. Tampering with any record breaks all subsequent links, so verify exits non-zero the moment the chain fails — no need to trust the storage layer.

bash

firstpass export --out receipts.jsonl
firstpass verify --file receipts.jsonl && echo "chain intact"
firstpass calibrate --method conformal --alpha 0.10 --delta 0.05

Common mistake Running calibrate on too little traffic. Both methods produce valid bounds with small samples, but the confidence interval widens — the threshold may be set conservatively. A few hundred calls is workable; a few thousand is comfortable. Check firstpass evals to see how many gate samples you have before calibrating.

verify is the tamper-check (offline, no proxy needed); calibrate sets the serve threshold from your own traffic data, not a guess.

Rehearse & modes#

ope runs off-policy evaluation of a routing policy change against your recorded traffic — see the impact on cost and quality before you touch live traffic. modes shows what each named mode resolves to in practice.

Command	Does
`firstpass ope`	Policy rehearsal — off-policy evaluation of a routing change against recorded traffic, using IPS, SNIPS, or DR estimators. See a policy's effect before you ship it.
`firstpass modes`	List modes and their resolved knobs.

Key idea You don't need new live traffic to test a policy change. ope uses your existing receipts as a dataset and importance-weights them to correct for the fact that your current policy selected which requests ran where. This means you can safely evaluate "what if I added a second rung" or "what if I raised the gate threshold" without any live experiment.

ope gives you evidence before you ship a policy change; modes tells you what the current config actually resolves to.

MCP server#

firstpass mcp starts a stdio MCP server exposing routing decisions, savings, evals, and receipts as tools. This is a receipts-audit surface — it runs alongside the proxy, not on the routed traffic path.

firstpass mcp starts a stdio MCP server so an agent can inspect routing, savings, evals, and receipts as tools. See API & MCP for the tool list.

Under the hood firstpass mcp speaks stdio MCP — your agent connects to it as an MCP client and discovers its tools (trace, savings, evals, and others) automatically. It reads from the same receipt store the proxy writes to, but it is not in the request-handling path: routed traffic never passes through it. Use it when you want an agent to reason about its own routing history or audit its own receipts.

bash

firstpass mcp        # stdio MCP server