Docs / Configuration

Configuration

Firstpass reads a single firstpass.toml plus a handful of environment variables. This is the full reference — ladder, gates, bandit, speculation, budgets, providers, routes and modes — and the multi-tenant options.

A working config is a [ladder] and one [[gate]]. Every other section has a default and ships off — an optional feature absent from your config file is byte-invisible to callers. Add sections only when you reach for that behaviour.

A working config is a ladder and one gate. Every other section has a default and ships off — add it only when you reach for it.

Config shape#

Two required blocks ([ladder] and [[gate]]), seven optional ones. Point Firstpass at the file with FIRSTPASS_CONFIG (default: ./firstpass.toml).

A minimal config is a ladder and one gate. Everything else has a default. Point Firstpass at a file with FIRSTPASS_CONFIG (default ./firstpass.toml).

toml

# firstpass.toml — a minimal, working config
[ladder]
rungs = ["anthropic/claude-haiku-4-5", "anthropic/claude-sonnet-5"]

[[gate]]
kind = "json-valid"

Key idea Every section not present in the file keeps its default — and the default for every optional section is off. A request through a minimal config is externally identical to a direct provider call: same body, same model reply, same latency path. The config file is how you opt in to routing, gating, and learning; it does not change anything you do not touch.

Two required blocks; every optional section defaults to off and adds no behaviour until you add it.

[ladder]#

The ordered list of models, cheapest first. Firstpass always opens on rungs[0] and escalates up only on a gate failure. max_rungs caps how far a single request may climb.

Key	Type	Default	Meaning
`rungs`	list of `provider/model`	—	Ordered cheapest→most capable.
`max_rungs`	int	`len(rungs)`	Cap on how far a request may climb.

Common mistake Listing the strongest model first inverts the ladder. Firstpass always opens on rungs[0], so that entry must be your cheapest, fastest model. If rungs[0] is anthropic/claude-opus-4-8, every request starts at the top rung and pays the premium rate — there is nothing to escalate to, and the cost benefit disappears entirely.

Order matters: cheap → capable. Firstpass opens at rungs[0] every time and only escalates on a gate failure.

[[gate]]#

Repeatable. Each block adds a gate applied to every rung; all gates must pass for an output to be served from that rung. A gate failure triggers escalation — never an error to the caller.

Repeatable. Each block adds a gate; all gates on a rung must pass. See Gates for the semantics of each kind.

Gate kinds

Key	Applies to	Meaning
`kind`	all	`non-empty` \| `json-valid` \| `schema` \| `subprocess` \| `judge` \| `consistency`
`on_abstain`	all	`fail_open` (default) \| `fail_closed`
`schema`	schema	JSON-Schema subset: `type`, `required`, per-property `type`.
`cmd`, `timeout_ms`	subprocess	Executable + argv, and a timeout (→ abstain).
`model`, `rubric`, `threshold`	judge	Judge model (must differ from candidate), rubric, score bar.
`k`, `threshold`	consistency	Resample count and agreement bar.

Under the hood Gates run after the rung responds but before that response is forwarded to the caller. A gate failure escalates to the next rung silently — the caller never sees the failing output or the gate verdict, only the final served answer. on_abstain = fail_open (the default) treats an inconclusive gate as a pass; fail_closed treats it as a failure and triggers escalation. fail_open is the safe default for gates that may time out or be unavailable.

All gates on a rung must pass; failure means escalation, not an error. fail_open is the safe default for gates that may not always resolve.

[bandit]#

Default-off. When enabled, Firstpass learns which start rung tends to win for a given traffic pattern and opens there instead of always at rungs[0]. Gate evaluation and escalation are unchanged.

The learned start-rung (detail). Default-off; default algorithm UCB1.

toml

[bandit]
enabled = false
algo    = "ucb1"        # ucb1 (deterministic) | thompson (Beta, discounted)

Key idea The bandit controls only the start rung. Gate evaluation, escalation, and the served-failure guarantee are unchanged. A request that opens on rung 1 because the bandit says so still escalates to rung 2 if rung 1's gate fails. Bandit off (the default) means every request opens at rungs[0]. Leave it off until you have enough traffic for meaningful signal.

The bandit picks the opening rung; everything else — gates, escalation, the guarantee — runs identically. Safe to leave off until you have traffic history.

[speculation] & budgets#

Both default-off. Speculation fires the next rung in parallel while the current rung is being graded, trading tokens for latency in the marginal zone. Budget keys cap spend per request, session, and day.

[speculation]

When enabled and the current rung's gate score falls in speculation_band, Firstpass fires the next rung speculatively. If the current rung passes, the speculative call is discarded. If it fails, the next rung's answer is already in flight — saving one full rung latency at the cost of running (and paying for) a speculative call.

toml

[speculation]
enabled          = false
lookahead        = 1
speculation_band = [0.35, 0.65]   # only prefetch in the marginal zone

[budget]
per_request_usd = 0.05
per_session_usd = 1.00
per_day_usd     = 250.00

[budget]

Budget limits are hard ceilings enforced before any upstream call. A request that would exceed any ceiling is not forwarded.

Key	Default	Ceiling scope
`per_request_usd`	`0.05`	All rungs attempted for a single request
`per_session_usd`	`1.00`	Cumulative spend for a session
`per_day_usd`	`250.00`	Daily ceiling

Common mistake Speculation pre-pays for the next rung on every marginal call. If most calls in the speculation_band end up passing the cheap rung anyway, you are paying for two model calls on each one — the latency saving is real, but so is the token cost. Profile your gate's actual flip rate in the band before enabling.

Key idea Elastic verification (escalation.elastic) is a separate opt-in feature — config-gated and off by default. It adjusts verification intensity proportional to doubt on the serving path and requires a calibrated λ. Do not enable it without reading Elastic verification first.

Speculation trades tokens for latency; only useful if your gate flip rate in the band is high enough to justify the extra calls. Budget ceilings reject requests before they start.

[[provider]]#

Repeatable. Each block registers an OpenAI-compatible endpoint as a named provider. Reference it in ladder rungs as name/model-id.

Repeatable. Adds an OpenAI-compatible endpoint as a provider (detail).

toml

[[provider]]
name     = "myllm"
base_url = "https://api.internal.example.com/v1"
dialect  = "openai"      # openai | anthropic | gemini
auth     = "api_key"     # api_key | aws_sigv4 | gcp_oauth

Key	Meaning
`name`	Identifier used as prefix in ladder rungs: `name/model-id`
`base_url`	Base URL of the OpenAI-compatible endpoint
`dialect`	Wire format: `openai` \| `anthropic` \| `gemini`
`auth`	Auth method: `api_key` \| `aws_sigv4` \| `gcp_oauth`

One block per custom provider; reference it in the ladder as name/model-id. See Providers for API key wiring.

[[route]] & modes#

Routes bind a URL pattern to a ladder and a default routing_mode. The per-request x-firstpass-mode header overrides the route default. Six modes from full passthrough (observe) to always-top (max).

Routes let different traffic use different ladders and a per-route default routing_mode (overridden by the x-firstpass-mode header — see mode precedence).

toml

[[route]]
match        = "/v1/chat/completions"
routing_mode = "cost"        # observe|cost|balanced|quality|latency|max

Routing modes

Mode	Behaviour
`observe`	Byte-passthrough; no routing decisions, receipts recorded only
`cost`	Cheapest rung that clears the gate
`balanced`	Balance cost against quality
`quality`	Prefer quality; climb earlier
`latency`	Prefer lowest latency
`max`	Pin to the top rung (`rungs[len-1]`)

Routes scope a ladder to a URL path; modes control how aggressively Firstpass optimises within that ladder. See Routing & escalation for full semantics.

Environment variables#

Seven variables control listen address, config file, and operating mode. Start with FIRSTPASS_MODE=observe — it records receipts and passes every request through unchanged.

Variable	Values	Meaning
`FIRSTPASS_MODE`	`observe` \| `enforce`	Shadow vs live. Start with observe.
`FIRSTPASS_CONFIG`	path	Config file (default `./firstpass.toml`).
`FIRSTPASS_DB`	path	Receipt / state store location.
`FIRSTPASS_BIND`	`host:port`	Listen address (default `:8080`).
`FIRSTPASS_RECEIPTS`	`best_effort` \| `durable`	Never-drop receipts (detail).
`FIRSTPASS_MODE_PROFILE`	mode name	Env-level default mode.
`FIRSTPASS_MAX_CONCURRENCY`	int	Max in-flight upstream requests.

Under the hood FIRSTPASS_MODE=observe is a byte-passthrough: every request is forwarded verbatim to the upstream, receipts are written, but no routing or gate decisions are made. The model and the caller see no difference from a direct connection — it is the zero-risk way to instrument an existing agent. Switch to enforce once you have validated your ladder and gate config against observe-mode receipts.

FIRSTPASS_MODE=observe is shadow mode — zero behaviour change, full receipts. Switch to enforce to activate routing.

Multi-tenant#

Opt-in and default-off. When enabled: Argon2id keyed auth, AES-256-GCM key custody, per-tenant rate limits. Not GA — pending external security review. Do not use for tenant isolation in production yet.

Multi-tenancy is opt-in and default-off. When enabled it provides Argon2id keyed auth, AES-256-GCM key custody, and per-tenant rate limits.

Pre-external-review The multi-tenant path is implemented but not GA — it is pending external security review. Do not rely on it for tenant isolation in production until that review lands. See ADR 0004.

Multi-tenancy is off until you configure it and not GA until the external security review lands. Do not rely on it for isolation in production today.