Docs / Configuration

Configuration

Firstpass reads a single firstpass.toml, named explicitly by FIRSTPASS_CONFIG, plus a handful of environment variables. This is the full reference — routes, ladders, gates, escalation, budgets, providers, and modes — and the multi-tenant options.

Config shape#

A minimal config is one [[route]]: match the traffic, pick a mode, list the ladder. The built-in gate ids non-empty and json-valid need no extra block — only custom gate kinds (schema, subprocess, judge, consistency, decision) do. Point Firstpass at the file with FIRSTPASS_CONFIG=/path/to/firstpass.toml; there is no default path, so leaving it unset means routing runs with no config at all, not a config named firstpass.toml in the working directory.

# firstpass.toml — a minimal, working config
[[route]]
match  = {}
mode   = "enforce"
ladder = ["anthropic/claude-haiku-4-5", "anthropic/claude-sonnet-5"]
gates  = ["non-empty"]

[[route]]#

Each [[route]] binds a slice of traffic to a mode and a ladder. Routes are matched top to bottom; the first one whose match claims the request wins.

[[route]]
match = { agent = "claude-code", subagent = ["test-runner", "explore"] }
mode  = "enforce"
ladder = ["anthropic/claude-haiku-4-5", "anthropic/claude-sonnet-5"]
gates  = ["non-empty", "judge"]
deferred_gates = ["consistency-check"]
routing_mode = "quality"   # optional — observe|cost|balanced|quality|latency|max

# catch-all last: an empty match claims everything above it did not
[[route]]
match = {}
mode  = "observe"
ladder = ["anthropic/claude-opus-4-8"]

Gates#

A [[gate]] block has an id plus exactly one of cmd (subprocess), judge, consistency, schema, or decision — there is no kind field; the kind is inferred from which one is set, and setting zero or more than one is rejected at parse.

[[gate]]
id     = "shape"
schema = { type = "object", required = ["total", "items"] }
on_abstain = "fail_closed"

[[gate]]
id  = "unit-tests"
cmd = ["python", "run_tests.py"]
timeout_ms = 5000

[[gate]]
id    = "judge"
judge = { model = "anthropic/claude-opus-4-8", threshold = 0.7, rubric = "correct and complete" }

decision — TypeSafe Jev (experimental)

The decision gate calls TypeSafe's Jev, a cheap external decision model, and asks a single yes/no "does this response satisfy the request?" question — pass iff P(yes) >= threshold. It is orders of magnitude cheaper than a frontier LLM judge, and unlike the judge gate it is not a native LLM call through your own provider credentials.

[[gate]]
id       = "decision"
decision = { provider = "typesafe", model = "jev-latest", threshold = 0.6 }
# api_key_env defaults to TYPESAFE_API_KEY; base_url defaults to https://api.typesafe.ai

[escalation]#

[escalation]
max_rungs_per_request = 3
session_promotion = { after_failures = 3, window = "30m" }
speculation = 0
# speculation_band = [0.3, 0.7]   # marginal zone; requires a warm bandit
# serve_threshold = 0.5           # omit to serve on a plain gate Pass

[escalation.bandit] — learned start rung

Default-off. When set, Firstpass learns which start rung tends to win for a given traffic pattern and opens there instead of always at ladder[0]. Gate evaluation and escalation are unchanged — a request that opens on rung 1 because the bandit says so still escalates to rung 2 if rung 1's gate fails (detail).

[escalation.bandit]
algorithm         = "thompson"   # ucb1 (default, deterministic) | thompson (Beta, stochastic)
discount          = 0.99         # (0,1]; 1.0 = no forgetting (default)
min_observations  = 50           # cold-start guard: below this, every request starts at rung 0
exploration       = 1.0          # UCB1 constant c (Auer et al. 2002); ignored by thompson

[escalation.prior] — decision-model prior (experimental, default-off)

A pre-generation prior from TypeSafe's Jev — a per-query estimate of which ladder rung is the least capable one that fully handles the request, blended into the start-rung bandit as a Beta pseudo-observation. This is a prior, not a verifier: the gate still runs on every served attempt exactly as without it, and a confident prior can only move where the ladder starts, never what gets served (ADR 0013).

[escalation.prior]
provider    = "typesafe"                # the only accepted value today
model       = "jev-latest"              # default shown
api_key_env = "TYPESAFE_API_KEY"        # default shown
base_url    = "https://api.typesafe.ai" # default shown
timeout_ms  = 150                       # default shown; a timeout fails open (no prior, no cost)
strength    = 10                        # Beta pseudo-count weight vs. real gate-verdict counts
rungs = [
  "rung 0 is the least capable tier that fully handles this request",
  "rung 1 is the least capable tier that fully handles this request",
]  # one line per ladder rung — length must match the enforce route's ladder

OpenJev (local)

Both [escalation.prior] and the decision gate speak the same POST /v1/systemone contract, so either can point at a locally-run OpenJev (Apache-2.0, DiffusionGemma 26B-A4B — razorback16/openjev) instead of TypeSafe's hosted Jev — this is what the replay above measured.

# Apple Silicon, ~16 GB unified memory — binds 127.0.0.1:8080
OPENJEV_BACKEND=mlx python -m openjev
# NVIDIA, 24 GB+ VRAM:
# docker compose up

[escalation.prior]
provider    = "typesafe"              # still the only accepted value — OpenJev speaks the same wire contract
base_url    = "http://127.0.0.1:8080"
api_key_env = "TYPESAFE_API_KEY"      # the client always sends a bearer token — set any placeholder, e.g. TYPESAFE_API_KEY=local

api_key_env is required even against a keyless local server — the client always attaches bearer_auth, so a missing env var disables the prior fail-open rather than sending an empty token (crates/firstpass-proxy/src/run.rs, crates/firstpass-proxy/src/gate.rs). Point the decision gate's base_url at the same server to use OpenJev as the verifier instead of the prior.

[budget]#

[budget]
per_request_usd = 0.50
per_session_usd = 10.00
per_day_usd     = 250.00
on_exhausted    = "serve_best_attempt"   # serve_best_attempt (default) | error

[[provider]]#

Repeatable. Adds a provider endpoint, keyed by id (detail).

[[provider]]
id          = "groq"
dialect     = "openai"             # openai | anthropic | gemini — wire format only
base_url    = "https://api.groq.com/openai"
api_key_env = "GROQ_API_KEY"       # omit entirely for a keyless local endpoint
auth        = "api_key"            # api_key (default) | aws_sigv4 | gcp_oauth

[[price]]
model           = "myllm/local-7b"
input_per_mtok  = 0.0    # USD per 1M input tokens
output_per_mtok = 0.0    # USD per 1M output tokens

Routing modes#

[[route]]
match        = { task_kind = "code_edit" }
mode         = "enforce"
ladder       = ["anthropic/claude-haiku-4-5", "anthropic/claude-sonnet-5"]
gates        = ["non-empty"]
routing_mode = "cost"        # observe|cost|balanced|quality|latency|max

Progressive delivery#

Enforce used to be all-or-nothing, and observe couldn't answer the question it existed for. These three compose into shadow → ramp → guard. Design rationale is in ADR 0009.

Shadow — get your own number first

On a sample of observed requests, Firstpass runs the ladder off the hot path and records what it would have served and what that would have cost. The response your caller gets is untouched — shadow work starts only after it has already gone back. It makes real model calls, which is why there is no default sample rate and the daily ceiling is required rather than optional.

[route.shadow]
sample_rate     = 0.10      # fraction of observed requests scored
max_usd_per_day = 5.00      # hard ceiling; shadow stops and records that it stopped

Rollout — ramp a slice, not a guess

Enforce only part of a route's matched traffic. Bucketing is a deterministic function of a stable key, so a conversation never flips arm mid-thread. That matters for more than user experience: the published served-failure bound is computed over the population that was served, and a per-request coin flip would make that population an unstable sample of nothing. Raising percent never ejects a session that was already enforcing.

[route.rollout]
percent = 5.0
key     = "session"         # session (default) | request | tenant

Guardrail — defend the number you published

Watches the same Hoeffding bound that earns the published guarantee, over a trailing window of resolved outcomes from /v1/feedback. Never gate verdicts: a guardrail scored on Firstpass's own opinion would keep enforcing exactly when that opinion had drifted.

# top-level Config field — NOT nested inside [guardrail]
guardrail_cooldown_secs = 3600

[guardrail]
alpha    = 0.10             # the served-failure target being defended
delta    = 0.05             # confidence; matches the published bound
window   = 500              # trailing resolved outcomes
min_n    = 200              # never act on less evidence than this
action   = "demote"         # demote | alarm

Environment variables#

Multi-tenant#

Multi-tenancy is opt-in and default-off. When enabled it provides Argon2id keyed auth, AES-256-GCM key custody, and per-tenant rate limits.