Errors, rate limits & retries

Real traffic fails. 429 rate-limit and 529 overloaded are transient — back off and retry, respecting the retry-after header. 5xx server faults are also retryable with a cap. 413 request-too-large is not — shrink the payload. The quiet trap: a call can bill output tokens even when it fails; the model may have started writing before the error was signalled. In a hundred-call agent run, a naive retry loop pays for those partial outputs twice.

On 429 / 529 / 5xx: wait exponential-backoff time (plus jitter) before retrying — never hammer a rate-limited endpoint.

Take the running example: the coding agent makes six calls to handle the “add a total guard and a test” task. On call 4 — edit orders.py, run the test suite — the provider returns a 429. The tool loop is mid-flight. Crashing here leaves a half-applied change; hammering the endpoint without a delay digs the hole deeper.

Which errors to retry

The status code tells you the move — retry the transient ones, fix the rest:

Status	Retry?	Why
`429` rate limit	yes · after `retry-after`	per-minute request / token cap — transient
`529` overloaded	yes · after `retry-after`	Anthropic's overloaded variant — transient
`500`–`503`	yes · capped + backoff	transient server fault
`413` too large	no · shrink the payload	prompt exceeds the size limit
`400` / `401`	no · fix the request	malformed / unauthenticated — re-sending won't help

A 429 body (Anthropic format), then a retry loop that handles all three transient cases:

http

# HTTP 429 — provider tells you exactly when to try again
HTTP/1.1 429 Too Many Requests
retry-after: 30
content-type: application/json

{
  "type": "error",
  "error": {
    "type": "rate_limit_error",
    "message": "Rate limit exceeded. Please retry after 30 seconds."
  }
}

python

import random, time

def call_with_backoff(payload, max_attempts=4):
    for attempt in range(max_attempts):
        resp = session.post(ENDPOINT, json=payload)
        if resp.status_code == 200:
            return resp.json()
        if resp.status_code in (429, 529) or resp.status_code >= 500:
            # respect retry-after if present; otherwise exponential + jitter
            wait = float(resp.headers.get("retry-after", 2 ** attempt))
            wait += random.uniform(0, wait * 0.25)  # spread retries across clients
            time.sleep(min(wait, 60))               # cap at 60 s
            continue
        resp.raise_for_status()  # 400, 401, 413 → surface immediately, don't retry
    raise RuntimeError(f"still failing after {max_attempts} attempts")

Common mistake — failed ≠ free A call that returns 429 or 5xx after the model started generating can still bill output tokens — the provider writes the usage meter for tokens produced before signalling the error. On a streaming response, every chunk received before the error fires has already been billed. Log usage even on error responses. In a six-call agent loop that retries two calls twice, the retry overhead can add 30–50% to the task cost if the model was mid-stream on both.

Idempotency in the tool loop

Retrying the model is safe — it's stateless, so resending the same messages just gets a fresh generation. Retrying a tool is the trap:

Key idea — retry the right layer If call 4's tool_use (“edit orders.py”) reached the model but the response dropped on the wire, a naive replay applies the same Edit twice — a duplicate guard; a POST to an external API submits twice. The tool_use.id on each block exists precisely so you can skip re-executing one you've already run.

What Firstpass does on provider errors

A 5xx/529 on the first-pass rung is read as a gate abstain — the cheap model didn't fail, the provider failed to serve — so Firstpass fails over to another provider or rung and your agent sees a clean response. The receipt records who actually served and the original error. Full shape: Every scenario, end to end → Failover.

Under the hood A failover on a 529 is not an escalation in Firstpass’s sense. The cheap model never got to answer, so there is no quality comparison to make. The router finds another healthy rung at the same tier (or the next tier if no same-tier rung is healthy) and replays the request. The serving attempt that errored is recorded in the receipt but does not count as a rung escalation in the gate logic.

Retry 429 / 529 / 5xx with exponential backoff and a retry-after-aware cap; never retry 413 or 4xx client errors. Failed calls can still bill output tokens — log usage on every response including errors. Track tool_use.id to avoid double-executing tools on retry.