Structured outputs & JSON

Asking the model to “return JSON” in the prompt is fragile — you get prose, markdown fences, or truncated objects. Two reliable paths: tool-as-schema (define a single tool whose input_schema is your target shape, force the model to fill it via tool_use) and provider structured/JSON mode (OpenAI’s response_format). Either way, the data arrives schema-validated in a predictable field. Token billing is unchanged — same rates, shaped output.

Tool-as-schema or JSON mode enforces shape at generation time — no prose wrapping, no code fences, and a truncation is signaled by stop_reason rather than a silent parse failure.

When an agent needs machine-readable output — a patch plan, a routing decision, a list of findings — parsing freeform model text is the wrong tool. Two failure modes are common: the model wraps the JSON in a prose sentence or a markdown code fence and the parser breaks; or max_tokens cuts the output mid-object and you receive invalid JSON. Both are avoidable by using the API’s structured-output surface instead of prompt engineering.

Tool-as-schema — the reliable cross-provider approach

Define a single tool whose input_schema is the exact JSON shape you want. Set tool_choice to force the model to call only that tool. The response arrives as a tool_use block — the structured data sits in content[0].input, already validated against the schema, with no freetext prose mixed in. Works on Anthropic and OpenAI.

Here the coding agent plans the POST /orders patch as a typed object before writing a single line:

json

// ① REQUEST — one tool whose input_schema IS the target shape
{
  "model": "claude-haiku-4-5",
  "max_tokens": 512,
  "tool_choice": { "type": "tool", "name": "emit_patch_plan" },
  "tools": [{
    "name": "emit_patch_plan",
    "description": "Output the patch plan as structured data.",
    "input_schema": {
      "type": "object",
      "properties": {
        "file":      { "type": "string" },
        "old":       { "type": "string" },
        "new":       { "type": "string" },
        "test_name": { "type": "string" }
      },
      "required": ["file", "old", "new", "test_name"]
    }
  }],
  "messages": [{
    "role": "user",
    "content": "Plan the fix for POST /orders: reject requests missing ‘total’."
  }]
}

// ② RESPONSE — structured data in content[0].input, stop_reason: tool_use
{
  "stop_reason": "tool_use",
  "content": [{
    "type": "tool_use",
    "id":   "toolu_03C",
    "name": "emit_patch_plan",
    "input": {
      "file":      "orders.py",
      "old":       "body = await request.json()",
      "new":       "body = await request.json()\n    if ‘total’ not in body:\n        raise HTTPException(422, ‘total required’)",
      "test_name": "test_rejects_missing_total"
    }
  }],
  "usage": { "input_tokens": 312, "output_tokens": 71 }
}

Read response.content[0].input directly as your parsed object. You are using the tool mechanism as a schema, not driving a loop — there is no tool_result to send back.

OpenAI structured / JSON mode

OpenAI’s POST /v1/chat/completions has a response_format field with two variants. json_object guarantees syntactically valid JSON but doesn’t enforce a shape. json_schema with strict: true enforces a full JSON Schema at generation time — no invalid shapes reach you. Anthropic’s POST /v1/messages has no response_format field; use the tool-as-schema approach above.

json

// OpenAI — strict JSON Schema enforcement at generation time
{
  "model": "gpt-4o",
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name":   "patch_plan",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "file":      { "type": "string" },
          "old":       { "type": "string" },
          "new":       { "type": "string" },
          "test_name": { "type": "string" }
        },
        "required": ["file", "old", "new", "test_name"],
        "additionalProperties": false  // required when strict: true
      }
    }
  },
  "messages": [{ "role": "user", "content": "Plan the fix for POST /orders …" }]
}
// response: choices[0].message.content is a JSON string; JSON.parse() it once
// finish_reason: "stop" on success, "length" on truncation (see below)

Common mistake Parsing the output without first checking stop_reason (Anthropic) or finish_reason (OpenAI). If the model runs out of tokens mid-JSON — stop_reason: "max_tokens" / finish_reason: "length" — the output is a truncated fragment that throws on parse. The fix is to check the stop reason before parsing and either retry with a higher max_tokens or surface the error cleanly. The response anatomy is covered in Module 1.

Key idea Structured output is a constraint on how the model writes output tokens — it is not a different kind of call or a different billing unit. The same per-Mtok input and output rates apply. The schema definition inside input_schema or json_schema is part of the request and is billed as input tokens, the same as any tool definition in Module 4.

Under the hood With tool_choice: {type: "tool", name: "…"} Anthropic guarantees the model calls exactly that tool and nothing else — no leading prose, no alternative tool. Without tool_choice, the model might emit a text block first or pick a different tool. For structured-output use, always pin tool_choice.

Structured output works by using the API’s own schema surface — a single forced tool on Anthropic, response_format: json_schema on OpenAI — so the data arrives validated in a known field. Always check stop_reason before parsing: a max_tokens cut produces a truncated fragment, not valid JSON.