Streaming

Session replies arrive as three named server-sent events on GET /v1/sessions/{id}/events: message_delta, message_completed and message_error. Provider tokens flow through the control plane and Redis pub/sub to connected clients with no buffering in between, so words appear as the model writes them. You render deltas as they land and treat message_completed as the final text.

Token streamingSession replies flow provider to client with no buffering, except tool-using turns, which return one finished message.

Tool-free turnTool-using turnProvider fails mid-stream

Swipe sideways to pan.FitShow steps

⏮◀▶ Play▶

Scrub through steps1 / 5

1. The provider streams

Tokens arrive from the provider as the model writes them.

• Outside PAL
• PAL Runtime services
• Data stores

Sessions only

This contract covers interactive sessions. Headless microVM runs stream step_started, step_completed and step_failed events, not token deltas. Tool-free turns stream every token; tool-using turns arrive as one message_completed.

The three events

A turn produces events in this order. turnId is stable across the turn.

Event	When	Payload
message_delta	Once per token chunk. Not emitted for tool-bearing turns	sessionId, turnId, seq (monotone per turn, starts at 1), delta
message_completed	Exactly once, after all deltas	sessionId, turnId, text (full assembled reply), usage (tokensIn, tokensOut, costUsd)
message_error	The LLM call fails after at least one delta was sent	sessionId, turnId, error. Discard the partial reply for this turnId

On the wire:

event: message_delta
data: {"sessionId":"...","turnId":"...","seq":1,"delta":"Hello"}

event: message_delta
data: {"sessionId":"...","turnId":"...","seq":2,"delta":", world"}

event: message_completed
data: {"sessionId":"...","turnId":"...","text":"Hello, world","usage":{...}}

Legacy clients that subscribe to agent.message keep receiving it; the named events are additive.

Failure before the first delta

If the provider fails before any delta is sent, no message_error is emitted. The turn ends with a legacy agent.message (a canned “give me a sec” reply) plus session.status_idle, which streamMessage() ignores. Put a timeout around the iterator.

Read the stream with the SDK

sessions.streamMessage() is an async iterator. It reorders out-of-order seq values, ignores unknown event kinds, and throws a typed MessageStreamError on message_error.

import { PalClient } from "@pal-runtime/sdk";

const client = new PalClient({ apiKey: "...", baseUrl: "http://localhost:8080" });

for await (const event of client.sessions.streamMessage(sessionId, {
  content: "Summarise the last quarter in 3 bullets.",
})) {
  if (event.type === "delta") process.stdout.write(event.delta);
  else if (event.type === "completed") console.log("\nCost:", event.usage.costUsd);
}

Events are a union: { type: "delta"; delta: string } or { type: "completed"; text: string; usage: { tokensIn; tokensOut; costUsd } }.

How it works underneath

The control plane calls the provider in streaming mode (streamWithFailover). Each chunk publishes a message_delta to the Redis session channel, and the SSE handler fans it out. The provider request carries an Idempotency-Key, so a crash retry dedups at the provider.

If the primary provider fails before any delta, the failover chain retries on the next provider (Models and routing). If it fails after a delta, you get message_error. The dashboard chat and the webchat widget (/widget.js) consume these same events. For run events (not session tokens), see Durable execution.