LangGraph × Distil

Two routes. pre_model_hook compresses graph state in your own process — no proxy, no network hop. The proxy needs no code change at all beyond a base URL. Pick one; they are not meant to be stacked.

In-process — distil.integrations.langgraph

Duck-typed: this module never imports langgraph or langchain, so it costs nothing to import if you end up not using it. pre_model_hook() plugs straight into LangGraph’s own seam for transforming state right before the model node.

from distil.integrations.langgraph import pre_model_hook

graph = create_react_agent(model, tools, pre_model_hook=pre_model_hook())

# or, manually, inside any node:
from distil.integrations.langgraph import compress_state
state = compress_state(state, verbatim=True)

Both helpers work on a dict-like state (state["messages"]) or an attribute-style state (state.messages); a state with no message list is returned untouched. pre_model_hook returns only {"messages": ...}, so every other field on the graph state is left exactly as LangGraph produced it.

Tool and function messages get the reversible Tier-1 digest, human and system messages are Tier-0 lossless, and assistant messages are never rewritten — a model’s own words are not distil’s to edit. Every digest is byte-exact recoverable through distil_expand. Pass verbatim=True for Tier-0 only, which is the right choice when no recovery tool is wired up.

The same two hooks are also published as the standalone langchain-distil package, listed in LangChain’s own community middleware integrations — see LangChain × Distil if you arrived from there.

Setup — the proxy route

Start the proxy against the provider this SDK talks to, then point the client at it. Nothing else in your code changes — including graphs that call the model directly rather than through create_react_agent.

$ distil proxy --port 8788 --upstream https://api.anthropic.com &

from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(
    model="claude-opus-4-8",
    base_url="http://127.0.0.1:8788",     # ← the only change
)

What actually happens

The proxy intercepts only the compressible paths — /v1/messages, /v1/chat/completions, /v1/responses and the Gemini generateContent routes. Everything else passes through untouched, and your API key travels in the request headers exactly as normal: the proxy never logs or stores it.

Large tool results are replaced by reversible digests carrying a content handle, and the original stays on your machine. The agent can pull any of it back mid-task through the distil_expand tool, so nothing is permanently discarded — which is what lets the compression be aggressive without being a gamble.

Verify it is actually routing

The most common failure is silent: the client never reaches the proxy and everything still works, just uncompressed. Two checks:

$ curl -s localhost:8788/distil/health
{"status":"ok"}

$ distil dashboard          # live savings; zero here means nothing is routing

Every response also carries x-distil-tokens-saved, so a request that went through the proxy is identifiable from its headers alone. The in-process hook has no proxy to check against — distil stats after a run is the equivalent signal.

Full matrix of every supported SDK, including the in-process hooks that need no proxy: Integrations. Distil is a proxy, so anything that lets you set a base URL works — this page is a worked example, not the boundary of support.