ASGI Middleware × Distil

Every other integration on this site is a client pointed at the proxy. This one is for the opposite shape: you host the LLM-facing endpoint — a FastAPI/Starlette/Litestar backend that builds a request and forwards it to Anthropic, OpenAI, or Gemini itself. Wrap your ASGI app once; its own outbound call sees a compressed body.

Setup

DistilMiddleware is pure ASGI — it never imports Starlette or FastAPI — so it wraps any ASGI 3 application, whichever framework built it.

from fastapi import FastAPI
from distil.integrations.asgi import DistilMiddleware

app = FastAPI()

@app.post("/v1/messages")
async def proxy_to_anthropic(request):
    # request.body() here already has compressed messages —
    # the middleware rewrote it before FastAPI's routing even saw it.
    ...

app = DistilMiddleware(app)                 # wrap once, at the bottom of the file
# app = DistilMiddleware(app, verbatim=True)  # Tier-0 lossless only

Under Starlette/FastAPI, wrap in main.py after the routes are declared (middleware wraps the whole ASGI callable, not a per-route decorator). Under a bare ASGI server, wrap whatever callable you pass to it: uvicorn.run(DistilMiddleware(app)).

What actually happens

The middleware inspects only POST requests whose path matches a compressible shape: /v1/messages, /v1/chat/completions, /v1/responses, or a Gemini generateContent route — the exact same detection distil proxy uses, imported rather than reimplemented. Everything else, including every other verb, passes through with the original ASGI receive untouched, at zero cost.

For a match, it drains the body, runs the same reversible compression the sidecar proxy uses (adapters.anthropic.compress_messages for the Anthropic/OpenAI shape, adapters.gemini.compress_generate_request for Gemini's), fixes up content-length, and replays the compressed bytes to your app as a normal http.request event. Digest handles land in the same on-disk restore store the proxy uses, so a handle minted here expands anywhere — distil_expand, the MCP server, or a later request through the proxy.

Fail-open by construction. A non-JSON body, an unrecognized shape, or a compression error all forward the original bytes unchanged rather than breaking the request. A body larger than the proxy's own size guard is forwarded uncompressed rather than buffered in full.

When to reach for this instead of the proxy

Use the sidecar proxy (distil proxy) whenever you can — zero code, works for any client. Reach for DistilMiddleware specifically when your own backend is the thing constructing the provider request, so there is no client base_url to redirect: the compression has to happen inside your process, on the way out.

Full matrix of every supported SDK, including the in-process hooks that need no proxy: Integrations.