Metrics & Observability

Enterprises scrape; they do not poll a dashboard. distil exposes its savings as a Prometheus endpoint and as OpenTelemetry counters — the same numbers over two transports, with no dependency added to the core.

Prometheus

The gateway serves the standard text exposition format at /distil/metrics:

$ distil gateway --port 8788
distil gateway listening on http://127.0.0.1:8788
  dashboard: http://127.0.0.1:8788/distil/dashboard
  metrics:   http://127.0.0.1:8788/distil/metrics  (Prometheus)

$ curl -s localhost:8788/distil/metrics | head
# HELP distil_requests_total Requests processed, by tenant.
# TYPE distil_requests_total counter
distil_requests_total{tenant="widget-co"} 12
# HELP distil_tokens_saved_total Input tokens kept off the wire.
# TYPE distil_tokens_saved_total counter
distil_tokens_saved_total{tenant="widget-co"} 38000

Series

Exported Prometheus series
Metric	Type	Meaning
`distil_requests_total`	counter	Requests processed, per tenant.
`distil_tokens_baseline_total`	counter	Input tokens that would have been sent.
`distil_tokens_sent_total`	counter	Input tokens actually sent.
`distil_tokens_saved_total`	counter	Tokens kept off the wire.
`distil_dollars_saved_total`	counter	Value of saved tokens at the configured input price.
`distil_compression_ratio`	gauge	Fraction saved, 0–1.
`distil_build_info`	gauge	Always 1; carries the `version` label.

Names follow Prometheus convention — distil_ prefix, base units, _total on monotonic counters — so rate() and increase() work without relabeling. Totals are deliberately not exported as separate series: Prometheus sums labelled series itself, and shipping both invites double counting.

The endpoint is authenticated. Series are labelled by tenant, so an open /distil/metrics would publish your customer list to anyone who can reach the port. It sits behind exactly the same gate as /distil/stats: open on loopback for local use, and on any non-loopback bind it requires --admin-token and refuses without it.

scrape_config for a non-loopback gateway
scrape_configs:
  - job_name: distil
    metrics_path: /distil/metrics
    authorization:
      type: Bearer
      credentials: $DISTIL_ADMIN_TOKEN
    static_configs:
      - targets: ["distil-gateway:8788"]

OpenTelemetry

distil emits GenAI-convention spans and, independently, counters. Both are optional and both fail open — if the SDK is absent, or an exporter throws, the request path is untouched.

Exported OpenTelemetry instruments
Instrument	Unit	Mirrors
`distil.requests`	`{request}`	`distil_requests_total`
`distil.tokens.baseline`	`{token}`	`distil_tokens_baseline_total`
`distil.tokens.sent`	`{token}`	`distil_tokens_sent_total`
`distil.tokens.saved`	`{token}`	`distil_tokens_saved_total`

Counters are recorded at the same instrumentation point as the span attributes, so tracing and metrics can never disagree about a request — and they are recorded before the span check, so metrics work with tracing sampled off.

# 1. the API distil records against (this is all the extra pulls in)
$ pip install "distil-llm[otel]"

# 2. an SDK + exporter — what actually ships the data off the box
$ pip install opentelemetry-distro opentelemetry-exporter-otlp

# 3. standard OTel env vars apply — no distil-specific config
$ export OTEL_EXPORTER_OTLP_ENDPOINT=http://collector:4317
$ opentelemetry-instrument distil proxy

Step 2 is not optional. The [otel] extra installs opentelemetry-api only. The API on its own is a no-op recorder — spans and counters are accepted and discarded, and nothing leaves the process until an SDK and exporter are configured. That is deliberate: it is what keeps the extra dependency-light and why a missing or misconfigured OTel install can never break the request path. If you see no data, check that step 2 ran before assuming distil is silent.

What to alert on

The useful alert is not "savings dropped" — savings vary with traffic. It is compression stopped happening at all, which usually means a config or upstream change, not a distil bug:

# compression effectively disabled for 15 minutes
rate(distil_tokens_saved_total[15m]) == 0
  and rate(distil_requests_total[15m]) > 0

Pair it with the decision-equivalence signal from the certificate: savings without equivalence is not a win, and distil is the only layer that reports both.

Zero runtime dependencies: the Prometheus exporter is stdlib-only, and the OTel path imports lazily behind an extra. Neither can fail in a way that reaches your traffic.