Relay docs

Telemetry & dashboards

Every request through Relay is recorded: model, provider, tokens, cost, latency, status, workspace, API key, end user and more. That data powers the in-app dashboards, cost attribution, budgets and alerting.

#Storage

Relay writes one row per observation — every model call, agent run, tool call, retrieval and workflow step — to relay_observation in ClickHouse, and scores to relay_score, whenever ClickHouse:ConnectionString is set. Both tables are created, and new columns added, automatically at startup. The dashboard, the Telemetry page and every Observability page read from them; see Observability for traces, sessions, users, scores, evaluators and datasets.

OTLP export to a ClickStack collector is independent (Gateway:Otlp:Enabled), so HyperDX can show the same traces. To read the usage dashboard from the collector's otel_traces table instead, set Telemetry:DashboardSource = otel. The old Telemetry:Backend switch is no longer read at all: Relay always writes its own store, because traces, captured bodies and scores cannot live in otel_traces.

Builds before the observability store wrote a relay_usage table; Application.Database.Log/schema/05_backfill_relay_usage.sql copies that history into relay_observation.

#OpenTelemetry export

OtlpExporter is a BCL-only OTLP/HTTP JSON exporter (no OTel SDK). It's gated by Gateway:Otlp:Enabled and sends to Gateway:Otlp:Endpoint (default http://localhost:4318/v1/traces), deriving the logs/metrics endpoints unless Gateway:Otlp:LogsEndpoint / MetricsEndpoint override them, as service Gateway:Otlp:ServiceName (default relay-gateway). It emits:

  • Traces — one span per observation, with the same trace, span and parent ids as the panel's trace tree, and gen_ai.* / relay.* attributes (model, tokens incl. cached and reasoning, cost, latency, status, workspace, key, user, session, environment, prompt, cache hit, attempts, fallback).
  • Logs — one record per observation, correlated to its span.
  • Metrics — cumulative counters (relay.requests, relay.tokens, relay.errors, relay.cost.usd) and a relay.latency gauge holding the running average, for generation and embedding observations, every Gateway:Otlp:MetricsIntervalSeconds (default 30).

ClickStack/HyperDX authenticates OTLP ingestion — set Gateway:Otlp:ApiKey to the HyperDX Ingestion API Key, or the collector returns 401: missing or empty authorization header.

This exporter is separate from the OpenTelemetry SDK pipeline that Application.ServiceDefaults adds for the Aspire dashboard: the SDK exports generic ASP.NET Core / HttpClient / runtime telemetry only when OTEL_EXPORTER_OTLP_ENDPOINT is set, as relay-api / relay-panel, so the two never mix in HyperDX.

#In-app dashboards

The panel reads telemetry via api/Telemetry (configured, summary, timeseries, top-models, recent, attribution):

  • Dashboard — usage for the active workspace: KPI tiles (requests, cost, tokens, average cost per request, p50/p95 latency, error rate and count, traces, end users, sessions, cache-hit rate, p95 time to first token), a time series switchable between requests / tokens / cost / errors, top models, spend by provider, spend by app, prompt-vs-completion token split, top end users, score averages, spend by agent, recent requests, and a cost attribution table groupable by model, provider or app.
  • Telemetry — a detailed request log with cost/usage attribution grouped by team, app (API key), model, provider, end user, agent, prompt or environment, and a Request ID filter that opens the full trace.

#Filtering

The dashboard filters by model, provider, app (API key) and status. Filter values come from the catalogue rather than from observed traffic, so a model that exists but has not been called yet is still selectable — otherwise "no traffic" and "not an option" would look identical.

End-user attribution comes from the X-Relay-User-Id header (or the OpenAI user field) that apps send. The End users page, the dashboard's top end users and the attribution tables group by it; requests from apps that send nothing are attributed to their API key only.

#Workspace scoping

Telemetry queries are hard-scoped to one workspace. The active workspace is taken from the X-Workspace-ID header the panel sends, it overwrites any teamId in the request body, and a request with no workspace returns an empty result rather than an unscoped one.

This matters more here than for the SQL entities: ClickHouse holds no row-level tenancy and there is no query filter behind these endpoints, so that scoping is the entire boundary between one workspace's usage and another's. The public /v1/telemetry/* API applies the same rule from the calling key's workspace.

#Budgets & alerting

  • Budgets — BudgetRefreshService recomputes each workspace's spend, and each end user's, from telemetry every Gateway:BudgetRefreshMinutes (default 5) for hard-stop enforcement. Only workspace-level rows are refreshed: the query filters by workspace, so running it over a per-member budget would write the whole workspace's spend into that member's row. See Budgets & spend limits.
  • Gateway-wide alerting — AlertingService (opt-in, Gateway:Alerts:Enabled) watches the error rate (default threshold 10%) and optionally p95 latency over a look-back window (15 minutes) every 300 seconds, across all workspaces, and fires AlertNotifier (logs + optional Slack-compatible webhook + optional SMTP, shared with the provider health probe) on breach and on recovery. Per-workspace rules — spend, volume, error rate, latency, TTFT, score averages, new error types — live on the panel's Alerts page; see Observability.