Architecture

Relay is a .NET 9 solution made of focused projects. The public API and the control panel are separate ASP.NET Core apps that share the same domain, data and provider libraries.

#Projects

Alongside the solution, Application.Database.Log is a plain folder of ClickHouse scripts (schema and dashboard queries — ClickHouse has no DACPAC tooling), sdks/ holds the thin client SDKs, and deploy/clickstack/ a ClickStack compose file.

#Data stores

• SQL Server — the relational store, split across two DbContexts:
  • GatewayDbContext (database relay_gateway) — workspaces and members, providers, models, API keys, budgets, prompts, agents and versions, skills, tools, MCP servers, connectors, workflows and runs, knowledge-base metadata, batches, evals, conversations, observability configuration (score configs, queues, evaluators, datasets, alert rules), audit.
  • UserManagementDbContext (database identity) — ASP.NET Identity users and roles, per-workspace SSO setup, and read-only application-access tables. Tenancy itself (workspaces) lives in relay_gateway.
  • Naming is snake_case; rows carry soft-delete + audit columns via a shared base model.
• ClickHouse — the observability store: one row per observation in relay_observation and scores in relay_score, always written when a connection string is configured. The usage dashboard can optionally read a ClickStack collector's otel_traces instead (Telemetry:DashboardSource=otel).
• Qdrant — the vector store for knowledge bases (RAG).

#Request lifecycle (chat)

A POST /v1/chat/completions flows through the gateway pipeline:

1. Authenticate the API key (ApiKey scheme) and load the key + its workspace. The panel's Playground arrives here too, with a short-lived delegation token that names one of your keys.
2. Apply key policy — scopes, the per-key rate limit and the model allow-list — then open a trace from the attribution headers.
3. Route the requested model through ModelRouter → a concrete provider adapter + invocation context, resolving the explicit model → Gateway:DefaultModel → alias remap → auto / tier strategies, decrypting the provider's key, building the fallback chain, and applying the workspace budget hard-stop.
4. Execute via GatewayChatService: image validation, moderation, the response cache and idempotency store, transient retry, provider-health circuit breaking, fallback to the next model in the chain, and SSE streaming when requested.
5. Record the generation as an observation (tokens, cost, latency, status, routed model, fallback used, captured bodies per policy) — written to ClickHouse and, optionally, exported over OTLP.

Agents, workflows, batches, evals, realtime sessions and knowledge search run through the same router and tracer, so their model calls appear in the same trace tree.

#Deliberate constraints

Two constraints shape how Relay is built and extended:

• No new NuGet packages. The build environment cannot restore packages beyond what is already cached. Every new capability is written with the BCL only — raw HttpClient, System.Text.Json, hand-written parsers (the OTLP exporter, the RAG reranker, the PDF/DOCX/XLSX/PPTX/HTML extractors, the workflow engine, even this docs site's Markdown renderer are all BCL-only).
• DACPAC-only schema. New tables/columns are authored as .sql in Application.Database and added to the DbContext model — not via EF migrations. Deploy schema by publishing the DACPAC.

See Configuration & secrets and Deployment for how these play out in practice.