Relay AI Gateway
Relay is GMRL's internal AI gateway: a single, OpenAI-compatible endpoint that sits in front of every large-language-model provider your teams use. Apps talk to Relay; Relay handles routing, keys, cost, caching, guardrails, retrieval, agents and observability.
Think of it as the control plane for AI in the organisation — one place to manage who can call which models, how much they spend, and what the models are allowed to do.
#Why a gateway
Without a gateway, every app wires directly to OpenAI/Anthropic/Gemini with its own key, its own retry logic, and no shared visibility. Relay centralises all of that:
- One API, many providers. Apps use the OpenAI wire format — text, images and tool calling included; Relay translates to OpenAI, Anthropic, Gemini, Azure Foundry or Ollama behind the scenes.
- Governance. Scoped API keys, per-workspace and per-end-user budgets with hard stops, per-key model allow-lists and rate limits, and a full audit trail.
- Resilience. Transient retry, provider health circuit-breaking, and fallback along each model's configured fallback chain.
- Cost control. Live cost attribution per workspace/app/model/user/agent/prompt, a cost simulator, and budget enforcement.
- Higher-level building blocks. A versioned prompt library; agents with skills, tools, MCP servers, knowledge bases (RAG), dataset (SQL) grounding and Microsoft 365 connectors; visual workflows; file attachments; realtime voice; batch jobs and evaluations.
- Observability. Every request is a trace of typed observations in ClickHouse — with sessions, end users, scores, annotation queues, LLM-as-a-judge evaluators, datasets and experiments — plus OpenTelemetry export to ClickStack/HyperDX.
#The three surfaces
Relay is delivered as three things:
| Surface | What it is | Who uses it |
|---|---|---|
| Gateway API | The public /v1/* HTTP API (OpenAI-compatible). | Your applications, authenticated with an API key. |
| Control panel | A Blazor web app for administration. | Platform admins and team owners, signed in through the company identity app. |
| This documentation | The site you're reading. | Everyone. |
#Where to go next
- New here? Read Architecture then Run it locally.
- Building an app against Relay? Start with API overview and Authentication & keys. Building Nook? See Using Relay from Nook.
- Administering the platform? See The panel, page by page.
Relay speaks the OpenAI API. In most SDKs you only need to change the base URL to your Relay host and use a Relay API key — no other code changes.