API overview
The gateway API is OpenAI-compatible. Point your existing OpenAI client at the Relay base URL and use a Relay API key.
- Base URL (local):
http://localhost:5300 - Auth:
Authorization: Bearer <relay-api-key>on every request. The exceptions carry their own credential:GET /health, the realtime WebSocket (GET /v1/realtime, which takes the single-use token minted byPOST /v1/realtime/sessions), and workflow webhook / resume URLs (the token in the URL is the credential). - Content type:
application/json, exceptPOST /v1/filesandPOST /v1/knowledge/{id}/files, which aremultipart/form-data.
#Endpoint map
| Method | Path | Purpose | Scope |
|---|---|---|---|
POST | /v1/chat/completions | Chat completions (streaming or not). | chat.completions |
POST | /v1/embeddings | Text embeddings. | embeddings |
GET | /v1/models | List the models this key can call. | models.read |
POST | /v1/responses | Minimal OpenAI Responses shim. | — |
GET | /v1/tools | Function-tool registry. | — |
GET · POST | /v1/prompts · /v1/prompts/{name} · /v1/prompts/{name}/compile | Prompt library: list, fetch, compile. | — |
GET · POST | /v1/agents · /v1/agents/{id}/chat/completions | Agents: discover and run. | — (data.query / connectors.use for dataset / connector agents) |
POST · GET · DELETE | /v1/files · /v1/files/{id} · /v1/files/{id}/content | File attachments. | — |
POST | /v1/knowledge/{id}/documents · /files · /url · /search | Knowledge (RAG) ingest & search. | — |
POST · GET | /v1/batches · /v1/batches/{id} · /v1/batches/{id}/results | Batch jobs. | — |
GET · POST | /v1/workflows · /v1/workflows/{idOrName}/runs · /v1/workflows/runs/{id} (+ /cancel, /approve, /reject, /events) · /v1/workflows/{idOrName}/chat/completions | Workflows. | workflows.run |
| any | /v1/workflows/hooks/{token} · /v1/workflows/resume/{token} | Workflow webhook triggers and resume URLs (no key). | — |
POST | /v1/realtime/sessions · /v1/agents/{id}/realtime/sessions | Mint a realtime voice session (for a model, or for an agent). | — |
GET | /v1/realtime | WebSocket: bidirectional audio relay. | token |
POST · GET · DELETE | /v1/scores · /v1/scores/batch · /v1/scores/{id} | Scores (feedback, app-side checks). | scores.write / telemetry.read |
GET · POST | /v1/telemetry/summary · /traces · /traces/{id} · /observations · /sessions · /users · /export | Read telemetry. | telemetry.read (+ telemetry.bodies) |
POST | /v1/metrics | Aggregate metrics. | telemetry.read |
POST | /v1/traces/ingest · /v1/otlp/v1/traces | Send your own spans (Relay JSON, or OTLP/HTTP JSON). | traces.write |
GET | /v1/pricing · POST /v1/cost/simulate | Pricing & cost estimation. | — |
GET | /health · /v1/providers/health · /v1/models/metrics | Health & rolling metrics. | — / models.read |
A dash in the scope column means any valid key may call the endpoint; a key with an empty scope list is unrestricted everywhere. See Authentication & keys.
#Errors
Errors follow the OpenAI envelope. code is the stable field to branch on; for most gateway errors type repeats it, and invalid_request_error appears as the type for request-shape problems:
{ "error": { "message": "…", "type": "model_not_allowed", "code": "model_not_allowed" } }
| Status | Codes | When |
|---|---|---|
400 | invalid_request_error, image_too_large, invalid_image_url, image_url_not_supported, too_many_files, content_flagged, unsupported_capability, stream_not_supported, user_header_required | Malformed or unsupported request; moderation block; an agent that needs the acting end user and did not get it. |
401 | invalid_api_key | Missing, revoked or unknown key. |
402 | budget_exceeded, end_user_budget_exceeded | A workspace hard-stop budget, or a per-end-user spend limit, is exhausted. |
403 | model_not_allowed, insufficient_scope, agent_not_shared | The key's model allow-list or scope list refuses it; or the named end user is not on a shared agent's list. |
404 | model_not_found, agent_not_found, prompt_version_not_found, file_not_found, knowledge_base_not_found | Unknown or out-of-workspace resource. model_not_found is a 400 when no model was sent and Gateway:DefaultModel is unset. |
429 | rate_limit_exceeded | Per-key rate limit; Retry-After says when to retry. Upstream 429s are retried by the gateway first. |
503 | vector_store_unavailable, dataset_backend_unconfigured | Qdrant or the data app is not configured. |
5xx | provider error | The upstream failed after retries and any fallback. |
The two user_header_required / agent_not_shared cases are about the acting end user rather than the key: a dataset agent, a connector agent or an agent shared with named users needs X-Relay-User-Id.
#Request headers Relay reads
| Header | Purpose |
|---|---|
X-Relay-User-Id | The end user of your app. Drives per-user telemetry and spend limits, and is the acting user for agents and file ownership (an Entra object id in those cases). X-User-Id is accepted as a legacy alias. |
X-Relay-Session-Id, X-Relay-Tags, X-Relay-Trace-Name, X-Relay-Environment, X-Relay-Release, X-Relay-Metadata, X-Relay-Prompt | Trace attribution — see Observability. |
X-Relay-Trace-Id, X-Relay-Parent-Span-Id, traceparent | Join an existing trace / nest under a span. |
X-Relay-Capture | metadata or none: lower body capture for this request (never raise it). |
X-Relay-Cache | no (or false) bypasses the response cache. |
Idempotency-Key | Replay the stored response for a repeated non-streaming call. |
#Useful response headers
| Header | Meaning |
|---|---|
X-Relay-Request-Id | The id for this request (also in telemetry). |
X-Relay-Trace-Id, X-Relay-Observation-Id | The trace and the observation this call produced, for scoring or joining later. The trace id is omitted when capture resolves to none. |
X-Relay-Cost-Usd | Computed cost. Non-streaming chat only (including agent runs and /v1/responses); 0 on cache hits and idempotent replays. Streamed and embeddings calls record cost in telemetry only. |
X-Relay-Cache | HIT / MISS when the response cache is enabled. |
Idempotency-Replayed | true when an Idempotency-Key matched a stored response. |
X-RateLimit-Limit, X-RateLimit-Remaining, Retry-After | Present on every response from a rate-limited key; Retry-After on a 429. |
X-Relay-Moderation-Flagged, X-Relay-Moderation-Output-Flagged | Moderation flagged the input / the output (flag-only mode). |
X-Relay-Thread-Id | Set by the agent endpoint when using ?thread=. |
X-Relay-Dataset-Id, X-Relay-Sql-Queries, X-Relay-Rls-Applied | Set by a dataset agent run. |
X-Relay-Workflow-Run-Id | Set when a workflow run is started. |
#Streaming
Set "stream": true on chat requests to receive Server-Sent Events (data: {chunk} lines, terminated by data: [DONE]), exactly like OpenAI. If the provider fails after output has begun, the stream ends with a data: {"error": {…}} line and no [DONE]. See Chat completions.