API reference

Two HTTP surfaces:

• The gateway (data plane, port 8080 by default) is what applications call: text endpoints, the jobs API and file downloads.
• The control plane (port 4000 by default) is for operators: dashboard, admin API, metrics. Applications never talk to it.

Contents:

• Authentication
• Errors
• Text endpoints
• Using the official SDKs
• Jobs API
• File URLs
• Webhooks
• Control plane: admin API, metrics, dashboard

The examples use the keys and routes of config/omnigate.dev.json (see the Quickstart).

Authentication

Send a gateway key in either header:

Authorization: Bearer og-unlimited-key

x-api-key: og-unlimited-key

Every gateway endpoint needs a key except GET /healthz and a file URL that carries a valid signature.

A key that has been disabled, deleted or rotated away is answered like a key that never existed: 401 invalid_api_key.

Errors

Shape

Errors the gateway raises itself are written in the shape of the protocol the client is speaking, so that a program built on the OpenAI or the Anthropic SDK handles them like the provider's own. A provider's own errors are passed on, and put into that shape when they are not in it: see Provider errors.

On OpenAI-style endpoints, and on everything that is not listed under Anthropic below (the jobs API included):

{"error": {"message": "unknown gateway key", "type": "invalid_request_error", "param": null, "code": "invalid_api_key"}}

On Anthropic-style endpoints (/v1/messages, /v1/messages/count_tokens, and /v1/models when the request carries an anthropic-version header):

{"type": "error", "error": {"type": "authentication_error", "message": "unknown gateway key", "code": "invalid_api_key"}, "request_id": "0123456789abcdef0123456789abcdef"}

• error.code is the gateway's own reason, the same string in both shapes. It is the stable thing to match on, and the reason label the request is counted under in omnigate_rejected_requests_total.
• error.type is the provider's broader category for that reason, from the table below. It differs between the two protocols.
• param is always null. request_id (Anthropic shape) is the request's x-omnigate-request-id.

Earlier commits put the reason in error.type and used one shape for both protocols. A client that matched on error.type must now match on error.code.

One shape, whichever process refuses. The jobs API and file downloads are answered by the control plane, and so is everything on its own port (the admin API, /metrics, /internal/*). Its errors have the same OpenAI-style envelope as the data plane's:

{"error": {"message": "no such job", "type": "invalid_request_error", "param": null, "code": "job_not_found"}}

The reason is in error.code and the category in error.type, taken from the same table as the OpenAI column below (reasons that are not in it get their category from the status). A client of /v1/jobs therefore cannot tell, and need not care, which of the two processes said no. The control plane always answers in the OpenAI shape; the jobs API has no Anthropic form. Until commit 10bc009 the control plane used a shape of its own, {"error": {"type": "<reason>", "message": "..."}}: a client that read a job error's reason from error.type must read error.code now.

Reasons

The mapping is typesFor in dataplane/src/errors.zig. A reason that is not in that function's table gets its category from the status alone: 401, 403, 404 and 429 as in the rows above, 400 and 405 to 428 invalid_request_error, anything else server_error / api_error. That is how concurrency_limit_exceeded, request_memory_exceeded and request_too_complex get theirs (the unit test there that lists the reasons covers these three since the close-out commit).

Provider errors

A provider's answer with a status of 400 or above that is not retried (429 and 5xx fail over, see below) reaches the client with the provider's status. What its body looks like:

• Already the protocol's error envelope: passed on byte for byte. For an OpenAI-style upstream that is an object with an error object that has a message; for an Anthropic-style one, "type": "error" with an error object that has a type and a message.

• Anything else is put into that envelope (since the close-out commit, the one after 55a294d; before, it was passed on as it came, and an SDK showed the raw text or could not read it). The message is the provider's own words, code is the provider's when it gave one, and the content-type becomes application/json. type follows the status: in the OpenAI shape 401 authentication_error, 403 permission_error, 404 not_found_error, 429 rate_limit_error, 5xx server_error, anything else invalid_request_error (a type the provider put in its error object is kept instead); in the Anthropic shape the same names, with 413 request_too_large and 5xx api_error. Real examples, both from "OpenAI-compatible" endpoints answering a bad provider key:

  The provider sent	The client gets
  [{"error": {"code": 400, "message": "API key not valid. ...", "status": "INVALID_ARGUMENT"}}] (Google: an array)	{"error": {"message": "API key not valid. ...", "type": "invalid_request_error", "param": null, "code": "INVALID_ARGUMENT"}}
  {"code": "Client specified an invalid argument", "error": "Incorrect API key provided: ..."} (xAI: error is a string)	{"error": {"message": "Incorrect API key provided: ...", "type": "invalid_request_error", "param": null, "code": "Client specified an invalid argument"}}

  The message is looked for in error.message, error (a string), message and detail, in an object or in the first element of an array; a body with none of them, or one that is not JSON (a proxy's HTML page), becomes the message as text, and an empty or non-text body becomes upstream returned HTTP <status>. code is a string code, else a string status, else a numeric code in digits, else null (left out in the Anthropic shape).

• When the request was translated, the error is converted to the client's protocol, as before, and reads the same shapes.

• Not covered: an error body over 64 KiB, or one sent as an event stream, is relayed as it comes.

tools/sdk_compat.py checks this with both SDKs against the mock providers, which send the two bodies above; the bodies themselves were seen from the real services, the wrapping has only met the mock.

The jobs API has more reasons. The ones above are raised by the data plane. The control plane, which owns the jobs, adds job_not_found, idempotency_conflict and the others listed below, in the same shape.

There is no 408. A client that takes longer than OMNIGATE_CLIENT_TIMEOUT_MS to send its request, or leaves a keep-alive connection idle that long, has its connection closed without a response.

Request heads, paths and queries

The gateway is strict about the bytes of a request before it looks at what the request asks for. Anything two HTTP implementations could read differently is refused, because that disagreement is what request smuggling is made of, and nothing a client sends is copied into a request of the gateway's own unless it is plain. Ordinary requests, and the ones the official SDKs send in tools/sdk_compat.py, are not expected to run into these rules; a hand-built request, or a client with unusual header values, can.

The head. Checked first, before the key. A refusal at this stage is always in the OpenAI shape, has no request-id headers, is counted as a refused request under the key - and the reason invalid_request, leaves no access-log line, and closes the connection.

The path must be in its one canonical form, on every endpoint except GET /healthz: it starts with /, has no trailing slash, no empty segment (//), no . or .. segment, and every segment consists only of letters, digits and - . _ ~ : @. A percent sign is not in that set, so a percent-encoded path is refused, as are a backslash and a ;. /v1/jobs/ is not /v1/jobs: it answers 400, where earlier commits forwarded it as a job creation that the rate limit did not count.

The query string may be at most 2,048 characters of letters, digits and - . _ ~ = & + , :. A %, a /, a space or a quote answers 400. The signed file URLs the gateway hands out, and limit=N on the job list, fit.

Forwarded headers. Three request headers are copied into requests the gateway makes, and each must be plain or the request answers 400:

On every other endpoint an Idempotency-Key is neither forwarded nor looked at, so the keys the official SDKs attach to their calls pass whatever they contain. (Until commit e4e30e4 the header was checked on every endpoint, against a narrower set: letters, digits, -, ., _ and :.)

The body of a text request is walked once, in place, before anything else is done with it, and must stay inside these bounds:

The last rule exists because the gateway acts on those three fields and then, for a provider of the same protocol, forwards the client's own bytes with only the model name replaced: the provider parses the body again, and the two readings must not be able to differ. A job request body is checked for nesting depth only; the control plane parses it.

JSON inside a tool call's arguments string that is nested deeper than 64 levels is not refused: in a translated request it is handled like arguments that are not a JSON object, and travels on as a string instead of being parsed.

Limits on what one request and one key may hold

Before a request body is read, the data plane checks what the key already has in flight. Each refusal is an ordinary answer in the client's protocol with retry-after: 1, so an SDK backs off and tries again by itself:

They apply to every text endpoint and to every POST on the jobs API, and come before the rate limit: a request refused here has not used up a rate-limit token. A stream whose client has gone but whose upstream answer is still being read for billing (Streaming) keeps its place until that is finished, so hanging up and coming back is not a way around the cap.

A same-protocol request is forwarded without ever being parsed into a tree. A tree is built only where one is needed: translation to the other protocol, the response cache's key, the fixes for provider quirks, and rewriting a stream_options the client sent without include_usage. Those steps have limits of their own:

An ordinary request is nowhere near these: measured on real request shapes a translation needs 10 to 70 times its text. The 1,024-member bound is the one a legitimate request could meet (a tool schema with more than 1,024 properties in one object, say), and only on a route that translates.

When the gateway answers before it has read a request's body (a bad key on a POST, for example), the response says connection: close, and the gateway reads and discards what the client is still sending for a short time before it closes, so the client sees the answer rather than a reset connection. The unread body is never parsed as a second request.

tools/malformed_test.py sends these cases, and a few hundred generated variations, at a running gateway; see Operations.

When every target fails

{
  "error": {
    "message": "every target for this model failed",
    "type": "server_error",
    "param": null,
    "code": "all_upstreams_failed",
    "attempts": [
      {"upstream": "mock-flaky", "outcome": "http_503"},
      {"upstream": "mock-ok", "outcome": "ConnectionRefused"}
    ]
  }
}

outcome is ok, http_<status>, unknown_upstream, protocol_mismatch, untranslatable_request, stream_aborted, or the name of the network error (UpstreamTimeout, ConnectionRefused and so on).

The answer depends on whether any target answered 429:

• At least one did: the client gets 429 upstream_rate_limited, so that an SDK backs off as it would at the provider. The response carries the longest retry-after that any of the rate-limited targets sent (a value in plain seconds wins over an HTTP date, which cannot be compared).
• None did (only 5xx answers, timeouts, connection errors): 502 all_upstreams_failed. If the last target that answered sent a retry-after header, the 502 carries it.

A route whose first target is rate limited and whose second target is down therefore answers 429: a later target that was merely down does not make the rate limit go away. (Until commit 10bc009 only the last target tried decided, and that route answered 502.) On a metered endpoint both answers are written to the ledger as a request with that status and no tokens.

Provider errors are not rewritten. When a provider answers with a status that is not 429 or 5xx (a 400 for a bad parameter, a 401 for a bad provider key), the gateway passes that status and body to the client. If the request was translated, the error body is converted to the client's protocol. A 429 or 5xx from a provider triggers failover instead; the client sees the outcome only when every target has failed, as described above.

The gateway checks a text request in this order: the request head, the path, the query string and the forwarded headers (above), config present, key present and known, budget, endpoint, the key's concurrency cap and the request-memory budget, the rate limit, the declared body size, and only then is the body read: nesting depth, JSON and its bounds, model, model limits, route, and the response cache if the route has one.

The rate limit is charged before the body is read. A request that is then refused for its size, its JSON, its model or the key's model list has used up one request of the key's rpm. (Until commit e4e30e4 the body was read first, so that a disallowed model cost nothing; the price was that a key could make the gateway read and parse megabytes before any limit was consulted.)

What an over-budget key can still do

A key that has spent its budget may not start anything new, but keeps access to what it already has:

Text endpoints

"Metered" means the call counts against the key's rate limit and is written to the usage ledger. count_tokens does neither.

An endpoint that is not translated can only be served by an upstream of its own protocol. On a route that has none (embeddings on a route whose targets are all Anthropic-style, count_tokens on a route whose targets are all OpenAI-style) the answer is 400 endpoint_not_supported; earlier commits answered 502 there.

Request

The body is the provider's own format with one difference: "model" is the name of a route in your config. The gateway replaces it with the target's model id and forwards the rest unchanged (or translated, see below). The body must be JSON and at most 8 MiB.

curl http://127.0.0.1:8080/v1/chat/completions -H "Authorization: Bearer og-unlimited-key" -d '{"model":"demo-chat","messages":[{"role":"user","content":"hi"}]}'

curl http://127.0.0.1:8080/v1/messages -H "x-api-key: og-unlimited-key" -d '{"model":"demo-claude","max_tokens":64,"messages":[{"role":"user","content":"hi"}]}'

Request headers the gateway acts on:

Other request headers are not forwarded to the provider. The values of the first three are restricted to plain characters; see Request heads, paths and queries. A request body needs a Content-Length: chunked request bodies answer 411.

Response

The provider's response, with these headers added:

Request ids are on every answer the gateway produces, with three exceptions: provider responses, the gateway's own errors, the model endpoints, cache hits, and the jobs and files API all have them. On those that involve no provider, all three headers carry the gateway's id. The exceptions are GET /healthz, the 503 overloaded that is sent without reading the request, and a request whose head was refused (the 400, 411, 417 and 431 of Request heads, paths and queries).

A 429 upstream_rate_limited or 502 all_upstreams_failed carries the three id headers, and retry-after when an upstream sent one (When every target fails says whose).

Framing. The gateway reads a provider's response body with its own bounded reader and frames its answer to the client itself; a length the provider declared is never passed on.

• An answer that is not an event stream and is at most 64 KiB is read to its end, then sent in one piece with a content-length the gateway counted. If the provider cuts it short, or sends its head and then nothing for OMNIGATE_STREAM_IDLE_TIMEOUT_MS, nothing has been sent yet, and the request fails over like any other failed attempt (UpstreamTruncated, StreamIdleTimeout).
• An event stream, and any longer answer, is sent with transfer-encoding: chunked as it arrives. If the provider breaks off part way, or sends a body shorter than the length it declared, the gateway drops the client's connection without the closing chunk. An HTTP client reports that as an incomplete or broken response, which is what it was. The call is recorded with status 499.
• A provider response whose head is ambiguous about its length (both a length and chunked encoding, two different lengths, a transfer coding other than chunked) or whose chunk sizes do not parse is treated as a failed attempt (UpstreamUnparseable), and the next target is tried.

Provider headers. The provider's content-type is kept (a value with a control byte in it, or longer than 256 bytes, fails the attempt), and the gateway sets cache-control: no-cache. Beyond that, a fixed allowlist of provider response headers is passed to the client under their original names:

• Names are matched without regard to case. Everything else is dropped: connection and framing headers, cookies, the provider's organization and project headers, CDN headers.
• A translated response keeps the upstream's names: a client that asked in the OpenAI format and was served by an Anthropic upstream gets anthropic-ratelimit-* headers, not x-ratelimit-*.
• At most 32 headers are passed per response, and a header with an empty value or a value longer than 512 bytes is dropped.
• These headers describe the provider's limits for your provider key, shared by every gateway key that uses the upstream. They are not the gateway key's own rpm limit.

The allowlist is covered by unit tests, and the SDK suite checks the request-id headers and retry-after end to end against the mock providers. The rate-limit headers are not exercised end to end, and none of this has been checked against a real provider.

Streaming

Set "stream": true as the provider's protocol defines. Chunks are flushed to the client as they arrive.

For /v1/chat/completions and /v1/completions, OpenAI-style providers report token usage on a stream only when asked. If the client did not set stream_options.include_usage, the gateway sets it to meter the call and removes the extra usage chunk from what the client receives.

Failover happens only before the first byte of a response reaches the client. If a stream breaks after that, the client's connection is dropped (without the closing chunk, see Framing) and the call is recorded with status 499 in the ledger.

Two timeouts watch a response in progress:

• The upstream goes silent. When it sends nothing for OMNIGATE_STREAM_IDLE_TIMEOUT_MS (five minutes by default), the gateway closes both sides and records the call with status 499.
• The client stops reading. When a client stays connected but takes none of a relayed response for OMNIGATE_CLIENT_STALL_TIMEOUT_MS (30 seconds by default), the gateway gives up on it. From then on it is handled like a client that hung up.

A response that ends early is still billed. Hanging up on a stream does not make it free: the provider charges for what it generated.

• When the client disconnects or stalls, the client's connection is finished with, and the gateway keeps reading the upstream in the background: until the provider's closing usage figures, or the end of the answer, or for at most OMNIGATE_ABORT_DRAIN_MS (two minutes by default) and OMNIGATE_ABORT_DRAIN_MAX_MB (64) more megabytes. If the figures arrive, the call is billed exactly those. Only then is the upstream connection closed, which is what makes a provider stop generating.
• Whenever a response on a metered endpoint ends without those closing figures (they did not arrive within those bounds, the upstream went silent or broke off, more than 256 such answers were already being read on, or the gateway was stopped), the call is billed an estimate, and its usage event is marked "estimated": true. The estimate is never below what was actually metered.
• The same goes for an event stream that reaches its proper end without the provider ever reporting usage: some OpenAI-compatible servers ignore stream_options.include_usage. Such a call is billed the estimate, not nothing.
  • Input that was not reported is taken as the request body's size in bytes divided by 4, plus 1,600 tokens for every image, document, file or audio part the request names.
  • Output that was not reported is, for an event stream, the larger of one token per event received and the generated text received (the string values of content, text, delta, arguments, partial_json, thinking, reasoning_content, reasoning and refusal) divided by 4; for anything else, the response's bytes divided by 4.
• The estimate errs on the high side for the request and is rough for the response. It exists so that an abandoned stream costs something close to what the provider charged, not so that the two agree.

How to see which calls were estimated is in Operations.

Response cache

A route with cache_ttl_s answers repeated identical requests from the data plane's memory; see Configuration for what "identical" means and what bounds the cache. From the client's side:

• Only non-streaming requests take part. The first one is forwarded and comes back with x-omnigate-cache: miss next to the usual headers.
• A repeat within the TTL, from the same gateway key, comes back with the stored status, content-type and body, and these headers only: x-omnigate-cache: hit and a new request id (in x-omnigate-request-id, x-request-id and request-id). There is no x-omnigate-upstream or x-omnigate-attempts, because no upstream was called, and no provider headers.
• A hit is byte for byte the stored answer, including the provider's response id and token usage figures from the original call. The gateway bills it as zero tokens and zero cost.
• Only complete 2xx answers are stored. Errors are never cached.
• Translated answers are cached too, in the client's protocol.
• A request under privacy mode's no_store or redact (from the key or from x-omnigate-privacy) takes no part: it is neither answered from the cache nor stored in it, and carries no x-omnigate-cache header.

curl -i http://127.0.0.1:8080/v1/chat/completions -H "Authorization: Bearer <gateway key>" -d '{"model":"<a route with cache_ttl_s>","messages":[{"role":"user","content":"hi"}]}'

(The dev config has no cached text route; add "cache_ttl_s": 60 to one of its routes to try this.)

Translation

/v1/chat/completions and /v1/messages can be served by an upstream of the other protocol, including streaming, tools and images. The response carries the route name in model, in the client's protocol.

• Fields with no equivalent on the other side are dropped, not rejected: thinking blocks, cache control, penalties, log probabilities and similar.
• Structured output is translated.
  • OpenAI client, Anthropic-style upstream: response_format of type json_schema becomes output_config.format with the same schema (the OpenAI-only name and strict are dropped). {"type": "json_object"} has no Anthropic parameter, so it becomes one sentence appended to the system prompt asking for a single JSON object and nothing else.
  • Anthropic client, OpenAI-style upstream: output_config.format of type json_schema becomes response_format with the schema under the name response and "strict": true. Other members of output_config are dropped.
  • Both directions are covered by unit tests and by the SDK suite against the mock providers. The Anthropic side has not been verified against the live Anthropic API: neither that output_config.format is accepted as the gateway sends it, nor how the OpenAI side treats a schema written for Anthropic under strict.
• Audio and file parts cannot be translated. A target that would need them is skipped with the outcome untranslatable_request; if no target is left that could be sent the request, the answer is 400 untranslatable_request.
• When a translated request is refused because the model rejects temperature, top_p or top_k, the gateway retries without them and remembers that for the model, for the gateway key that ran into it (not for other keys, so one client cannot change how another's requests are sent). The same applies to models that demand a thought signature on earlier tool calls (Gemini 3 and later). Requests in the provider's own protocol are not touched and get the provider's error (Provider errors).

The README lists what has not been verified against real providers here: images in translated requests, long conversations, large payloads and structured output. The checks against real providers used plain HTTP; tools/sdk_compat.py drives the official SDKs through the gateway in both translation directions, against the mock providers only. See Using the official SDKs.

GET /v1/models and GET /v1/models/{id}

GET /v1/models lists the routes the key may use: the text routes first, then the media routes (the models of the jobs API). A key with a models list sees only the names on it. The list does not say which kind a model is, or which protocol its upstreams speak; a media model is used with POST /v1/jobs, not with the text endpoints. A key over its budget can still call both endpoints.

The answer has the shape of the client's protocol. By default that is OpenAI's:

{
  "object": "list",
  "data": [
    {"id": "demo-chat", "object": "model", "created": 1794312000, "owned_by": "omnigate"},
    {"id": "demo-claude", "object": "model", "created": 1794312000, "owned_by": "omnigate"},
    {"id": "demo-image", "object": "model", "created": 1794312000, "owned_by": "omnigate"}
  ]
}

A request that carries an anthropic-version header (the Anthropic SDK always sends one) gets Anthropic's:

{
  "data": [
    {"type": "model", "id": "demo-chat", "display_name": "demo-chat", "created_at": "2026-11-10T12:00:00Z"},
    {"type": "model", "id": "demo-claude", "display_name": "demo-claude", "created_at": "2026-11-10T12:00:00Z"},
    {"type": "model", "id": "demo-image", "display_name": "demo-image", "created_at": "2026-11-10T12:00:00Z"}
  ],
  "has_more": false,
  "first_id": "demo-chat",
  "last_id": "demo-image"
}

(Shortened: the dev config has more routes. The times are made up.)

• A gateway model is a route, not a provider model, so it has no real creation date: created / created_at is the time this data-plane process started, the same for every model. display_name repeats the id.
• The list is never paged. has_more is always false, and query parameters such as limit or after_id are ignored.
• GET /v1/models/{id} returns one object of the list. A model that does not exist and a model the key may not use get the same answer, 404 model_not_found, so the reply does not reveal which other models exist. Errors from both endpoints are in the same protocol's shape as the list would have been.
• The id is part of the path, so it falls under the path rule: a route whose name has a character outside letters, digits and - . _ ~ : @ (a space or a +, say) is listed and can be used in a request body, but GET /v1/models/{id} answers 400 invalid_request for it. A / in a name is fine.

GET /healthz

Answers 200 with the body ok and needs no key. It says the process is up; it does not say that a config has been loaded.

Using the official SDKs

The OpenAI and Anthropic SDKs work against the gateway by changing two things: the base URL and the API key (a gateway key, not a provider key). "model" is a route name from your config.

tools/sdk_compat.py runs the official Python SDKs through the gateway against the mock providers: both SDKs against both kinds of upstream, sync and async, streaming and not, tools, embeddings, the Responses API, the exception classes each SDK raises, and the details an SDK user would notice if the gateway differed from the real APIs (request ids, error codes and envelopes, the model endpoints, a provider's 429, structured output through translation). It is the basis for this section. Every case must pass; there is no list of tolerated gaps any more. It has not been run against real providers, and only the Python SDKs are covered.

The suite was written against openai 3.49.0 and anthropic 2.8.0, which is what CI installs (.github/workflows/ci.yml pins them). The snippets below use the quickstart's key and routes. Install the SDKs yourself (pip install openai anthropic); the repository does not depend on them.

OpenAI SDK. The base URL ends in /v1:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="og-unlimited-key")

reply = client.chat.completions.create(
    model="demo-chat",
    messages=[{"role": "user", "content": "hi"}],
)
print(reply.choices[0].message.content)

Anthropic SDK. The base URL is the gateway's address without /v1 (the SDK adds /v1/messages itself):

from anthropic import Anthropic

client = Anthropic(base_url="http://127.0.0.1:8080", api_key="og-unlimited-key")

message = client.messages.create(
    model="demo-claude",
    max_tokens=64,
    messages=[{"role": "user", "content": "hi"}],
)
print(message.content[0].text)

Either SDK can use either kind of route: model="demo-claude" with the OpenAI SDK, or model="demo-direct" with the Anthropic SDK, goes through translation.

Things to know:

• Environment variables. Both SDKs read OPENAI_* / ANTHROPIC_* variables (API key, base URL) when an argument is not given. Pass base_url and api_key explicitly, or make sure your shell does not hold a provider key that would be sent to the gateway instead.
• Either header works. The Anthropic SDK sends the key as x-api-key; built with auth_token= instead of api_key= it sends Authorization: Bearer, which the gateway accepts as well.
• Retries. The SDKs retry 429 and 5xx answers by themselves. The gateway's own 429 rate_limit_exceeded carries Retry-After, and so does a provider's 429 that reaches the client as 429 upstream_rate_limited when the provider sent one. Set max_retries=0 on the client if you want to see such answers instead of waiting through them.
• Errors. Gateway errors arrive as the SDK's usual classes by status: 400 BadRequestError, 401 AuthenticationError, 403 PermissionDeniedError, 404 NotFoundError, 429 RateLimitError, 502 InternalServerError. With the OpenAI SDK, exc.code is the gateway's reason (model_not_found, rate_limit_exceeded, ...) and exc.type the category. With the Anthropic SDK, exc.type is Anthropic's category (not_found_error, rate_limit_error, ...) and the reason is exc.body["error"]["code"]. See Errors.
• Request ids. _request_id on a response and request_id on an exception are set: to the provider's id when the provider sent one, to the gateway's otherwise.
• Models. models.list() and models.retrieve(id) work in each SDK's own shape.
• Structured output. With the OpenAI SDK, response_format with a JSON schema, and chat.completions.parse() with a pydantic model, work on a route served by an Anthropic-style upstream; with the Anthropic SDK, output_config.format works on a route served by an OpenAI-style upstream. Against the mocks only; see Translation.
• Gateway headers. with_raw_response exposes x-omnigate-request-id, x-omnigate-upstream and x-omnigate-attempts.

Differences from the real APIs that an SDK user can still see:

Jobs API

One async API in front of image, video and speech providers. The data plane authenticates the key and hands these requests to the control plane, so they need the control plane to be reachable.

POST /v1/jobs                 create a job
GET  /v1/jobs/{id}            fetch one job
GET  /v1/jobs?limit=N         this key's jobs, newest first
POST /v1/jobs/{id}/cancel     cancel a job

Only POST /v1/jobs counts against the key's rate limit (listing has an allowance of its own, below). A key sees only its own jobs; another key's job id answers 404. A managed key keeps its jobs when its secret is rotated (Admin).

Creating jobs is limited in these ways:

• The data plane's limits for any request with a body. The key's concurrency cap and the request-memory budget (above) apply to every POST on the jobs API.
• The key's rpm, twice. The data plane charges its token bucket, as for a text request. The control plane then counts the creation again by itself, with the same token bucket (a burst of up to rpm, refilled at rpm per minute), so that the limit on work that costs money does not depend on the data plane recognising the request. Either can answer 429 rate_limit_exceeded. A request that repeats an Idempotency-Key the key already used creates nothing and is not counted by the control plane (the data plane still counts it).
• Unfinished jobs per key. A key may have at most max_open_jobs jobs that are queued or running (100 unless the key or media.max_open_jobs says otherwise; 0 means no limit). One more answers 429 too_many_open_jobs. A job that was canceled while its provider call was in flight counts until that call is over (Cancel a job). Jobs made under the key's earlier secrets count too, and a replay by Idempotency-Key is never refused for this.
• Jobs per day per key. A key may create at most max_jobs_per_day jobs per UTC day (10,000 unless the key or media.max_jobs_per_day says otherwise; 0 means no limit), whatever became of them. One more answers 429 too_many_jobs_today; the count starts again at midnight UTC, and Retry-After counts down to it. Jobs made under the key's earlier secrets count too.
• Size. input and metadata together may be at most media.max_input_bytes (1 MiB by default) as JSON, and metadata alone at most 16 KiB; more answers 413 payload_too_large. A webhook_url may be at most 2,048 bytes.
• What the route accepts. A route can list the input fields it takes (allowed_input), the values a field may have (price_multipliers) and a longest duration (max_seconds); see Create a job.

Listing is rate-limited too. GET /v1/jobs has a bucket of its own in the control plane, so listing never uses up the allowance for creating jobs: the key's rpm when it has one, 600 per minute otherwise. Over it the answer is 429 rate_limit_exceeded ("this key is listing jobs too often"). Fetching one job by id is not limited.

Every one of these 429 answers carries Retry-After: the data plane's own, and the control plane's, which the data plane passes on: the seconds until the next request fits for the rate limits; 10 for too_many_open_jobs; and for too_many_jobs_today the seconds until midnight UTC, when the count starts again, but at most 3,600 (until the close-out commit it was 10 there as well).

A key over its budget cannot create a job (402 budget_exceeded) but can still fetch, list and cancel its jobs and download their files.

Create a job

Request body:

Optional headers: Idempotency-Key (1 to 255 visible ASCII characters, without spaces), and x-omnigate-privacy: no-store (or on), which makes this one job private: its input and metadata are kept out of the job log and erased when the job ends, and its outputs are deleted a day later (Privacy mode). Every job of a key whose privacy mode has no_store is private without the header.

Some members of input are checked whatever the adapter, because they feed the price. Anything else than what this list allows answers 400 invalid_request:

• duration and seconds must be a number of seconds, 0 or more: a number, a string that is one, or a number followed by s (5, "5", "5s").
• n, num_outputs and num_images must be a whole number of 0 or more.
• On a route priced per second, a job must name a duration above 0 unless the operator configured a default for the route's targets (default_seconds), and may not ask for more than a target's max_seconds.
• On a route with allowed_input, input may only have the listed fields. The error names the fields that are not allowed and the ones that are.
• On a route with price_multipliers, a field that has factors may only carry one of the listed values. The error lists them.
• On a route whose price has variants, the request must be one of the priced combinations (a resolution, with or without audio, and so on). The error names the fields and lists the combinations that have a price.

The last four are the operator's settings (Configuration); a route that sets none of them accepts what it accepted before.

curl -i http://127.0.0.1:8080/v1/jobs -H "Authorization: Bearer og-unlimited-key" -H "Idempotency-Key: order-A-17" -d '{"model":"demo-image","input":{"prompt":"a lighthouse at dusk"},"metadata":{"order":"A-17"}}'

Response, 202 Accepted:

{
  "object": "job",
  "id": "job_5f0c2a9e4b7d81c3a6e2f490",
  "status": "queued",
  "model": "demo-image",
  "input": {"prompt": "a lighthouse at dusk"},
  "metadata": {"order": "A-17"},
  "outputs": [],
  "error": null,
  "cost_usd": 0.0,
  "cached": false,
  "upstream": null,
  "target_model": null,
  "attempts": [],
  "webhook": null,
  "created_at": "2026-11-10T12:00:00.000Z",
  "updated_at": "2026-11-10T12:00:00.000Z",
  "started_at": null,
  "finished_at": null
}

(The id and times in these examples are made up, as are the file size and the numbers in later examples; the shape is what the code returns.)

Fetch a job

curl http://127.0.0.1:8080/v1/jobs/job_5f0c2a9e4b7d81c3a6e2f490 -H "Authorization: Bearer og-unlimited-key"

A finished job on the dev config, where the first target fails and the second delivers:

{
  "object": "job",
  "id": "job_5f0c2a9e4b7d81c3a6e2f490",
  "status": "succeeded",
  "model": "demo-image",
  "input": {"prompt": "a lighthouse at dusk"},
  "metadata": {"order": "A-17"},
  "outputs": [
    {
      "url": "http://127.0.0.1:8080/v1/files/job_5f0c2a9e4b7d81c3a6e2f490/out-0.png?exp=1794315603&sig=<hex>",
      "content_type": "image/png",
      "bytes": 68,
      "sha256": "<hex sha-256 of the file>"
    }
  ],
  "error": null,
  "cost_usd": 0.003,
  "cached": false,
  "upstream": "mock-replicate",
  "target_model": "mock/image",
  "attempts": [
    {"upstream": "mock-replicate-broken", "outcome": "provider_failed: <the provider's reason>"},
    {"upstream": "mock-replicate", "outcome": "ok"}
  ],
  "webhook": null,
  "created_at": "2026-11-10T12:00:00.000Z",
  "updated_at": "2026-11-10T12:00:03.412Z",
  "started_at": "2026-11-10T12:00:00.004Z",
  "finished_at": "2026-11-10T12:00:03.410Z"
}

Fields:

error.type of a failed job:

List jobs

curl "http://127.0.0.1:8080/v1/jobs?limit=5" -H "Authorization: Bearer og-unlimited-key"

{"object": "list", "data": [ { "object": "job", "id": "job_..." } ]}

limit is between 1 and 100 and defaults to 20. Jobs are ordered newest first.

A list leaves large inputs out: a job whose input is over 8 KiB as JSON is listed with "input": null and "input_omitted": true. Fetch the job by its id to get the input. (One response can therefore not be made of a hundred megabyte-sized inputs.) Listing is rate-limited; see above.

Cancel a job

curl -X POST http://127.0.0.1:8080/v1/jobs/job_5f0c2a9e4b7d81c3a6e2f490/cancel -H "Authorization: Bearer og-unlimited-key"

Returns the job. An unfinished job becomes canceled at once; a job that had already finished is returned unchanged. The README notes that cancellation at a real provider has not been verified.

Whether a canceled job costs anything depends on what the provider did:

Until that is known the job, though canceled, has cost_usd 0, still counts towards the key's max_open_jobs, and its webhook is held back; the webhook is sent with the final cost. In the ledger a canceled job has status 499, and its attempts end with completed_after_cancel when it was billed. Canceling is therefore not a way to have a provider work for nothing.

Idempotency

With an Idempotency-Key header, a second POST /v1/jobs from the same gateway key with the same key value returns the existing job with status 200 instead of creating another. If the body differs (model, input, metadata, webhook_url or cache), the answer is 409 idempotency_conflict.

Idempotency keys are remembered for as long as the job is. If retention removes the job, the key can be used again.

Caching

A job may reuse the stored outputs of an earlier, identical, successful job instead of calling a provider. "Identical" means the same route name and the same input (key order does not matter).

• A request with "cache": true allows reuse on any route.
• A route with cache_ttl_s allows reuse by default, for outputs at most that many seconds old. A request with "cache": false opts out.
• With "cache": true on a route that has no cache_ttl_s, a stored output of any age can be reused.

A cache hit answers 202 like any other create, but the job is already succeeded, with "cached": true, "cost_usd": 0.0 and one attempt whose outcome is cache_hit.

From the README: the cache is shared across gateway keys, and two identical cacheable jobs submitted at the same moment both run.

Two kinds of job stay out of it, in both directions: a private job (privacy mode, no_store), and a job of a key with encrypt_to, whose stored output only that key's owner can read. "cache": true changes nothing for them.

Job errors

In addition to the general errors, which the data plane raises, the control plane raises the ones below. They have the same OpenAI-style shape, with the reason in error.code (see Shape), and the same x-request-id as any other answer. budget_exceeded, rate_limit_exceeded, invalid_request, invalid_json and payload_too_large can come from either side; for jobs, model_not_allowed and model_not_found come from the control plane.

Receipts

GET /v1/attestations/{request id}

Only on a gateway with an attested upstream. The receipt of one answer that came from such an upstream: the provider's signature over the exact request and response bytes, and whether it verified against the attestation checked before the request. {request id} is the x-omnigate-request-id of the response. The fields, an example and what a receipt does and does not prove are in A receipt for each answer.

Only the key that made the request (across its rotations) can read it. tools/verify_receipt.py checks one offline.

Payments

Only on a gateway whose config has a payments section; everywhere else these paths answer 404 not_found. The holder of a prepaid key buys credit with them, also when the key's credit is spent. The whole of it (the flow, the states, the errors, what is checked and what is trusted) is in Payments.

Their errors use the gateway's error shape; the codes are invalid_request, payment_quote_refused (400), payment_not_found (404), not_prepaid, payment_closed (409), too_many_open_payments, rate_limited (429), payment_provider_error (502) and payment_provider_unavailable (503). POST requests count against the key's rpm.

File URLs

GET /v1/files/{job id}/{name}?exp=<unix seconds>&sig=<hex>

Job outputs are copied into storage you control as soon as the job finishes, so the URL in a job keeps working after the provider's own link expires.

Local storage. outputs[].url points at the gateway (<media.public_base_url>/v1/files/...) and carries an expiry and an HMAC-SHA256 signature. Anyone holding the URL can download the file until it expires (media.storage.url_ttl_s, one hour by default); no gateway key is needed. Fetch the job again to get a fresh URL.

The same path without the query string works for the gateway key that owns the job:

curl -O http://127.0.0.1:8080/v1/files/job_5f0c2a9e4b7d81c3a6e2f490/out-0.png -H "Authorization: Bearer og-unlimited-key"

How a stored file is served. The bytes, and the type they were declared with, came from a provider, so they are never served as something a browser would render as a page on the gateway's origin.

• Content type. The stored type is kept only when it is one of image/png, image/jpeg, image/webp, image/gif, video/mp4, video/webm, video/quicktime, audio/mpeg, audio/wav, audio/x-wav, audio/ogg, audio/flac, audio/pcm or application/json. Anything else (text/html, image/svg+xml, XML, unknown types) is served as application/octet-stream. The job object still reports the stored type in outputs[].content_type.
• Download, not display. The control plane sends the file with content-disposition: attachment; filename="<name>", x-content-type-options: nosniff, content-security-policy: sandbox; default-src 'none' and cache-control: private, max-age=3600.
• What reaches the client. The data plane passes these four headers on, with the status, content type and body, so a client gets them from the gateway itself, with or without a proxy in front. (For a short while, between commits 3a0b147 and 10bc009, it did not, and the included proxy added them; that rule is gone from deploy/Caddyfile.)
• Framing. A file is sent with its content-length, whatever its size, so a client can show progress and can tell a whole file from a cut one; the size is also in the job (outputs[].bytes), with the SHA-256 to check the result against. (Between commit e4e30e4 and the close-out commit a file over 64 KiB was sent with transfer-encoding: chunked and no length.) If the download breaks off inside the gateway, the connection is dropped short of that length. A client that stops reading for OMNIGATE_CLIENT_STALL_TIMEOUT_MS (30 seconds) is disconnected.

To show an output in a web page, fetch it and use it from your own origin (or use bucket storage); do not point a page at the gateway URL and expect it to render inline.

Bucket storage. outputs[].url is a presigned GET URL on your bucket (or a plain public_base_url URL, if you configured one), not a gateway URL. A request to the gateway's /v1/files/... path for such an output answers 302 with that URL in Location. (Earlier commits dropped the Location header on the way through the data plane.)

missing_api_key is raised by the data plane when the request has neither a key nor a sig parameter. The others, and missing_api_key for a URL with sig but no exp, are raised by the control plane. Both use the same shape, with the reason in error.code.

Webhooks

Pass webhook_url when creating a job and the gateway POSTs the result there when the job reaches a final status (succeeded, failed or canceled). A signing secret must be configured for the key; see Configuration.

The request follows Standard Webhooks:

POST <webhook_url>
content-type: application/json
user-agent: omnigate-webhooks/0.1.0
webhook-id: msg_job_5f0c2a9e4b7d81c3a6e2f490
webhook-timestamp: 1794312003
webhook-signature: v1,<base64 HMAC-SHA256>

{
  "type": "job.succeeded",
  "timestamp": "2026-11-10T12:00:03.415Z",
  "data": { "object": "job", "id": "job_5f0c2a9e4b7d81c3a6e2f490", "status": "succeeded" }
}

type is job. followed by the status. data is the same job object that GET /v1/jobs/{id} returns.

Delivery and retries

• Any 2xx answer counts as delivered. Anything else, a redirect included, is a failed attempt.
• Only the status of the receiver's answer is used. At most 16 KiB of its body are read, then the connection is closed; a receiver cannot make the control plane hold a large answer.
• An attempt is recorded before its request is sent. A delivery that a crash or a restart cuts short has therefore used up its attempt, and a receiver gets at most max_attempts requests per job whatever happens.
• Deliveries are capped. At most media.webhooks.max_concurrent (16) run at once, and at most max_concurrent_per_key (4) for the jobs of one gateway key. The rest wait their turn, fairly across keys, so a delivery can start later than its schedule says when many are due at once.
• The host name is resolved once per attempt. Every address it resolves to is checked against the private-address rule, and the connection is then made to the address that was checked (with the host name still used for the Host header and the TLS certificate), so a DNS answer that changes between the check and the call cannot redirect a delivery inwards.
• Failed attempts are retried after the delays in media.webhooks.retry_schedule_ms, up to max_attempts deliveries in total. After that the webhook's status is failed.
• webhook-id is the same on every retry of a job; webhook-timestamp is the time of that attempt. Use webhook-id to ignore a delivery you have already processed.
• Delivery state is stored with the job and survives a control-plane restart. It is visible as webhook in the job object: status is waiting (job not finished), pending, retrying, delivered or failed.

Verifying the signature

The signature is the base64 HMAC-SHA256 of <webhook-id>.<webhook-timestamp>.<raw body>, prefixed with v1,. If the secret starts with whsec_, the key is the base64-decoded remainder; otherwise the secret's bytes are used as they are.

Verify against the raw request body, before parsing it:

import base64
import hashlib
import hmac
import time


def signing_key(secret: str) -> bytes:
    if secret.startswith("whsec_"):
        try:
            return base64.b64decode(secret[len("whsec_"):], validate=True)
        except ValueError:
            pass
    return secret.encode()


def verify(secret: str, headers, body: bytes, tolerance_s: int = 300) -> bool:
    """headers: a mapping with lower-case names. body: the raw bytes received."""
    msg_id = headers.get("webhook-id", "")
    timestamp = headers.get("webhook-timestamp", "")
    signatures = headers.get("webhook-signature", "")

    # Refuse old or future-dated deliveries, so a captured one cannot be replayed later.
    if not timestamp.isdigit() or abs(time.time() - int(timestamp)) > tolerance_s:
        return False

    signed = f"{msg_id}.{timestamp}.".encode() + body
    digest = hmac.new(signing_key(secret), signed, hashlib.sha256).digest()
    expected = "v1," + base64.b64encode(digest).decode()

    # The header may hold several space-separated signatures.
    return any(hmac.compare_digest(expected, s) for s in signatures.split(" "))

The five-minute tolerance is the receiver's choice (it is what the mock receiver in tools/mock_media.py uses); the gateway does not depend on it.

Control plane

Everything in this section is on the control plane's port. Apart from the two probes and the sign-in form it needs the admin token (or, for /internal/*, the internal token), never a gateway key.

GET /healthz and GET /readyz

GET /healthz answers 200 with the body ok. It says the process answers.

GET /readyz says it can do its job: 200 with {"status": "ready"} when the config is loaded, the job store is running, and the ledger is loaded and its file is writable. Otherwise it answers 503 with {"status": "not_ready", "reason": "..."}, naming the first thing that is not ready. The container image's health check calls it.

Neither needs authentication. While the control plane is starting, the port is not open yet (the ledger is loaded before the listener starts), so a probe gets a refused connection rather than a 503.

Once it is up, the listener stays up even if the ledger, the job store, the keys or the audit log have to restart: those run as a group of their own, which is restarted (with a pause that grows from half a second to thirty) without touching the listener. During such a restart /healthz still answers 200, /readyz answers 503 with the reason, and GET /internal/config answers 503 not_ready, so data planes keep the config they have and no key is unblocked by a ledger that has not loaded its totals yet. Other requests that need the missing piece fail instead of being answered from half-loaded state.

Errors and request paths

Control-plane errors have the same shape as the gateway's OpenAI-style errors: the reason in error.code, a broader category in error.type:

{"error": {"message": "send Authorization: Bearer <admin token>", "type": "invalid_request_error", "param": null, "code": "unauthorized"}}

(Until commit 10bc009 the control plane's errors were {"error": {"type": "<reason>", "message": "..."}}. A script that matched on error.type must match on error.code.) Every response from the control plane, errors included, carries an x-request-id header: the gateway's id for a request that the data plane relayed, a fresh one otherwise. Besides the errors of each endpoint, any request can get:

The admin API adds reasons of its own: key_not_found (404), key_exists (409), read_only_key (409) and, for a change from a browser session without the CSRF token, csrf_failed (403). They are described in Admin.

A request is handled in this order: the path check above, then the credential for the path's first segment, then the body, then the CSRF check, then the endpoint.

• Unknown paths are denied, not passed on. A first path segment the control plane does not serve is answered 404 at the guard. Under /admin and /internal an unknown path answers 401 without the right credential and 404 with it.
• A body is read only for a caller who has authenticated, and for the sign-in form. /admin/* and /internal/* bodies are parsed after the credential check. POST /login (and POST /logout from a signed-in browser) may carry at most 4,096 bytes, by declared length. Nobody else's body is read at all: a body on POST /healthz, on a GET, on DELETE /login or on an unknown path is left unread, the request gets the answer its path deserves, and the response says connection: close so that the unread bytes are never taken for a next request.
• Wrong credentials are counted, wherever the token is accepted. Five failed logins, or 20 wrong bearer tokens, from one address within a minute lock that address out of that way in until the minute is over. The bearer count covers /admin/*, /metrics, the dashboard page (/) and /login, which also accept the token; until commit d79406b the last two compared it outside the throttle. A locked-out address gets 429 too_many_attempts on the first three and on GET /login (which showed the form with 200 until the close-out commit). A request with no Authorization header at all is not a guess and is not counted, and a signed-in browser session is not affected by a bearer lockout.
• HTTP/1.1 only. The control plane's listener has HTTP/2 and WebSocket upgrades switched off; nothing it serves needs them.
• The path check is the fix for an authentication bypass in earlier commits, where /%61dmin/keys reached the admin API without a token; see SECURITY.md.

Admin API

Request bodies, answers, error types and curl examples for all of these are in Admin. In short:

curl http://127.0.0.1:4000/admin/keys -H "Authorization: Bearer dev-admin-token-change-me"

{
  "keys": [
    {
      "name": "dev",
      "source": "file",
      "created_at": null,
      "budget_usd": 0.01,
      "budget_period": "total",
      "period_spend_usd": 0.0108,
      "period_resets_at": null,
      "rpm": 0,
      "max_concurrent": 0,
      "models": null,
      "max_open_jobs": 100,
      "max_jobs_per_day": 10000,
      "disabled": false,
      "blocked": true,
      "state": "blocked",
      "shadowed": false,
      "has_webhook_secret": false,
      "requests": 4,
      "errors": 0,
      "input_tokens": 36,
      "cached_input_tokens": 0,
      "cache_write_input_tokens": 0,
      "output_tokens": 24,
      "cached_responses": 0,
      "jobs": 0,
      "cost_usd": 0.0108
    }
  ]
}

(Numbers are illustrative.)

A key that was rotated keeps one row: usage recorded under its earlier secrets is included.

Without a valid token or session the answer is 401, with a WWW-Authenticate: Bearer header:

{"error": {"message": "send Authorization: Bearer <admin token>", "type": "invalid_request_error", "param": null, "code": "unauthorized"}}

After 20 wrong tokens from one address within a minute, the answer to any bearer token from that address is 429 too_many_attempts with retry-after: 60, until the minute is over.

A change (POST, PATCH, DELETE) authenticated by a session cookie instead of the bearer token must also carry the session's CSRF token, or the answer is 403 csrf_failed. Requests with the bearer token need nothing extra. See Admin.

GET /metrics

Prometheus text format (version 0.0.4), behind the same admin token:

curl http://127.0.0.1:4000/metrics -H "Authorization: Bearer dev-admin-token-change-me"

The key label is the key's name, never its hash, and covers both file and managed keys. See Operations for a scrape config and caveats.

Dashboard and sessions

A valid session cookie is accepted wherever the admin token is, so /admin/* and /metrics also work in a signed-in browser (reads without anything extra, changes with the CSRF token).

The dashboard and login pages are sent with a Content-Security-Policy that allows nothing from other origins, X-Frame-Options: DENY, Referrer-Policy: no-referrer and Cache-Control: no-store.

/internal/*

Used by data planes only, with the internal token in the x-omnigate-internal header: GET /internal/config (which answers 503 not_ready while the ledger is starting), POST /internal/usage/batch (answered with how many events were accepted, were duplicates or were invalid), POST /internal/counters (counts of refused requests), and the jobs and files API under /internal/v1/. It is not a public interface. The Compose proxy answers 404 for these paths from outside, and the control plane itself refuses any percent-encoded spelling of them. The data plane builds the address of such a call from its own configuration and sets the client's path and query, which it has validated character by character, as separate components; a client cannot make it call anything but /internal/v1/jobs* and /internal/v1/files/*.

This page on GitHub