omnigate

Pre-release 0.1.0-rc.1 · source-available

One gateway for every model you call. Yours to run.

omnigate puts one port in front of OpenAI- and Anthropic-style text APIs and seven kinds of media provider, with gateway keys, budgets, failover and a usage ledger. It is two small processes and files on disk: no database, no queue, no cloud account.

Run it locally Source on GitHub

your appsdata planecontrol planetext providersmedia providersyour bucket or disk Zig · the public portElixir · ledger, budgets, jobsOpenAI-style, Anthropic-style /v1configusage eventskeys, limits, failoverjobs

What it does

Text gateway

Chat, messages, embeddings and responses endpoints, streaming and not. An OpenAI client can use an Anthropic-style upstream and the reverse, with tools, images and structured output translated.

API reference

Failover and limits

The next target is tried on timeouts, 429 and 5xx before any byte reaches the client. Each key has a model list, a rate limit, a concurrency limit and a budget.

Configuration

Media jobs

Images, speech, sound, music and video as jobs with signed, retried webhooks. Outputs are copied to your own bucket or disk, and a job can fail over between providers.

Providers

A ledger that survives outages

Usage is read from the provider's own counts, spooled to disk and deduplicated by request id. A call made while the control plane was down is billed once.

Architecture

Privacy mode

Redact personal data and secrets before a provider sees them, keep no request content, encrypt job outputs, and unlink usage from the key. Per key or per request.

Privacy mode

Small enough to audit

The data plane uses the Zig standard library only; the control plane has two direct Hex packages. The security model, known limits and past advisories are written down.

Security policy

Optional · NEAR AI Cloud

Proof that a prompt only went to verified hardware

For models that run in a trusted execution environment, the gateway checks the provider's evidence itself instead of taking its word:

  • The Intel TDX quote is verified in the control plane against Intel's pinned root, with revocation lists and patch status.
  • The signing key, the software configuration and the GPU evidence are bound to that quote.
  • A key marked attested gets 503 instead of an answer when verification fails. It cannot reach an unverified model.
  • Each answer comes with a receipt carrying the provider's signature over the request and the response, which anyone can check offline.

Without an attestation block in the config none of this is active, and omnigate is a plain model gateway.

How attested inference works, and what it does not prove

The Private chat page: the conversation, what left the machine after redaction, and the list of verification checks with a signed receipt.
The Private chat page, with a real answer from NEAR AI Cloud. The personal details are sample data.

Optional · NEAR Intents

Keys that pay for themselves

A prepaid key holds credit: one credit is one dollar when it is added, and a request burns what the provider charged times a multiplier you set. When the credit runs out, the holder of the key, or an agent using it, can buy more without anyone in between:

  • It asks the gateway for a payment and gets a deposit address for the asset it wants to pay with, from about 230 assets on 39 chains.
  • The funds are swapped to USDC and delivered to the operator's account. The gateway never holds a private key.
  • What arrived is credited to the key, once.

Payments · The credit rule

# a key that is out of credit can still do this
curl $GATEWAY/v1/payments \
  -H "Authorization: Bearer $KEY" \
  -d '{"amount_usd": 5,
       "asset": "nep141:wrap.near",
       "refund_to": "payer.near"}'

# → a deposit address, the amount to send, a deadline
# then: awaiting_deposit → processing → settled

Measured, with the caveats attached

0.5 msadded latency at the median
1.9 MBdata plane binary
13 MBdata plane memory under the benchmark
669,000requests in a five-minute load test, none lost or billed twice

One machine, localhost, against a mock provider written in Python. This shows the gateway is stable and not a bottleneck in front of a mock; it is not a comparison with any other gateway. How these were taken

Where it stands

omnigate is a pre-release. The text gateway and seven of the eight media adapters have been checked by hand against the real provider APIs, and every push runs the unit, end-to-end, SDK, container and hostile-input suites. It has never served production traffic.

Other gateways are ahead in places, and the README says where:

  • They speak to far more providers natively. Azure, Bedrock and Vertex are not reachable from omnigate.
  • They have production mileage.
  • Cost-aware routing, guardrails, SSO and tracing integrations are not here.

What has been verified · How it compares

Run it

With Zig 0.17, Erlang/OTP 29 with Elixir 1.20 and Python 3.9 or newer:

git clone https://github.com/Legatia/omnigate
cd omnigate/dataplane && zig build
cd .. && python tools/e2e.py

The last command starts mock providers, both processes and a few hundred checks; no API key is needed. On a server, a Compose file with a TLS proxy is included.

Quickstart · Deployment

License

omnigate is source-available under the PolyForm Noncommercial License 1.0.0: free for personal projects, learning, research and noncommercial organisations. Use in or for a business needs a commercial license; write to hello@legatia.solutions.