Building on omnigate

omnigate is a model gateway. What it does inside a request (routing, failover, translation, redaction, metering) is the project's own code, and changes there are made in the repository. Everything else is meant to be built around it, over HTTP, by programs that omnigate knows nothing about: a payment service that credits keys, a billing tool that reads usage, an agent harness that calls models.

This page is for the people writing those programs. It names the interfaces you can rely on, shows the three most common things built around a gateway, and says plainly what is not offered.

• The interfaces
• Agents, harnesses and apps
• Taking payments: prepaid keys and top-ups
• Reading usage
• Reacting to media jobs
• Operating notes
• What is not offered
• What "stable" means before 1.0

Three runnable examples go with it, in examples/, Python standard library only. The end-to-end test (tools/e2e.py) runs each of them against the mock providers on every change, so they do not rot:

The interfaces

There is no fifth one. In particular there is no plugin API and no hook that runs your code inside a request; see What is not offered.

Agents, harnesses and apps

Anything that takes a base URL and an API key works, because the gateway speaks the providers' own protocols:

"model" is the name of a route in the gateway's config, not a provider's model id. Which provider answers, and in which protocol, is the gateway's business: an OpenAI-style client can be routed to an Anthropic model and the other way round (translation). The official SDKs are covered in Using the official SDKs.

One key per agent, tenant or end user. A gateway key is the unit everything is enforced and counted by. Create one with POST /admin/keys (Admin) and give it:

These hold whatever the agent does, because they are enforced by the gateway and not by the agent's own code.

What a client owes the gateway. Three answers deserve handling, and examples/agent_client.py shows all three:

Every response carries x-omnigate-request-id. It is the id of that request in the usage feed, so a client can log it and a billing tool can find the same request later.

Tagging requests. A client may send x-omnigate-tag: <label> on a text request or on POST /v1/jobs: its own name for whose request this is (a tenant, a run, an end user's id in your system). The gateway records it with the request's usage and does nothing else with it:

• 1 to 64 characters of letters, digits, ., _, : and -. Anything else, or the header sent twice, is 400 invalid_request: a tag that would be dropped or cut is refused, so that your books never silently miss a request.
• It is never sent to a provider, and it is in no metric (a label per end user would not survive Prometheus). It is in the ledger and therefore in the usage feed.
• Privacy mode does not remove it: unlink is about what a provider can tie to a person, and the tag never leaves your own machine. The same goes for no-store. If your tags must not identify people, send an opaque id, not an e-mail address (the character set already keeps @ out).
• One tag per request. For finer attribution use more keys.

curl http://127.0.0.1:8080/v1/chat/completions -H "Authorization: Bearer og-unlimited-key" -H "x-omnigate-tag: tenant-42" -d '{"model":"demo-chat","messages":[{"role":"user","content":"hi"}]}'

Taking payments: prepaid keys and top-ups

omnigate has a budget per key and one call that adds to it safely. A payment service of any kind (a chain watcher, a card processor's webhook receiver, a manual back-office tool) is a small program next to the control plane, and this section is how to write one.

(There is now also one payment provider built in, opt-in: with a payments section in the config, key holders buy credit themselves in crypto through NEAR Intents, with no program of yours in between. See Payments. It credits keys through the same top-up described here, and everything else on this page works next to it. What credit burns and earns, whoever paid for it, is in Prepaid credit.)

1. A customer gets a key with an empty prepaid budget. A total budget is credit bought in advance: it does not start over by itself.

   curl http://127.0.0.1:4000/admin/keys -H "Authorization: Bearer dev-admin-token-change-me" -H "Content-Type: application/json" -d '{"name":"customer-42","budget_usd":0}'
   

   The answer contains the key itself, once. With a budget of 0 the key is answered 402 until it is topped up.

2. Each payment is one top-up, named by the payment.

   curl http://127.0.0.1:4000/admin/keys/customer-42/topup -H "Authorization: Bearer dev-admin-token-change-me" -H "Content-Type: application/json" -d '{"amount_usd":5,"reference":"0x9f2c...e1"}'
   

   {
     "key": { "name": "customer-42", "budget_usd": 5.0, "period_spend_usd": 0.0, "state": "active", "...": "..." },
     "topup": { "reference": "0x9f2c...e1", "key": "customer-42", "amount_usd": 5.0, "budget_usd": 5.0, "at": "2026-11-11T18:00:00Z", "replayed": false }
   }
   

   reference is your id for the payment: a transaction hash, the payment provider's id, an invoice number. The budget and the reference are written to disk together. Sending the same top-up again adds nothing and answers with what the first call did, "replayed": true. So your service may deliver each payment at least once (retry on a timeout, rescan after a restart) and omnigate turns that into exactly once.

3. The key works again within seconds. The data plane learns of the new budget with its next config poll (OMNIGATE_POLL_MS, two seconds by default); no restart.

What can come back, besides 200:

Things to know:

• How much is left is budget_usd - period_spend_usd in GET /admin/keys. omnigate_key_budget_usd, omnigate_key_period_cost_usd and omnigate_key_blocked are the same in metrics, for an alert before a customer runs dry.
• Spending is checked after the fact. The request that crosses the budget completes, and so can a few that were in flight; then the key is blocked. A prepaid balance can therefore end slightly below zero. Size the smallest top-up accordingly, or keep a margin.
• The price is yours. cost_usd is computed from the prices in your config (Configuration), so what a customer's dollar buys is whatever you set there, markup included. omnigate does not know what the provider billed you.
• Refunds and corrections are not top-ups: a negative amount is refused. Set the budget outright with PATCH /admin/keys/{name} ({"budget_usd": ...}), which the audit log records as a change.
• References are remembered in the key store file, the newest 10,000 across all keys. A payment replayed after 10,000 later ones would be credited again; keep your own cursor (as the example does) rather than rescanning from the beginning of time.
• Every top-up is audited (key.topup, with the reference, the amount, the budget after it, and whether it was a replay): GET /admin/audit.
• Subscriptions instead of prepaid: give the key a monthly budget. It unblocks by itself when the month turns; your service only changes the budget when the plan changes (Budgets and periods).

examples/prepaid_topup.py is this service with a file standing in for the payment source. Replace its FileSource with whatever tells you that money arrived; it only has to hand out {reference, key, amount_usd} and may hand out the same payment more than once.

Reading usage

Three views, for three purposes:

The usage feed. GET /admin/events returns what the ledger recorded, oldest first: one event per metered text request and one per media job.

curl "http://127.0.0.1:4000/admin/events?limit=2" -H "Authorization: Bearer dev-admin-token-change-me"

{
  "events": [
    {
      "cursor": "1-412", "id": "0d5f...", "kind": "text", "ts": "2026-11-11T18:00:01.120Z", "received_at": "2026-11-11T18:00:01.640Z",
      "key": "customer-42", "tag": "tenant-42", "model": "demo-chat", "upstream": "mock-ok", "target_model": "ok-1",
      "status": 200, "stream": false, "input_tokens": 11, "cached_input_tokens": 0, "cache_write_input_tokens": 0, "output_tokens": 7,
      "outputs": null, "cached": false, "estimated": false, "latency_ms": 31, "cost_usd": 0.0000041, "credits_usd": 0.0000041
    }
  ],
  "next": "1-824",
  "more": true,
  "gap": false
}

How to consume it:

• Loop until more is false, passing next as after; then poll with the last next for new events. Save the cursor after you have stored the page. A page can be shorter than limit, or empty, while more is true (a filtered page skips other keys' events, and one call reads at most 4 MiB of the ledger): keep going.
• The order is the order of recording, which is close to but not the same as the order of ts. ts is when the request finished on the data plane; received_at is when the event reached the ledger, which can be later after an outage.
• id is unique. For a text request it is the x-omnigate-request-id the client received; for a media job it is the job id. If your consumer crashes between storing a page and saving the cursor, drop repeats by it.
• gap: true means the events between your cursor and the oldest one left were deleted by retention.ledger_days (Operations). Per-key totals stay exact; the single events are gone. With the default (0, keep forever) it never happens. A cursor from another installation, or one that was edited, is 400 invalid_cursor.
• key is a name, or null for a key that has been deleted since. Key hashes are never in the feed.
• No content. The ledger holds no prompt, answer, job input or output, so neither does the feed.
• cost_usd and credits_usd. The first is what the provider cost. The second is what the event burned of the key's credit: the same number, unless the key is prepaid and a burn multiplier applies.
• "kind": "credit" is credit added to a key, in the same order as its usage: amount_usd, source (admin, a payment provider's name, bonus), reference, bonus_eligible and budget_usd (the budget after it); cost_usd and credits_usd are 0. A consumer that only wants usage skips this kind. (New with the credits commit: a consumer written before it sees rows without a model or tokens.)
• estimated: true marks token counts the gateway had to estimate because a response ended before the provider's own figures arrived; cached: true marks an answer from the gateway's response cache, counted and not charged.

examples/usage_export.py is a complete consumer: it appends to a CSV file and keeps its cursor next to it.

Reacting to media jobs

A job can carry a webhook_url. When it finishes, the control plane sends a signed POST there, with retries (delivery). A receiver must verify the signature against the raw body before trusting it; Verifying the signature has the code. The key's webhook_secret (set on the key, in the config or through PATCH /admin/keys/{name}) is the signing secret.

A job's metadata (up to 16 KiB of your own JSON) comes back in the webhook and when the job is fetched; x-omnigate-tag on POST /v1/jobs puts a label on its event in the usage feed. Use the first for what your receiver needs, the second for what your billing needs.

There are no webhooks for text requests, for a key running out of budget, or for configuration changes. For a budget alert use the metrics above.

Operating notes

• There is one admin token, and it can do everything: create and delete keys, change budgets, read usage, read the audit log. There are no scoped or read-only tokens yet. So run a payment service or a billing exporter where the control plane runs (the same host or private network), give it the token from the same secret store, and treat it as part of your trusted back office.
• The control plane's port must not be public. Only the data plane's port is for clients. Security hardening has the details, including the admin allow-list in the bundled proxy.
• Wrong admin tokens are throttled per client address (Admin). A service with a stale token will see 429 after a few tries, not endless 401s.
• A script uses the bearer token, not a browser session: no CSRF token is needed (CSRF).
• Changes reach the data plane within its poll interval (two seconds by default). A key you just created, topped up or disabled is live then, not in the same instant.
• Back up the control plane's data directory. Managed keys (with the top-up references) and the ledger live there (Security hardening).

What is not offered

• No plugins, no in-process hooks. You cannot register code that runs when a request arrives, before it is routed, or when a response comes back. Routing rules, protocol translation, redaction classes, a new kind of provider adapter: those are changes to omnigate itself, made in the repository (CONTRIBUTING.md).
• No per-request callback to your service. The gateway does not ask anyone whether a request may proceed. What may proceed is decided by the key's settings, which you change through the admin API ahead of time.
• No scoped admin tokens (see above).
• No event push for usage. The usage feed is polled.
• No multi-node ledger. One control plane owns the keys and the ledger.

If something you need cannot be built from keys, budgets, tags, the usage feed, webhooks and metrics, that is worth an issue: it is how the list of interfaces grows.

What "stable" means before 1.0

omnigate has not reached 1.0, and nothing here is frozen. What the project commits to until then:

"Announced" means: in the changelog entry of the version that changes it and in the upgrade notes, before the release is tagged. It does not mean a deprecation period; before 1.0 there is none.

This page on GitHub