omnigate

Attested upstreams

Some providers run their models inside confidential hardware: virtual machines whose memory the host cannot read, on processors that sign a statement of what is running in them. That statement is an attestation. omnigate can require one of an upstream, verify it itself, keep it verified, and refuse to send a request anywhere else.

It is built for NEAR AI Cloud, which runs its gateway and the models it serves itself in Intel TDX machines, with NVIDIA GPUs that attest too. "provider": "near" is the one kind there is.

All of this is opt-in. It is switched on by giving an upstream an "attestation" setting. A config without one fetches and checks nothing, restricts no request, and has no Private chat page (/private-chat and its endpoints answer 404, and the dashboard shows no link to it).

Read What a pass means and What is not checked before you rely on it.

Contents:

Switching it on

An attestation object on a text upstream. config/omnigate.near.json is a complete config that uses it:

{
  "name": "near-ai",
  "kind": "openai",
  "base_url": "https://cloud-api.near.ai",
  "api_key_env": "NEAR_AI_API_KEY",
  "attestation": {
    "provider": "near",
    "accepted_tcb": ["UpToDate"],
    "gpu": "if_present",
    "max_age_s": 900
  }
}
Field Type Default Meaning
provider string required near. Nothing else is known; anything else is refused when the file is loaded.
accepted_tcb list of strings ["UpToDate"] The platform patch statuses that count as a pass, as Intel names them: UpToDate, SWHardeningNeeded, ConfigurationNeeded, ConfigurationAndSWHardeningNeeded, OutOfDate, OutOfDateConfigurationNeeded. A revoked platform never passes. One or more.
gpu string if_present What to do about NVIDIA GPU evidence in a model's report. if_present: verify it when it is there. require: also fail when it is missing. skip: never look at it (and never call NVIDIA).
max_age_s whole number, 60 to 86,400 900 How old a verification may be. The control plane verifies again after half of it. See How it is kept verified.
measurements object every list empty Allowlists of what was measured, each a list of at most 64 hexadecimal values: gateway_app_compose_sha256, model_app_compose_sha256 (64 characters each), gateway_mr_td, model_mr_td (96 characters each). An empty list accepts any value. See What a pass means.

A setting that is not understood (another field, a misspelt one, a wrong type or range) stops the config from loading: at startup the control plane does not start, on a reload the previous config stays in force. Apart from gpu and accepted_tcb above, nothing switches a check off, and nothing names another trust anchor: not a setting here, not an environment variable.

Every model a route sends to that upstream is then attested by its canonical name. NEAR serves some models itself and relays others to third parties; only the first kind has an attestation. Its public model list (GET https://cloud-api.near.ai/v1/models, no key needed) marks them "owned_by": "nearai". A target on an attested upstream whose model has no attestation is simply never used; put such a model on an upstream without the setting if you want it reachable.

What is verified, in order

For each (upstream, model) the control plane makes up a nonce (32 random bytes) and asks the provider for a report bound to it: GET <base_url>/v1/attestation/report?model=<model>&provider=near&signing_algo=ed25519&nonce=<nonce> with x-no-aliasing: true. No API key is sent; the report is public. The answer has a report for the provider's gateway and one or more for the model. The gateway's report and every model report must pass all of the following, or the whole check fails. The code is control/lib/omnigate_control/attestation/ (near.ex, tdx.ex).

  1. The report claims our nonce. A report that echoes another nonce is somebody else's, or an old one; it is refused before anything is fetched for it.
  2. The quote is a genuine Intel TDX quote (tdx.ex, following Intel's DCAP verification as implemented in Phala's dcap-qvl):
    • the certificate chain inside the quote ends at the Intel SGX Root CA, whose certificate is pinned in the source;
    • no certificate in it is revoked, by Intel's revocation lists for the root and for the platform CA (both signed, both within their validity);
    • Intel's signed list of patch levels for this platform family ("TCB info") and its description of the quoting enclave are authentic (signed by a certificate under the same root) and current;
    • the quoting enclave is Intel's, in an acceptable version, and it signed the key the quote is signed with;
    • the quote's signature over its header and body holds;
    • the platform's patch level, looked up in that list, gives its status.
  3. The status is one the setting accepts (accepted_tcb).
  4. The machine is not in debug mode, and its other attributes are the ones Intel's verifier accepts (not migratable, no reserved bits, no service-TD extension, SEPT_VE_DISABLE set).
  5. The quote binds our nonce and a signing key. Its 64 bytes of report data are the signing identity the report names (hashed together with the TLS key fingerprint when the report carries one), then the nonce. So the quote was produced after the nonce was, and the machine vouches for that key.
  6. The configuration shown is the one measured. The report carries the machine's software configuration (app_compose: which container images, with which settings). Its SHA-256 must be the value the processor recorded at start (MRCONFIGID).
  7. The event log replays to the measured value. The boot events in the report, hashed in order, must give the register value in the quote (RTMR3). Each event's digest is recomputed from its content.
  8. GPU evidence, per gpu: the evidence in a model's report must carry the same nonce, and NVIDIA's remote attestation service (nras.attestation.nvidia.com) must return an overall verdict of true.
  9. The model report names the model that was asked for, and there is at least one.
  10. The measurement allowlists, where set.

What a pass means

The provider's gateway and the machine that serves the model are genuine Intel TDX virtual machines, on platforms in an accepted patch state, not in debug mode, running exactly the configuration the report shows, and they answered a number this gateway had just made up. With GPU evidence: the model's GPUs are genuine and in a state NVIDIA accepts.

It does not by itself mean that the configuration is software you trust. The checks establish "this is the hardware, and this is what runs on it"; whether what runs on it is, say, NEAR's published inference stack rather than something that copies prompts elsewhere is a policy question, and the policy is measurements. With the lists empty, as they are unless you fill them, any configuration on genuine hardware passes. To pin one: take app_compose_sha256 and mr_td of the gateway and the model from GET /admin/attestations after reading the configuration they stand for (the provider's verification pages say how), and put them in the lists. They change when the provider deploys a new version, and the target then fails until you add the new values: that is what pinning is.

What is not checked

And one thing that no attestation gives: the model reads the prompt. An enclave keeps the operator of the machine out; it does not keep the software inside it from seeing what it is asked. Redaction is what limits what the model itself is shown.

How it is kept verified

The control plane's attestation manager (control/lib/omnigate_control/attestation/manager.ex) looks at the config once a second.

Nothing is kept on disk. After a restart of the control plane every pair starts as pending and is verified again, which takes a few seconds (about five for the first, with Intel's collateral to fetch; under two after).

What the data plane does with the verdict

The config snapshot the data plane polls carries, for each target on an attested upstream, "attestation": {"verified": true|false, "max_age_s": N, "tcb": "UpToDate", "since": <unix seconds>}. The snapshot only changes when a verdict flips, not at every re-check.

A target is used only while both hold:

The second is there because a data plane keeps serving its last config when the control plane is down, and must not keep calling a target "verified" when nobody is verifying any more. A data plane that starts from its cached config while the control plane is unreachable does not use attested targets at all until the control plane answers.

Otherwise the target is passed over like one that is down: the route's next target is tried (the attempt is recorded as attestation_unverified). When that leaves no target that was tried, the answer is

HTTP/1.1 503 Service Unavailable
{"error":{"code":"attestation_unverified","type":"server_error","message":"the providers of this model must prove what they run on, and none has a verified attestation right now", ...}}

and nothing was sent anywhere. Such refusals are counted per key like the other refusals (omnigate_rejected_requests_total{reason="attestation_unverified"}).

A response that came from a target on an attested upstream carries

x-omnigate-attestation: verified; tcb=UpToDate; since=1794416000

since is the Unix time from which the pair has been verified without a gap. The header describes the provider's deployment as last checked (see What is not checked); it is not a signature over the response. That is the receipt, fetched with the response's x-omnigate-request-id.

How stale can "verified" be? At most max_age_s after the last pass as the control plane counts, plus, if the control plane stops answering right at the end of that, another max_age_s at the data plane: twice max_age_s in the worst case, half of it in the ordinary one. The two processes compare durations, not wall clocks, so clock differences between their machines do not matter.

Requests that may only go to attested providers

The above protects every request to an attested upstream. The fifth part of privacy mode, attested, is about the request instead: it may go only to a target whose attestation holds, whatever else the route has.

On a key:

{ "name": "legal-team", "key_sha256": "...", "privacy": { "redact": true, "no_store": true, "unlink": true, "attested": true } }

or per request, on any key:

x-omnigate-privacy: attested
x-omnigate-privacy: redact, no-store, attested

Seeing what a provider would be sent

x-omnigate-dry-run: privacy

on a text request makes the gateway answer with the request as privacy mode left it, and send nothing:

{
  "object": "omnigate.privacy_dry_run",
  "sent_upstream": false,
  "protections": "redact, unlink",
  "attested_only": false,
  "replaced": {"email": 1, "phone": 1, "card": 1, "ip": 0, "secret": 0},
  "body": "{\"model\":\"private\",\"messages\":[{\"role\":\"user\",\"content\":\"Write to [EMAIL_1] ...\"}]}"
}

body is the request body after unlinking and redaction, as a string. It went through the same code as a request that is forwarded, up to the point where a target is chosen; what a provider is sent differs from it only in what is done per target (the model name is replaced by the target's, a request for a provider of the other protocol is translated, OpenAI is added "store": false). The request is authenticated, rate-limited and counted against the key's concurrency like any other; it is not billed, not written to the ledger, and the values it replaced are counted in omnigate_redactions_total. The one value is privacy; anything else, or the header twice, is 400 invalid_request, so that a request meant to stay in the gateway is never sent on because of a typo. It is refused on POST /v1/jobs.

A receipt for each answer

The checks above are about the provider's deployment. A receipt is about one answer: the provider's signature over the exact bytes of the request and of the response, checked against the attestation that had been verified before the request.

For every answer (status 2xx, read to its end) from a target on an attested upstream:

  1. The data plane hashes what crossed the wire: SHA-256 of the request body exactly as it sent it to the provider (after redaction, with the provider's model name), and SHA-256 of the response body exactly as it arrived (before anything is translated or restored; for a stream, every event and blank line). It sends the request with x-no-aliasing: true, so that the provider refuses rather than serve another model under the name, and asks for an uncompressed answer. The two hashes and the provider's id for the completion go to the control plane with the usage event. No content does.
  2. The control plane asks the provider for its signature: GET /v1/signature/<completion id>?signing_algo=ed25519, with the upstream's API key (api_key_env must therefore be set for the control plane too). A signature is recorded a moment after the answer, so "not found" is asked again after 1, 2, 4 and 8 seconds, then given up on.
  3. It checks the signature:
    • the signed text is exactly <model>:<request hash>:<response hash> when the model's machine signed (provider_tee), or <request hash>:<response hash> when the provider's gateway did (gateway);
    • the Ed25519 signature is valid;
    • its key is the key of exactly one report that this gateway had verified before the request started: a model report for provider_tee, the gateway report for gateway. A report fetched afterwards is never used for this, and a key named by no report, or by two, does not pass. (The last four passes per model are kept, so a re-check that finished while a request was under way does not take its evidence away.)
  4. It keeps a receipt: verified, or not_verified with the reason. Anything missing, late, unknown or ambiguous is not_verified: no signature after the retries, a provider that says it has none, a signature that does not say who signed, a request that started before any attestation had been verified, an answer without a completion id.
curl http://127.0.0.1:8080/v1/attestations/<x-omnigate-request-id> -H "Authorization: Bearer $KEY"

answers the key that made the request (any other key, and an id without a receipt, get the same 404 receipt_not_found):

{
  "object": "attestation.receipt",
  "id": "b65bc543b0d001df8e2cc7f99faf3111",
  "created": 1794423638,
  "upstream": "near-ai",
  "model": "z-ai/glm-5.3-flash",
  "route_model": "private",
  "stream": false,
  "completion_id": "07280ebbfe224a289176da572f5d23f0",
  "request_sha256": "3ee5aaea...",
  "response_sha256": "d613c6d9...",
  "verdict": "verified",
  "reason": null,
  "reason_code": null,
  "signature": {
    "kind": "provider_tee",
    "text": "z-ai/glm-5.3-flash:3ee5aaea...:d613c6d9...",
    "signature": "<128 hexadecimal characters>",
    "signing_address": "5a0210f2442c7935...",
    "signing_algo": "ed25519"
  },
  "attestation": {
    "verified_at": 1794423626,
    "nonce": "f8fbf294...",
    "reports": [
      { "role": "gateway", "tcb_status": "UpToDate", "signing_address": "40599fb2...", "signing_algo": "ed25519",
        "app_compose_sha256": "73dbb78c...", "mr_td": "7fe60787...", "rtmr3": "f30f282d...", "signed_this_answer": false },
      { "role": "model", "tcb_status": "UpToDate", "signing_address": "5a0210f2...", "signing_algo": "ed25519",
        "app_compose_sha256": "a0e3c6f4...", "mr_td": "df4d77c8...", "rtmr3": "1e5b2b85...", "signed_this_answer": true }
    ]
  },
  "checked_at": 1794423639
}

A receipt is pending for the second or so the check takes (it is made when the usage event arrives, a moment after the answer; until then the id is a 404). It holds hashes, a signature and public measurements: no prompt, no answer, no key. GET /admin/receipts?limit=N lists the newest with the name of the key that made each request and counts by verdict; GET /admin/receipts/<id> is one; the metric is omnigate_receipts{verdict}. A not_verified receipt is logged as a warning.

Checking a receipt yourself.

python tools/verify_receipt.py receipt.json request.bin response.bin

recomputes the two hashes from the bodies, rebuilds the text that must have been signed, and verifies the Ed25519 signature under the key in the receipt, with the standard library only (its Ed25519 code is checked against RFC 8032's vectors: --self-test). It prints VERIFIED and exits 0, or NOT VERIFIED with the reason and exits 1; one changed byte in either body is enough. Without the two bodies it checks the signature against the hashes in the receipt and says so.

The two bodies are the bytes on the provider's side of the gateway. For a request sent without redaction and without no-store, a client can say what they are: the request it sent with the route's name replaced by the provider's model name, and the response body it received (for a stream, when the client itself asked for stream_options.include_usage). tools/near_live.py does exactly this. With redaction the gateway changes the request before it is sent and the answer after it arrives; the receipt is then about those changed bytes, which only the gateway saw, and a client can check the signature but not recompute the hashes.

What a receipt does not prove.

Receipts are kept in memory and in one file (OMNIGATE_RECEIPTS, default data/receipts.json): the newest 10,000, and none older than retention.receipts_days (30 unless set). A check that was under way when the control plane stopped is not_verified after the restart, since the evidence it would be matched to went with the old process.

Looking at it: the admin API, metrics, logs

GET /admin/attestations (admin token or session):

{
  "attestations": [
    {
      "upstream": "near-ai",
      "model": "z-ai/glm-5.3-flash",
      "provider": "near",
      "state": "verified",
      "verified": true,
      "reason": null,
      "reason_code": null,
      "checked_at": 1794416021,
      "age_s": 44,
      "verified_since": 1794415000,
      "until": 1794416921,
      "max_age_s": 900,
      "last_attempt_at": 1794416021,
      "next_check_at": 1794416471,
      "checks": 3,
      "policy": {"accepted_tcb": ["UpToDate"], "gpu": "if_present", "measurements": {"gateway_app_compose_sha256": [], "...": []}},
      "evidence": {
        "nonce": "7f2e5643...",
        "verified_at": 1794416021,
        "gateway": {"tcb_status": "UpToDate", "advisory_ids": [], "quote_version": 4, "td_attributes": "0000001000000000",
                    "report_data": "...", "signing_algo": "ed25519", "signing_address": "40599fb2...", "tls_cert_fingerprint": null,
                    "app_compose_sha256": "73dbb78c...", "mr_td": "...", "rtmr3": "...", "gpu": "skipped", "collateral_expires_at": 1797000000},
        "models": [{"tcb_status": "UpToDate", "gpu": "verified", "...": "..."}]
      }
    }
  ],
  "checks": {"ok": 3, "failed": 0, "unavailable": 0}
}

state is pending, verified or failed; reason says why in one line, reason_code is the name of the check that failed (nonce_mismatch, bad_quote, tcb_status_not_accepted, report_unavailable, measurement_not_allowed, ...). evidence is that of the last pass and stays there for the record after a later failure; verified is the verdict. The gateway's gpu is always skipped: GPU evidence belongs to models.

POST /admin/attestations/verify with {"upstream": "...", "model": "..."} runs a check now, with a new nonce, waits for it, and answers with the entry as above (404 when the config routes no such pair to an attested upstream). It is written to the audit log (attestation.verify).

Metrics (GET /metrics):

Metric Type Meaning
omnigate_attestation_verified{upstream,model} gauge 1 while the pair is verified and recent enough, else 0. Alert on 0.
omnigate_attestation_age_seconds{upstream,model} gauge Seconds since the last pass; absent until there has been one.
omnigate_attestation_checks_total{outcome} counter Checks since start: ok, failed (the evidence did not hold), unavailable (it could not be fetched).

Log lines of the control plane, one per change of state, never one per check:

[info] attestation: near-ai/z-ai/glm-5.3-flash verified (TCB UpToDate, signer 40599fb284f43953…)
[warning] attestation: near-ai/openai/gpt-5 NOT verified: no attestation report (HTTP 503: Provider error: No provider found that supports attestation reports for model: openai/gpt-5). Its targets are not used until it verifies.
[warning] attestation: near-ai/z-ai/glm-5.3-flash could not be checked again (...); the last pass stands until it is 900 s old
[warning] attestation: near-ai/z-ai/glm-5.3-flash was verified 912 s ago and has not been verified again: withdrawn (...)

The Private chat page

Only present while the config has an upstream with an "attestation" setting; otherwise it answers 404.

/private-chat on the control plane (same sign-in as the dashboard, linked from its top bar) is a chat with three panels:

Two switches: redaction, and "attested providers only". The model list has every text route, marked attested or not, so the refusal can be seen by picking one that is not.

How it works, so that nothing about it has to be taken on trust: the page adds no path around the gateway. The control plane sends each message to the data plane's public port with an ordinary gateway key, twice: once with x-omnigate-dry-run: privacy, once for real with the x-omnigate-privacy header the switches ask for (always including no-store). The key is a managed key named private-chat, created on first use; its secret is held in the control plane's memory only, and the key is rotated after a restart to get a new one. It is an ordinary key: it is in the key list, its usage is billed and counted, and it can be given a budget or be disabled there (a disabled key makes the page's messages fail with 401).

Nothing of a conversation is kept by the gateway: the page holds the history in the browser tab and sends all of it with each message, the control plane neither logs nor audits the body, and every request goes out as no-store. The end-to-end suite sends a marked prompt through the page's endpoint and then searches every file both processes wrote for it.

The control plane reaches the data plane at OMNIGATE_GATEWAY_URL, or, if that is not set, at the config's media.public_base_url (http://127.0.0.1:8080 by default). The Compose file sets it to http://dataplane:8080, the data plane's address inside the Compose network.

Without a provider key in the data plane's environment the verification panel works (attestation needs no key) and a chat is answered by the provider with its own 401, which the page says.

Trying it against the real service

python tools/near_live.py

needs the network and no key. It runs the control plane's attestation code once from the command line and prints the checklist; starts both processes with config/omnigate.near.json on ports of their own; waits for the verdicts (the model NEAR serves itself verifies, the one it relays does not); and checks that a request for attested providers only is let through to the first and refused, unsent, on the others. With NEAR_AI_API_KEY set it also sends one real completion with an e-mail address and a card number in it, and checks that the answer has them back and the response says x-omnigate-attestation: verified; then one more completion whole and one streamed, fetches the receipt of each, and checks it with tools/verify_receipt.py from the bytes it sent and received (and that one changed byte fails). Three short completions in all.

python tools/near_live.py --serve

starts the same on the usual ports and prints the address of the Private chat page and the admin token.

The check alone, without starting anything:

cd control && mix run --no-start -e "OmnigateControl.Attestation.Live.main()"

(main(["--all"]) for every model NEAR serves itself, ["--json"] for the evidence as JSON; in a release, bin/omnigate_control eval 'OmnigateControl.Attestation.Live.main()'.)

It is not part of tools/check.py or CI: it depends on three services on the internet.

What the control plane calls

With an attested upstream configured the control plane makes outbound HTTPS calls it otherwise would not:

Host For
the upstream's own host (cloud-api.near.ai) The attestation report (no API key is sent), and for each answer from it GET /v1/signature/<id> with the upstream's API key: the control plane needs api_key_env in its environment for receipts.
api.trustedservices.intel.com, certificates.trustedservices.intel.com Intel's collateral: patch-level lists, the quoting enclave's identity, revocation lists. Once a day per platform family.
nras.attestation.nvidia.com NVIDIA's verdict on a model's GPU evidence, unless gpu is skip. The evidence (public, from the report) is posted there at every check.

Allow these if the control plane's egress is filtered. Certificates are checked against the system's trust store; redirects are not followed.

Mixed versions

The data plane enforces the verdict; the control plane produces it. Both must be this version or later.

Control plane Data plane What happens
new old The snapshot carries the verdict; an old data plane ignores it and uses attested targets whether or not they are verified, and ignores a key's attested. An x-omnigate-privacy: attested header is refused by it with 400 (it does not know the word), which is at least not a silent pass.
old new The old control plane does not know the attestation setting. A config with it still loads there (older code ignores fields it does not know) and nothing is verified; the new data plane sees no verdict on any target, so nothing is treated as attested and attested requests are refused everywhere.

Upgrade both, data plane included, before telling anyone a route is attested, and check GET /admin/attestations and a response's x-omnigate-attestation header afterwards.

Receipts came after the verdict: a data plane from before them reports no hashes (no receipt is made, and GET /v1/attestations/<id> is a 404 from it), and a control plane from before them ignores the hashes a newer data plane reports. Neither breaks anything else.

What has been tested

This page on GitHub