What Is Private AI Inference? A Guide to TEE-Based AI

Updated 21 October 2026. Private AI inference is running a model on your prompt inside hardware that is sealed from whoever operates the server: a trusted execution environment, or TEE, that neither the host, the network operator nor the provider can read into. On SayGM every request passes through an Intel TDX enclave, and open-weight models run inside it end to end.

The limit: frontier models such as Claude, GPT and Gemini are relayed through the sealed gateway, but their maker still receives the prompt.

Table of Contents

• How private AI inference works
• When the Prompt Is the Sensitive Asset
• Why Encrypt Prompts If You Have Nothing to Hide?
• Ads Are Arriving in AI Assistants
• What changed since September
• The Cost Trade-off of Private Inference
• What each route actually hides
• Which Providers Offer Private Inference
• Common Questions About Private Inference
• Is Private Inference Right for Your Workload?

Private inference is what happens when an AI model processes your prompt inside hardware that's sealed off from whoever operates it. No one running the server can see what you sent or what came back, and neither can the gateway, the host, or the company that owns the infrastructure. The technology behind that claim is called a trusted execution environment, or TEE, and it's becoming the standard way to prove a privacy claim about AI rather than just state one.

Most AI providers ask you to trust a privacy policy: a promise that they won't look at your prompts, won't log them, won't train on them. A TEE replaces that promise with hardware you or an independent auditor can verify. This post covers what that protection does, how it works under the hood, and how to tell one privacy claim from another. If you want the product-level version of this first, our introduction to a gateway built around one covers the same mechanism from the other direction.

How private AI inference works

A trusted execution environment is a sealed region of a processor where code runs isolated from everything else on the machine, including the operator of that machine. Two separate pieces of hardware need to be sealed for this to work for AI inference specifically. The CPU side, running on chips like Intel TDX or AMD SEV-SNP, seals the host so the hypervisor and operating system underneath can't read what's happening inside. The GPU side, on NVIDIA's Hopper or Blackwell generations, seals the accelerator itself, since a CPU-only boundary does little good if an attacker can just read the model weights or your prompt straight out of GPU memory.

Encryption at rest and in transit leave one window open: data has to be decrypted to be computed on, and whoever runs that machine can read it while that's happening. Closing that window is what the industry calls data in use encryption, and a secure enclave on the processor is the mechanism that does it.

Before any data moves, the system runs a remote-attestation handshake. A client asks the enclave to prove it's running genuine hardware with an approved, unmodified piece of code, and the decryption keys that let a prompt be processed are released only once that proof checks out. Nothing travels in the clear before that handshake completes.

Even with HTTPS, once your data reaches the server, you're trusting the entire stack behind it. Weights could be extracted, prompts could be logged, the inference itself could be manipulated. Private inference closes that gap by moving the trust boundary from a company's word to hardware anyone can independently check.

When the Prompt Is the Sensitive Asset

A hospital using AI to draft clinical notes or triage patient messages is sending protected health information into whatever processes that request. A business associate agreement is a legal promise not to misuse that data. A TEE is a hardware guarantee the infrastructure operator can't read it in the first place, which is the difference between "we won't look" and "we can't."

A law firm running contract review through an AI tool is sending privileged material somewhere, and privilege can be compromised the moment a third party has technical access to it, even one that swears it will never look. A fintech team doing fraud detection or underwriting is handling regulated financial records that carry both compliance exposure and breach risk if they sit in a vendor's logs indefinitely. A software team feeding proprietary code through an AI coding assistant has a version of the same problem: that code is valuable, and most developers have little visibility into what happens to it once it leaves their editor.

In September 2026, mathematicians working on the Navier-Stokes problem questioned whether their unpublished work had informed OpenAI's claim to have produced a proof of it. OpenAI said it had never seen that work and had accessed no specific user data, while adding that, though it called this unlikely, it could not rule out that de-identified data from people's use of its products had helped improve its models. Whether a provider opened your session and whether your work reached its models through training are two separate things, and it is easy to be told the first and assume you have been told the second. We look at the dispute, and at what it means for choosing between a frontier model and a confidential one, in our OpenAI Navier-Stokes dispute deep dive.

None of this depends on how large or well known the provider is. A policy-based guarantee is a bet on a company's reputation and its incentive to protect it, which means your protection is only ever as good as that company's word. Hardware attestation removes the need to trust anyone at all. You can check for yourself, before you send a prompt, that the machine running it cannot read what you send. That check is what SayGM's privacy claim rests on.

Why Encrypt Prompts If You Have Nothing to Hide?

The nothing-to-hide question has come up before, in the same form, every time a category of technology moved to encrypted by default, and it's never held up well. Encryption on the web stayed a niche add-on for close to two decades after SSL shipped, mostly reserved for bank logins and checkout pages. Nobody won that argument. The friction just went away: Let's Encrypt made certificates free, Chrome started marking plain HTTP sites "Not Secure" in 2018, and HTTPS went from a minority of traffic to the overwhelming majority. Messaging followed faster. iMessage shipped end-to-end encrypted by default in 2011, WhatsApp reached a billion users in 2016, and government pressure to weaken both hardened the default instead of reversing it.

That fight is still going on. Apple spent this year in the UK's Investigatory Powers Tribunal over a government order for access to encrypted iCloud content, having pulled its Advanced Data Protection feature for UK users in 2025 rather than weaken it everywhere. AI inference sits about where unencrypted web traffic sat a decade ago. The hospitals, law firms and fintech teams above aren't hiding anything; they have data they are legally obliged to protect.

Ads Are Arriving in AI Assistants

In January 2026 Senator Ed Markey wrote to the heads of eight AI companies about advertising inside chatbots. An ad delivered inside a conversation can be indistinguishable from any other response, he argued, leaving a reader no reliable way to separate a recommendation from a paid placement. He also pressed the companies on whether what people type into these systems, including conversations about health, mental health and politics, would be used to target ads. (Senator Markey's letter)

If the operator can't read your prompts, there is nothing to build a profile from. A sealed enclave removes the raw material instead of restricting what a company promises to do with it, and a policy change two years from now can't expose what was never readable in the first place.

Ad injection is harder to rule out. Attestation tells you which model answered your request, so if that model is served with no ad-insertion logic, adding some would change the measurement and fail the check. What it can't tell you is whether commercial bias was built into the setup you're attesting, or trained into the weights themselves. Markey's third concern was companies paying to weight chatbot outputs toward commercial content, and no hardware seal detects that. Knowing which open-weight model you're talking to closes the rest of that gap.

There's no evidence that prompt content is being sold, and providers say it isn't. But search, social and free email all ended up ad-funded, and advertising has already reached AI assistants. It's a matter of when rather than if, so the question worth asking before committing to a provider is what they could do with your prompts once that pressure arrives.

What changed since September

• September 2026: Mathematicians working on the Navier-Stokes problem questioned whether their unpublished work had informed OpenAI's claim to have produced a proof. OpenAI said it had never seen that work, and also said it could not rule out that de-identified data from people's use of its products had helped improve its models. We cover it in our OpenAI Navier-Stokes dispute deep dive.
• 21 September 2026: NEAR AI Cloud's confidential models, including DeepSeek, GLM, Kimi and Qwen, joined the SayGM gateway. They run on Intel TDX with NVIDIA confidential GPUs, and NEAR AI issues an attestation quote on every request. Details in NEAR AI's Confidential Models, Now on SayGM.

The Cost Trade-off of Private Inference

Confidential-capable hardware is scarcer and rents for more than the generation below it, and there's a small performance cost on top, so the infrastructure costs more to run than an equivalent unprotected setup. GPU-level confidential computing exists only on NVIDIA's two newest generations, Hopper and Blackwell, so there's no option to run it on older, cheaper silicon. The A100 and anything before it can't do it at any price, and there's no degraded mode to fall back on. (NVIDIA developer blog)

Against the same open-weight model without a TEE, the sealed version costs more. On 21 October 2026, DeepSeek V4.1 Flash was $0.07 input and $0.29 output per million tokens on SayGM, and its sealed version was $0.27 and $1.08. Against a closed frontier model the comparison can go the other way: open-weight models are cheaper to serve than frontier models, and that gap can outweigh the cost of the pricier hardware underneath. On SayGM's own catalog, confidential-tier models are frequently priced below frontier-tier ones. Current rates are published in the live model catalog and readable programmatically, so they're worth checking rather than taking from any blog post, this one included.

Not every workload needs this. A support chatbot answering product FAQs isn't sending anything sensitive, so there's nothing for a TEE to protect and no reason to pay for one.

What each route actually hides

Which Providers Offer Private Inference

SayGM is one of several private AI providers: it runs the gateway inside an Intel TDX confidential VM, and splits its catalog into privacy tiers.

Confidential-tier models are open weights running inside the enclave, where the prompt is hidden from the model maker, the host, and SayGM alike. Frontier-tier models are Claude, GPT, and Gemini reached through a sealed route, where the maker you picked still receives the request under its own terms. Both guarantees sit on one catalog, and swapping in is a base URL and a key rather than a rewrite of your OpenAI, Anthropic, or Gemini integration. To get both seals at once, pick a model marked (TEE) in the model catalog; those carry tier: "confidential" in the API, so a client can check before a request goes out. The difference between a sealed route and a sealed model is laid out in SayGM's privacy model.

Chutes runs TEE-shielded open-weight inference on Bittensor Subnet 64. Tinfoil runs client-verifiable attestation across NVIDIA, AMD and Intel confidential hardware, and publishes its security-critical infrastructure as open source. Venice AI added a client-checkable TEE attestation report to its model selector, verifiable through NEAR AI Cloud or Phala. NEAR AI runs its own TEE-secured inference mode with attestation reports validated and stored for audit. These are credible products with real track records.

The useful question by this point isn't whether a provider uses a TEE at all. It's which specific guarantee is on offer, weight protection, routing-layer attestation, or full in-enclave inference, and whether the provider is precise about which one it means.

Common Questions About Private Inference

What is private AI inference?

Private AI inference is running an AI model on your prompt inside a hardware-sealed trusted execution environment, so the server operator cannot read the prompt or the response, as of 21 October 2026. A TEE proves what code is running through an attestation check before any data is released to it. On SayGM, open-weight models in the confidential tier run entirely inside the enclave; frontier models are relayed through a sealed gateway, but the maker still receives the prompt.

Is ChatGPT safe for confidential information?

OpenAI receives every prompt sent to ChatGPT, so safety depends on its terms rather than on hardware. OpenAI's API and enterprise terms differ from consumer ChatGPT on retention and training, so which product and which agreement you use matters. That may be fine for your use case. It's a policy assurance rather than a hardware one, which is a different category of guarantee.

Is Claude AI private?

Anthropic receives every prompt sent to Claude, under its published retention and training terms. It has also done public research on confidential inference. As with any frontier model, the maker still receives your prompt, which is why SayGM lists Claude under its frontier tier.

Does private inference make my AI HIPAA or GDPR compliant?

No. Private inference protects the prompt in transit and in processing; HIPAA and GDPR compliance also depend on access controls, agreements, retention and the rest of your programme. Neither regulation names a technology. Both require demonstrable control over who can access protected data, and hardware attestation is unusually strong evidence toward that because it's a technical statement rather than a contractual one. It's one control inside a compliance programme, and it doesn't replace the rest. On SayGM, the confidential tier is where that evidence exists, and the tier a request ran on is a field in the API response.

Can I verify the attestation myself?

Yes, if the provider exposes a client-checkable attestation report; on SayGM you can check a model's tier through GET /v1/models before a request goes out. A client-checkable report is the difference between a privacy claim and a privacy proof, and any provider serious about this will expose one rather than asking you to take the badge on faith. Which tier carries which guarantee is written down in the privacy model documentation.

About SayGM

SayGM is a drop-in inference gateway for teams who don't want to just take a company's word that their prompts are private. Every request is routed through a hardware-verified confidential environment - not even SayGM can see what's inside it. That's not a policy, it's provable. Swap in your existing OpenAI, Anthropic, or Gemini code and you're covered in minutes, at transparent, published rates with no hidden markup.

Say gm to AI at saygm.com.

Website | Twitter | Discord | Blog | Medium | Docs