What Is Private Inference? A Guide to TEE-Based AI

Private inference is AI inference where nobody can see your prompt. When it's done inside a trusted execution environment, or TEE, that holds even though the model is running on someone else's machine: the hardware stops the operator, the host and the gateway from reading what you sent, and you can check that seal is real before you send anything.

Table of Contents

• How Private Inference Works
• When the Prompt Is the Sensitive Asset
• Why Encrypt Prompts If You Have Nothing to Hide?
• Ads Are Arriving in AI Assistants
• The Cost Trade-off of Private Inference
• Which Providers Offer Private Inference
• Common Questions About Private Inference
• Is Private Inference Right for Your Workload?

Private inference is what happens when an AI model processes your prompt inside hardware that's sealed off from whoever operates it. No one running the server can see what you sent or what came back, and neither can the gateway, the host, or the company that owns the infrastructure. The technology behind that claim is called a trusted execution environment, or TEE, and it's becoming the standard way to prove a privacy claim about AI rather than just state one.

Most AI providers ask you to trust a privacy policy: a promise that they won't look at your prompts, won't log them, won't train on them. A TEE replaces that promise with hardware you or an independent auditor can verify. This post covers what that protection does, how it works under the hood, and how to tell one privacy claim from another. If you want the product-level version of this first, our introduction to a gateway built around one covers the same mechanism from the other direction.

How Private Inference Works

A trusted execution environment is a sealed region of a processor where code runs isolated from everything else on the machine, including the operator of that machine. Two separate pieces of hardware need to be sealed for this to work for AI inference specifically. The CPU side, running on chips like Intel TDX or AMD SEV-SNP, seals the host so the hypervisor and operating system underneath can't read what's happening inside. The GPU side, on NVIDIA's Hopper or Blackwell generations, seals the accelerator itself, since a CPU-only boundary does little good if an attacker can just read the model weights or your prompt straight out of GPU memory.

Encryption at rest and in transit leave one window open: data has to be decrypted to be computed on, and whoever runs that machine can read it while that's happening. Closing that window is what the industry calls data in use encryption, and a secure enclave on the processor is the mechanism that does it.

Before any data moves, the system runs a remote-attestation handshake. A client asks the enclave to prove it's running genuine hardware with an approved, unmodified piece of code, and the decryption keys that let a prompt be processed are released only once that proof checks out. Nothing travels in the clear before that handshake completes.

Even with HTTPS, once your data reaches the server, you're trusting the entire stack behind it. Weights could be extracted, prompts could be logged, the inference itself could be manipulated. Private inference closes that gap by moving the trust boundary from a company's word to hardware anyone can independently check.

When the Prompt Is the Sensitive Asset

A hospital using AI to draft clinical notes or triage patient messages is sending protected health information into whatever processes that request. A business associate agreement is a legal promise not to misuse that data. A TEE is a hardware guarantee the infrastructure operator can't read it in the first place, which is the difference between "we won't look" and "we can't."

A law firm running contract review through an AI tool is sending privileged material somewhere, and privilege can be compromised the moment a third party has technical access to it, even one that swears it will never look. A fintech team doing fraud detection or underwriting is handling regulated financial records that carry both compliance exposure and breach risk if they sit in a vendor's logs indefinitely. A software team feeding proprietary code through an AI coding assistant has a version of the same problem: that code is valuable, and most developers have little visibility into what happens to it once it leaves their editor.

In September 2026, mathematicians working on the Navier-Stokes problem questioned whether their unpublished work had informed OpenAI's claim to have produced a proof of it. OpenAI said it had never seen that work and had accessed no specific user data, while adding that, though it called this unlikely, it could not rule out that de-identified data from people's use of its products had helped improve its models. Whether a provider opened your session and whether your work reached its models through training are two separate things, and it is easy to be told the first and assume you have been told the second. We look at the dispute, and at what it means for choosing between a frontier model and a confidential one, in an upcoming post.

None of this depends on how large or well known the provider is. A policy-based guarantee is a bet on a company's reputation and its incentive to protect it, which means your protection is only ever as good as that company's word. Hardware attestation removes the need to trust anyone at all. You can check for yourself, before you send a prompt, that the machine running it cannot read what you send. That check is what SayGm's privacy claim rests on.

[IMAGE: Before the "Why Encrypt Prompts If You Have Nothing to Hide?" heading. A timeline of encryption defaults: SSL ships, Let's Encrypt, Chrome marks HTTP "Not Secure" in 2018, HTTPS majority, with AI inference marked as an open point at the right-hand end. Alt text: "Timeline of encryption becoming default on the web, with AI inference marked as unresolved." Suggested filename: encryption-default-timeline-private-inference.png]

Why Encrypt Prompts If You Have Nothing to Hide?

The nothing-to-hide question has come up before, in the same form, every time a category of technology moved to encrypted by default, and it's never held up well. Encryption on the web stayed a niche add-on for close to two decades after SSL shipped, mostly reserved for bank logins and checkout pages. Nobody won that argument. The friction just went away: Let's Encrypt made certificates free, Chrome started marking plain HTTP sites "Not Secure" in 2018, and HTTPS went from a minority of traffic to the overwhelming majority. Messaging followed faster. iMessage shipped end-to-end encrypted by default in 2011, WhatsApp reached a billion users in 2016, and government pressure to weaken both hardened the default instead of reversing it.

That fight is still going on. Apple spent this year in the UK's Investigatory Powers Tribunal over a government order for access to encrypted iCloud content, having pulled its Advanced Data Protection feature for UK users in 2025 rather than weaken it everywhere. AI inference sits about where unencrypted web traffic sat a decade ago. The hospitals, law firms and fintech teams above aren't hiding anything; they have data they are legally obliged to protect.

Ads Are Arriving in AI Assistants

In January 2026 Senator Ed Markey wrote to the heads of eight AI companies about advertising inside chatbots. An ad delivered inside a conversation can be indistinguishable from any other response, he argued, leaving a reader no reliable way to separate a recommendation from a paid placement. He also pressed the companies on whether what people type into these systems, including conversations about health, mental health and politics, would be used to target ads. (Senator Markey's letter)

If the operator can't read your prompts, there is nothing to build a profile from. A sealed enclave removes the raw material instead of restricting what a company promises to do with it, and a policy change two years from now can't expose what was never readable in the first place.

Ad injection is harder to rule out. Attestation tells you which model answered your request, so if that model is served with no ad-insertion logic, adding some would change the measurement and fail the check. What it can't tell you is whether commercial bias was built into the setup you're attesting, or trained into the weights themselves. Markey's third concern was companies paying to weight chatbot outputs toward commercial content, and no hardware seal detects that. Knowing which open-weight model you're talking to closes the rest of that gap.

There's no evidence that prompt content is being sold, and providers say it isn't. But search, social and free email all ended up ad-funded, and advertising has already reached AI assistants. It's a matter of when rather than if, so the question worth asking before committing to a provider is what they could do with your prompts once that pressure arrives.

The Cost Trade-off of Private Inference

Confidential-capable hardware is scarcer and rents for more than the generation below it, and there's a small performance cost on top, so the infrastructure costs more to run than an equivalent unprotected setup. GPU-level confidential computing exists only on NVIDIA's two newest generations, Hopper and Blackwell, so there's no option to run it on older, cheaper silicon. The A100 and anything before it can't do it at any price, and there's no degraded mode to fall back on. (NVIDIA developer blog)

That doesn't automatically mean a higher bill for whoever's calling the API, though. Open-weight models are inherently cheaper to serve than closed frontier models, and that gap can outweigh the cost of the pricier hardware underneath. On SayGm's own catalog, for example, confidential-tier models are frequently priced below frontier-tier ones. The infrastructure costs more and the price to a developer can still come out lower, because those are two different questions. Current rates are published in the live model catalog and readable programmatically, so they're worth checking rather than taking from any blog post, this one included.

Not every workload needs this. A support chatbot answering product FAQs isn't sending anything sensitive, so there's nothing for a TEE to protect and no reason to pay for one.

[IMAGE: Before the "Which Providers Offer Private Inference" heading. Two request paths side by side: one ending inside the enclave, one passing through a sealed gateway to an external model maker, with different "who can read this" labels on each. Alt text: "Frontier routing versus in-enclave private inference and who can read the prompt in each." Suggested filename: frontier-routing-vs-in-enclave-inference.png]

Which Providers Offer Private Inference

SayGm runs the gateway inside an Intel TDX confidential VM, and splits its catalog into tiers that are named in the API rather than in marketing copy.

Confidential-tier models are open weights running inside the enclave, where the prompt is hidden from the model maker, the host, and SayGm alike. Frontier-tier models are Claude, GPT, and Gemini reached through a sealed route, where the maker you picked still receives the request under its own terms. Both guarantees sit on one catalog, and swapping in is a base URL and a key rather than a rewrite of your OpenAI, Anthropic, or Gemini integration. To get both seals at once, pick a model marked (TEE) in the model catalog; those carry tier: "confidential" in the API, so a client can check before a request goes out. We'll unpack the difference between a sealed route and a sealed model in an upcoming post.

Chutes runs TEE-shielded open-weight inference on Bittensor Subnet 64. Tinfoil runs client-verifiable attestation across NVIDIA, AMD and Intel confidential hardware, and publishes its security-critical infrastructure as open source. Venice AI added a client-checkable TEE attestation report to its model selector, verifiable through NEAR AI Cloud or Phala. NEAR AI runs its own TEE-secured inference mode with attestation reports validated and stored for audit. These are credible products with real track records.

The useful question by this point isn't whether a provider uses a TEE at all. It's which specific guarantee is on offer, weight protection, routing-layer attestation, or full in-enclave inference, and whether the provider is precise about which one it means.

Common Questions About Private Inference

Is ChatGPT safe for confidential information?

It depends which product and which agreement, since OpenAI's API and enterprise terms differ from consumer ChatGPT on retention and training. What doesn't vary is that OpenAI receives the prompt. That may be fine for your use case. It's a policy assurance rather than a hardware one, which is a different category of guarantee.

Is Claude AI private?

Anthropic publishes retention and training terms and has done public research on confidential inference. As with any frontier model, the maker still receives your prompt, which is why SayGm lists Claude under its frontier tier.

Does private inference make my AI HIPAA or GDPR compliant?

No. Neither regulation names a technology. Both require demonstrable control over who can access protected data, and hardware attestation is unusually strong evidence toward that because it's a technical statement rather than a contractual one. It's one control inside a compliance programme, and it doesn't replace the rest. On SayGm, the confidential tier is where that evidence exists, and the tier a request ran on is a field in the API response.

Can I verify the attestation myself?

You should be able to. A client-checkable attestation report is the difference between a privacy claim and a privacy proof, and any provider serious about this will expose one rather than asking you to take the badge on faith. On SayGm you can check the tier programmatically through GET /v1/models before a request goes out, and which tier carries which guarantee is written down in the privacy model documentation.

Is Private Inference Right for Your Workload?

If your prompts are sensitive, regulated data, proprietary code, anything you wouldn't want sitting in a provider's logs indefinitely, private inference is solving a real problem, the same way TLS solved a real one for anyone who used to send a password over plain HTTP. If your prompts are already public, the overhead isn't buying you anything.

If your workload is in the first category, the fastest way to find out what it costs you is to point an existing integration at a confidential endpoint and compare. The quickstart gets you to a first request in a few minutes.

About SayGm

SayGm is a drop-in inference gateway for teams who don't want to just take a company's word that their prompts are private. Every request is routed through a hardware-verified confidential environment - not even SayGm can see what's inside it. That's not a policy, it's provable. Swap in your existing OpenAI, Anthropic, or Gemini code and you're covered in minutes, at transparent, published rates with no hidden markup.

Say gm to AI at saygm.com.

Website | Twitter | Discord | Blog | Medium | Docs