Confidential AI

Confidential AI is the practice of running machine learning training and inference inside hardware-based trusted execution environments (TEEs), so that data, prompts, and model weights stay protected even from the cloud provider, system administrators, and the host operating system. Encryption has long protected data at rest on disk and in transit across networks. Confidential AI closes the remaining gap, data in use, which conventional systems decrypt into CPU and GPU memory while a model processes it. It applies confidential computing, a field the Linux Foundation's Confidential Computing Consortium has promoted since 2019, to AI workloads.

How Confidential AI Works#

Three hardware mechanisms make the guarantee possible:

• Isolated execution: CPU technologies such as Intel TDX and AMD SEV-SNP run an entire virtual machine in memory encrypted with keys the hypervisor never sees, while Intel SGX isolates smaller enclaves inside a single application. NVIDIA added a confidential computing mode to its data center GPUs starting with the Hopper generation, which encrypts data moving between CPU and GPU and blocks the host from reading GPU memory.
• Remote attestation: Before receiving any secret, the TEE produces a signed report, rooted in keys fused into the chip, that lists the exact hardware, firmware, and software it is running. A data owner or key management service compares that report against expected values.
• Attested key release: Decryption keys for datasets or model weights go only to environments whose attestation passes, so protected assets are readable only inside the verified workload.

The performance cost is concentrated in data movement. Encrypting memory and CPU-to-GPU transfers adds overhead, while computation inside the GPU runs at full speed, so compute-heavy jobs such as large language model inference see only modest increases in inference latency.

What It Protects Against#

The threat model is a compromised or curious infrastructure layer: a malicious insider at the cloud computing provider, a hacked hypervisor, or another tenant on the same physical server. The protection runs in both directions. Data owners can send sensitive records to a model they do not control, and model owners can deploy proprietary weights to a partner's data center or an edge site without exposing them there. Because attestation lets a customer verify who can read its data, confidential computing is also a building block for sovereign AI offerings that run on foreign-owned hardware.

Confidential AI secures the execution environment, and it pairs with controls that cover the rest of the pipeline. Model outputs can still reveal training data through membership inference attacks, which training with differential privacy limits, and the application code inside the enclave still needs standard data security practices such as access control and patching.

Related Privacy Techniques#

• Federated learning keeps raw data on each participant's device or server and shares only model updates. Running the aggregation server in a TEE adds protection for those updates.
• Homomorphic encryption computes directly on encrypted data with no trusted hardware at all, but it remains orders of magnitude slower than TEEs for deep learning, so it is reserved for small models and simple operations.
• Edge AI avoids the question by processing data on the device that captured it, which works whenever the model is small enough to run there.

Where Confidential AI Is Used#

Private cloud assistants are the most visible deployments. Apple's Private Cloud Compute, introduced in 2024, handles Apple Intelligence requests too large for the device on Apple silicon servers that retain no user data after a request, give staff no privileged access, and publish their software images so independent researchers can verify what runs. Google's Private AI Compute, announced in November 2025, applies the same design to Gemini models on its own TPUs, with devices checking attestation before sending data.

Healthcare collaboration is another. Hospitals that cannot share patient records can let an outside developer validate or train a model for medical image analysis on their data inside a TEE. The hospital never sees the model weights, and the developer never sees the patients' scans, a pattern cloud providers support with services for confidential AI on Azure and elsewhere.

Vision systems also process personal data: cameras in stores, clinics, and workplaces capture faces and other information covered by data privacy law. Because confidential virtual machines run unmodified applications, an Ultralytics YOLO26 model exported to ONNX or TensorRT deploys inside one the same way it would on any GPU server, keeping each frame encrypted from the cloud operator while it is analyzed.