Confidential Computing Platform for AI Goes Open-Source
An open-source confidential computing platform named dstack has been released to enhance security for AI workloads. The platform uses Trusted Execution Environments (TEE) to enable private AI and secure data processing. Its compatibility with Docker is designed to allow for rapid, code-free deployment of confidential services.
- The platform, dstack, functions as a trustless orchestration layer for AI workloads, managing GPU runners, scheduling, and attestation, similar to a confidential version of Kubernetes. It is framework-agnostic, supporting PyTorch, TensorFlow, JAX, and custom CUDA code. - dstack is a project under the Linux Foundation's Confidential Computing Consortium, which is focused on securing data in use through open-source collaboration. It was developed by Phala Network, with contributions from individuals at Flashbots and Nethermind. - The system uses a combination of a guest agent running inside a Confidential VM (CVM), a key management service (KMS) running in its own TEE, and a minimized OS image to abstract the underlying hardware. This architecture ensures that application logic is protected by the CPU TEE, while model weights and inference data are secured by the GPU TEE. - For hardware, dstack currently supports Intel TDX (4th/5th Gen Xeon processors) and NVIDIA Confidential Computing on H100 and Blackwell GPUs, with plans to support AMD SEV-SNP in the future. This allows for cryptographic verification of what is running in both the CPU and GPU. - The platform provides cryptographic attestation for every application, allowing users to verify the integrity of the code and data being processed. This is designed to address scenarios like private inference, training on sensitive data, and ensuring AI agents cannot exfiltrate user data. - dstack is designed to be a full-stack solution, providing attestation verification, key management, and Docker orchestration out-of-the-box, unlike cloud providers like AWS, Azure, and GCP which offer the basic hardware primitives. - Phala Network, the primary developer of dstack, is in the process of migrating from its own Khala chain to Ethereum. This move is intended to leverage newer technologies like Intel TDX and NVIDIA GPUs for better performance and security in AI tasks. - The platform is already being used in production for AI infrastructure at companies like OpenRouter and NEAR AI. It also integrates with tools like SGLang's Model Gateway for disaggregated, low-latency inference.
