llm‑d donated to CNCF
IBM, Red Hat and Google donated 'llm‑d' — a Kubernetes blueprint for vendor‑neutral, scalable LLM inference — to the Cloud Native Computing Foundation, creating an open standard for serving models on any accelerator. The move aims to make Kubernetes‑native LLM orchestration more portable across clouds and vendors. (thenewstack.io; siliconangle.com)
llm-d was accepted into the CNCF Sandbox on March 24, 2026 during KubeCon + CloudNativeCon Europe in Amsterdam. (cncf.io)) Founding contributors named in the announcement include Google Cloud, Red Hat, IBM Research, CoreWeave and NVIDIA, with ecosystem participants listed by Red Hat that include AMD, Cisco, Hugging Face, Intel, Lambda and Mistral AI as well as university supporters at UC Berkeley and the University of Chicago. (cloud.google.com)) The project’s codebase and org describe llm-d as a Kubernetes-native stack built on vLLM and cloud‑native primitives, with components such as an inference scheduler, KV-cache service, and Inference Gateway integrations exposed in the llm-d GitHub organization. (github.com)) Core technical patterns the project packages for reproducible deployment are prefill/decode disaggregation, KV-cache‑aware routing and hierarchical KV offloading across GPU/CPU/filesystem tiers, capabilities highlighted in the llm-d v0.5 release notes. (llm-d.ai)) CNCF maintainers and the concluded Kubernetes WG Serving workstream point to llm-d leveraging existing cloud‑native building blocks such as Kueue, the Kubernetes Gateway API inference extensions and other serving primitives to integrate inference into Kubernetes operational models. (cncf.io)) Industry commentary from CoreWeave and live coverage at KubeCon frames the CNCF move as institutionalizing “well-lit paths” for production inference and bringing vendor-neutral governance to distributed LLM serving, a point emphasized by Red Hat executives during the KubeCon sessions. (coreweave.com))
