Report flags 5% average GPU utilization

Gilles, in a May 21 post on X, argued that the AI infrastructure boom is colliding with a deployment problem: enterprises are buying large amounts of GPU capacity but using little of it. The post tied a 5% average GPU utilization figure to Gartner’s estimate that AI infrastructure will add $401 billion in spending in 2026. The utilization figure traces to Cast AI’s April 21 report on non-optimized Kubernetes clusters, while Gartner’s January 15 forecast put total worldwide AI spending at $2.52 trillion this year. Taken together, the numbers have become a shorthand for a broader complaint about idle AI capacity. ### Where does the 5% figure come from? Cast AI said on April 21 that GPU utilization averaged 5% across the clusters analyzed in its 2026 State of Kubernetes Optimization Report. The company said the report drew on real-world utilization data from tens of thousands of Kubernetes workloads and focused on non-optimized clusters. Cast AI also reported average CPU utilization of 8% and memory utilization of 20%. (gartner.com) Laurent Gil, Cast AI’s co-founder and president, said in the release that “95% of GPU capacity is doing nothing.” The company framed the finding as a cost problem for organizations expanding GPU-equipped nodes for AI and machine-learning workloads on Kubernetes. ### What is the $401 billion number measuring? Gartner said on January 15 that AI infrastructure will add $401 billion in spending in 2026 as technology providers build out AI foundations. (cast.ai) In the same forecast, Gartner said worldwide AI spending would total $2.52 trillion in 2026, up 44% from the prior year, and that AI infrastructure spending would reach $1.366 trillion. John-David Lovelock, a distinguished vice president analyst at Gartner, said AI adoption is shaped by “the readiness of both human capital and organizational processes, not merely by financial investment.” He also said enterprises were increasingly prioritizing proven outcomes over speculative projects and that improved predictability of return on investment must occur before AI can scale more broadly inside companies. (gartner.com) ### Why are people linking those two numbers? VentureBeat reported on May 8 that the 5% utilization figure and Gartner’s $401 billion spending estimate point to a gap between capacity procurement and productive deployment. The article said enterprises secured GPU reservations during the shortage period, but many internal teams were still dealing with data readiness, governance and architecture issues. (gartner.com) The X thread summarized that problem in operational terms, pointing to resource allocation, scheduling and tooling gaps rather than chip scarcity alone. That framing is broadly consistent with Gartner’s emphasis on organizational readiness and with Cast AI’s argument that one-time configuration is not enough because workloads and traffic patterns change over time. (venturebeat.com) ### Does 5% mean all AI GPUs are idle? Cast AI’s 5% figure applies to the non-optimized Kubernetes clusters in its dataset, not to every GPU deployment in the market. Gartner’s spending forecast, meanwhile, covers worldwide AI spending and provider buildout, not a direct measure of enterprise utilization. The numbers are often paired as an illustration of mismatch, but they come from different sources measuring different parts of the market. (gartner.com) VentureBeat described the result as a sign that the market is shifting from securing capacity to improving the output of already deployed systems. That is an interpretation from the publication, not a direct finding from Gartner or Cast AI. ### What comes next for companies watching this? Gartner’s 2026 forecast and Cast AI’s April report give enterprise buyers a pair of benchmarks to watch through the rest of the year: how much AI infrastructure spending continues to rise, and whether utilization improves as organizations change scheduling, rightsizing and optimization practices. (gartner.com) Gartner said AI infrastructure spending is projected to rise to $1.748 trillion in 2027. (venturebeat.com)