Agentic AI Boosts CPU Needs

Artificial intelligence agents are turning CPUs into a fresh constraint inside AI data centers, not just the GPUs that have dominated buying plans. (nvidianews.nvidia.com) A standard chatbot mostly generates tokens on GPUs. An agent does extra work around the model — planning tasks, calling tools, running code, checking results, and moving data — and chip vendors now say that surrounding work is lifting CPU demand. (amd.com) NVIDIA said on March 16 that “reasoning and agentic AI” are shifting scale, performance, and cost toward the systems that support models while they plan tasks, run tools, interact with data, run code, and validate results. The company launched its Vera CPU the same day and called it “purpose-built for the age of agentic AI and reinforcement learning.” (nvidianews.nvidia.com) That change is already visible in rack layouts. NVIDIA’s current GB200 NVL72 rack pairs 72 Blackwell GPUs with 36 Grace CPUs, or one CPU for every two GPUs. (docs.nvidia.com) NVIDIA’s next Vera Rubin NVL72 system keeps the same 72-GPU count and again uses 36 CPUs, now branded Vera. The company is also selling a separate Vera CPU rack for workloads that need dense “CPU sandboxing,” a term it uses for running large numbers of isolated agent environments. (nvidia.com; nvidia.com; developer.nvidia.com) In that CPU rack, NVIDIA says 256 liquid-cooled Vera CPUs can sustain more than 22,500 concurrent CPU environments in a single rack. The pitch is that agent systems need far more general-purpose compute beside the accelerators than earlier large language model deployments did. (nvidianews.nvidia.com) AMD is making the same case from the other side of the market. In a March 13 post, AMD said agentic inference has become a “multistep workflow” and that CPUs now handle scheduling, data preparation, memory and input-output, and control flow that keep accelerators busy. (amd.com) Intel has argued that host CPUs now shape latency and throughput for large language model serving, especially when workloads have stricter response-time targets. The company warned at GTC 2026 that unbalanced systems can leave expensive GPUs underused. (community.intel.com) The practical effect is procurement, not theory. Operators that once sized clusters mainly around accelerator counts now have to budget for more CPU sockets, more memory bandwidth, and more rack power and cooling for non-GPU gear. (developer.nvidia.com; docs.nvidia.com) That is why chipmakers are no longer marketing CPUs as background components. In 2026, they are selling them as front-line hardware for the agent era, with GPU clusters increasingly designed around how much CPU work each rack can feed, supervise, and verify. (nvidianews.nvidia.com; amd.com)