Nvidia + Groq: AI Infra Arms Race
New industry analysis pegs AI infrastructure spend at about $650B for 2026 while Nvidia’s Vera Rubin platform dominated GTC messaging — and Nvidia-related moves (including Groq‑style rack accelerators like Groq 3 LPX) are positioning the AI stack to compete for the same silicon and network resources trading desks need. The upshot: inference-focused, rack-scale accelerators are being engineered for token-speed latency, which could squeeze availability and change procurement timelines. ( )
NVIDIA’s Groq 3 LPX rack is specified to contain 256 interconnected LPUs, with each LPU offering roughly 500 MB of on‑chip SRAM and 150 TB/s of SRAM bandwidth plus 2.5 TB/s scale‑up bandwidth per device. (nvidia.com) The LPX rack is built from 32 liquid‑cooled 1U compute trays, each housing eight LP30 LPU chips, and NVIDIA positions the design specifically to sustain thousands‑of‑tokens‑per‑second decode rates for interactive inference. (blockchain.news) NVIDIA describes Groq 3 LPX as a purpose‑built low‑latency inference path paired with Vera Rubin NVL72 GPUs in a heterogeneous “GPU+LPU” serving model, with per‑token offload handled transparently inside the Vera software stack. (developer.nvidia.com) NVIDIA’s Vera Rubin platform rollout at GTC added seven new chips and integrated rack products including NVL72 GPU racks, Vera CPU racks, BlueField‑4 STX storage racks and Spectrum‑6 SPX Ethernet racks to its MGX rack portfolio. (nvidianews.nvidia.com) CEO Jensen Huang said he now sees about $1 trillion in orders for Blackwell and Vera Rubin systems through 2027, a demand signal that follows multiple industry reports of constrained GPU supply and memory bottlenecks that could tighten availability. (cnbc.com) NVIDIA is offering LPX as an optional addition to Vera Rubin configurations without prescribing a fixed LPX:NVL72 ratio, and the company signaled Groq‑derived LPU shipments are expected in the third quarter, a cadence that will force customers to choose latency‑optimized LPX capacity versus general‑purpose NVL72 capacity during procurement cycles. (servethehome.com) Independent reviewers and NVIDIA claim up to ~35× higher inference throughput per megawatt and up to ~10× more revenue potential for trillion‑parameter workloads when LPX racks are combined with Rubin NVL72, framing LPX as a specialty, power‑efficient path for token‑rate economics rather than a wholesale GPU replacement. (storagereview.com) The combined effect—LPX racks engineered for token‑speed latency, Vera Rubin’s rack portfolio, and a multi‑hundred‑billion dollar buildout forecast—creates a scenario where buy cycles, rack power, DPU/Ethernet port counts and high‑performance interconnect slots will be the constrained procurement variables for low‑latency trading deployments. (nvidia.com)
