Used H100s Retain 95% of Value

While H100s hold value, Nvidia's next-gen Blackwell architecture, with chips like the B200 and the GB200 "superchip," is poised to significantly reset the performance landscape. The GB200 NVL72, a rack-scale system, integrates 72 Blackwell GPUs and 36 Grace CPUs, functioning as a single massive GPU to deliver up to 30 times faster real-time LLM inference compared to the H100. This leap in performance is driven by a new custom TSMC 4NP process, 208 billion transistors, and a 10 TB/s chip-to-chip interconnect. The competitive landscape is also intensifying, with AMD's MI300X offering 2.7 times more memory and 2.6 times more memory bandwidth than the H100, giving it an advantage in large model inference and leading to a 40% latency reduction for models like LLaMA2-70B. Intel's Gaudi 3 is positioned as a lower-cost alternative, claiming a 50% improvement in inferencing performance and a 40% increase in power efficiency over the H100 at a fraction of the price. An eight-accelerator Gaudi 3 baseboard is priced around $125,000, significantly undercutting comparable H100 systems. Hyperscalers are aggressively developing their own custom silicon to reduce reliance on third-party hardware and lower total cost of ownership (TCO) for their massive, predictable workloads. Google's TPU v7, Microsoft's Maia, and Amazon's Trainium chips are designed to optimize performance-per-watt for internal services like search and recommendation engines. This "build vs. buy" strategy creates a tiered hardware ecosystem where custom ASICs handle high-volume inference, while GPUs are still procured for diverse, external customer workloads on cloud platforms. The economics of AI infrastructure are staggering, with projections that global data center investment will exceed $6.7 trillion by 2030, with $5.2 trillion dedicated solely to AI. Building a single one-gigawatt AI data center can cost between $35 billion and $60 billion, driven by the high cost of GPUs and the necessity of advanced liquid cooling systems. This massive capital expenditure reflects a fundamental shift where compute capacity has become a critical economic resource. For go-to-market teams, Nvidia's strategy offers a case study in market creation and adoption acceleration. The company identifies specific industry domains, collaborates with leaders to address pain points, and provides pre-trained models and software libraries to speed up adoption. This ecosystem-building approach, combined with strategic partnerships with PC manufacturers, cloud providers, and data centers, has been crucial to its market dominance. The venture capital landscape for AI hardware remains robust, with startups in the space raising over $3 billion in Q4 2024 alone. Significant investments are flowing into areas beyond just processors, including interconnect technologies, chiplet designs, and novel materials. This funding surge highlights a broad investor belief in the continued growth and evolution of the AI hardware stack.