AI Chip Spending to Approach $1 Trillion; HBM4 Memory Nears

- The global AI chip market is projected to grow from approximately $121.73 billion in 2026 to over $1.1 trillion by 2035, demonstrating a compound annual growth rate of 27.88%. Another forecast predicts the market could approach $500 billion in 2026. This growth is largely driven by the expansion of AI data centers and the increasing integration of AI into consumer electronics and automotive systems. - HBM4 doubles the interface width of its predecessor, HBM3, to 2048 bits per stack, enabling a significant increase in bandwidth to over 2 TB/s. This architectural change is critical for feeding next-generation AI accelerators and mitigating the "memory wall" bottleneck, where processor speeds outpace memory performance. - The manufacturing of HBM4 is notably complex, involving the stacking of up to 16 DRAM dies connected by thousands of through-silicon vias (TSVs). This 3D stacking process contributes to higher production costs compared to traditional memory like GDDR, with the price for a 12-Hi HBM4 stack potentially exceeding $600. - Key players in the HBM4 supply chain include memory manufacturers SK Hynix, Samsung, and Micron, with SK Hynix projected to hold a majority of the market share for NVIDIA's orders. NVIDIA's next-generation AI accelerator, codenamed "Vera Rubin," is expected to be a primary consumer of HBM4 memory. - The base die of HBM4 memory is being produced on advanced logic processes, such as 4nm, a shift from the legacy memory processes used for HBM3. This allows for more sophisticated on-chip logic and potentially customized solutions for major clients like NVIDIA and AMD. - While AI model training has historically driven demand for high-bandwidth memory, the industry is seeing a significant shift toward AI inference, which emphasizes real-time responsiveness. This trend will sustain demand for high-capacity, high-bandwidth memory in server and edge computing applications. - Power efficiency is a central focus for HBM4, with the standard supporting lower voltage levels to manage the thermal challenges in densely packed AI systems. The intricate power delivery network design is a critical challenge due to HBM4's increased power demands at these lower voltages. - The rapid two-to-two-and-a-half-year innovation cycle for HBM technology, much faster than previous memory standards, presents significant challenges for development and testing. This accelerated cadence is driven by the yearly release schedule of major AI chip vendors.