Nvidia Prepares Next-Gen AI Hardware

Why it matters

- The collaboration with Tower Semiconductor is centered on their advanced silicon photonics (SiPho) platform, which is stated to enable up to double the data rate compared to previous solutions. Key executives involved are Russell Ellwanger, CEO of Tower Semiconductor, and Gilad Shainer, Senior Vice President of Networking at NVIDIA. - The move to 1.6T Ethernet interconnects is a direct response to the massive data throughput requirements of modern AI workloads, as traditional 400G architectures are no longer sufficient. These large, sustained data transfers, known as "elephant flows," can create congestion and latency in conventional networks, becoming a significant performance bottleneck. - At a previous GTC event, Nvidia unveiled its "Spectrum-X" (Ethernet) and "Quantum-X" (InfiniBand) co-packaged optics platforms, developed with partners including TSMC, Lumentum, and Coherent. The Quantum-X system is designed with 144 ports of 800Gb/s, delivering a total bandwidth of 115.2Tbps. - This push into co-packaged optics and silicon photonics aims to address critical challenges in power consumption and thermal management in data centers. As AI hardware becomes more power-intensive, integrating optics directly with silicon reduces energy loss over longer electrical traces found in traditional pluggable modules. - Analysts anticipate that Nvidia's upcoming GTC, scheduled for March 16-19 in San Jose, will feature the launch of a second-generation co-packaged optic switch. Other potential announcements include updates on the Feynman line of GPUs and a new language processing unit rack featuring SRAM-based on-chip memory for ultra-low latency inference. - The transition to 1.6T optical modules is seen as a crucial step for the industry, enabling a doubling of bandwidth capacity within the same rack unit, a key factor for hyperscale data centers facing space and power constraints. The market for datacom optical modules is projected to grow from over $6.25 billion in 2023 to $25.8 billion by 2029.

Key numbers

• To support its infrastructure, Nvidia has also partnered with Tower Semiconductor to use silicon photonics for 1.6T data center optical modules.
• The move to 1.6T Ethernet interconnects is a direct response to the massive data throughput requirements of modern AI workloads, as traditional 400G architectures are no longer sufficient.
• The Quantum-X system is designed with 144 ports of 800Gb/s, delivering a total bandwidth of 115.2Tbps.
• Analysts anticipate that Nvidia's upcoming GTC, scheduled for March 16-19 in San Jose, will feature the launch of a second-generation co-packaged optic switch.

What happens next

• This push into co-packaged optics and silicon photonics aims to address critical challenges in power consumption and thermal management in data centers.
• Analysts anticipate that Nvidia's upcoming GTC, scheduled for March 16-19 in San Jose, will feature the launch of a second-generation co-packaged optic switch.
• Nvidia is expected to unveil significant AI hardware innovations, including co-packaged optics and a new AI inference rack, at its upcoming GTC event.

Sources

• is expected
• also partnered
• The collaboration with
• The move to 1.6T Ethernet
• These large, sustained
• At a previous GTC event
• The Quantum-X system
• As AI hardware becomes
• Analysts anticipate
• The transition to 1
• The market for datacom

Quick answers