Micron Unveils Ultra-Efficient Data Center Memory
Micron has set a new benchmark with the first high-capacity 256GB LPDRAM memory module for data centers. The new SoCAMM2 format is designed for AI workloads, using one-third of the power and occupying a significantly smaller footprint than standard RDIMMs. This addresses key energy and space efficiency challenges as AI drives up server demands.
This new memory is built on Micron's industry-first 32Gb monolithic LPDDR5X die. This technological foundation allows a single 8-channel server CPU to be paired with up to 2TB of LPDRAM, a 33% increase in capacity over the previous 192GB SoCAMM2 modules. The SoCAMM2 standard, developed by JEDEC with contributions from companies like Micron and NVIDIA, is designed to bring the power efficiency of LPDDR memory, traditionally used in mobile devices, to the data center. Unlike conventional LPDDR solutions that are soldered directly to the motherboard, the modular SoCAMM2 design allows for easier serviceability, upgrades, and supports liquid-cooled server architectures. For artificial intelligence workloads, these modules can significantly accelerate performance. In tests running large language models like Llama3-70B, the increased memory capacity allows for a 2.3 times faster "time to first token" during real-time inference. For other high-performance computing tasks in standalone CPU applications, the new memory delivers over three times the performance per watt compared to standard RDIMMs. This innovation comes as data center electricity consumption is projected to more than double between 2024 and 2030, largely driven by the demands of AI. The International Energy Agency (IEA) estimates that data centers consumed about 1.5% of global electricity in 2024, a figure expected to rise significantly. More efficient components are critical to managing this growth and the associated operational costs.
