- Micron introduces dense 256 GB LPDDR5x module aimed directly at AI servers
- Eight SOCAMM2 modules can push server memory capacity to a massive 2TB
- AI-initiated workloads are increasingly shifting performance bottlenecks towards system memory capacity
Large language models (LLMs) and modern inference pipelines increasingly require huge memory pools, forcing hardware vendors to rethink server memory architecture.
Micron has now introduced a 256 GB SOCAMM2 memory module intended for data center systems where capacity, bandwidth and power efficiency all affect overall performance.
The module relies on 64 monolithic 32GB LPDDR5x chips that form a dense LPDRAM package that addresses the growing memory footprint required by today’s AI workloads.
The article continues below
Scaling memory capacity for AI server platforms
The module also increases the maximum available memory per processor configuration. When eight of these SOCAMM2 modules are installed in an eight-channel server CPU, the total capacity can reach 2 TB of LPDRAM.
This number exceeds the previous 192GB module generation by about a third, enabling systems to accommodate larger context windows and more demanding inference tasks.
Micron describes the SOCAMM2 design as more efficient than conventional server memory modules.
“Micron’s 256GB SOCAMM2 offering enables the most power-efficient CPU-attached memory solution for both AI and HPC,” said Raj Narasimhan, senior vice president and general manager of Micron’s Cloud Memory Business Unit.
“Our continued leadership in low-power memory solutions for data center applications has uniquely positioned us to be the first to deliver a 32Gb monolithic LPDRAM die, helping to drive industry adoption of more power-efficient, high-capacity system architectures.”
The company says the new LPDRAM module uses about a third of the power of comparable RDIMMs while taking up only a third of the physical footprint.
Its reduced power requirements and smaller module size allow higher rack density in large data center installations, while lower memory power reduces thermal load and infrastructure costs.
The SOCAMM2 architecture also follows a modular design intended to simplify maintenance and future expansion.
The format supports liquid-cooled server systems and can accommodate additional capacity as memory requirements grow along with model complexity and dataset scale.
Micron says the 256 GB SOCAMM2 module may affect certain inference operations under unified memory architectures.
The company reports more than a 2.3x improvement in time-to-first-token during long context inference when the module is used for key-value cache reading.
In standalone CPU workloads, the LPDRAM configuration reportedly delivers over 3x better performance per watts compared to regular server memory modules.
The company’s LPDRAM portfolio spans components from 8GB to 64GB and SOCAM2 modules ranging from 48GB to 256GB, and customer samples of the 256GB SOCAMM2 module are already shipping.
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews and opinions in your feeds. Be sure to click the Follow button!
And of course you can too follow TechRadar on TikTok for news, reviews, video unboxings, and get regular updates from us on WhatsApp also.
