What’s the M in MWC stand for? Memory if you're Micron or SK Hynix

Memory vendors used MWC week to highlight their AI focused memory chips this week, touting their potential to reconfigure the bit barn infrastructure balance.

Though whether these efforts will ripple down to the enterprise and consumer device markets which are being pummeled by soaring storage and memory prices is another story.

Micron this week unwrapped its 256GB LPDRAM SOCAMM2 parts, targeted specifically at datacenter infrastructure. The parts will consume a third of the power, and take up a third of the footprint of standard RDIMMs the firm said. 

They will be manufactured on what the firm said was “the industry's first monolithic 32Gb LPDDR5X die.”

Micron promised 1.33 times more capacity per module, which it said meant enabling “2TB LPDRAM per 8-channel server CPU for both AI and high-performance compute (HPC).”

This being an AI focused announcement, the firm highlighted that it is “collaborating with NVIDIA to co-design sophisticated memory for the needs of advanced AI infrastructure.”

More specifically, the parts will enable larger context windows and more complex inference workloads. This is turn will potentially ease the burden on fast SSDs. The supply of which is almost as precarious as that of memory.

Micron claimed that in unified memory architectures, the 256GB part “improves time to first token by more than 2.3 times for long context, real-time LLM inference when used for KV cache offload compared to currently available solutions.”

At the same time, its reduced power consumption will allow denser racks and, perhaps even reduce power bills through lower power consumption and heat generation.

Meanwhile SK Hynix chose to highlight its memory offerings at MWC this week – full aware what the M in the name actually stands for. This included High Bandwidth Memory and Datacentre Memory, as well as On-Device AI Memory and Automotive Memory. That included its HMB4, unwrapped last year, and 12 layer HBM3E parts. 

However, reports this week suggested that SK Hynix, and Samsung, were struggling to deliver precisely the bandwidth spec originally envisioned for use of HMBM4 memory in NVIDIA’s Rubin GPUs. The original spec was for 22TB/s but the reality seems to be nearer 20TB/s, according to Techpowerup.