High-bandwidth memory v4 supply takes shape

Samsung, SK hynix, and Micron are gearing up to supply v4 high-bandwidth memory (HBM) to Nvidia and other AI accelerator suppliers to make AI training and inference faster.

HBM is the layered DRAM placed right next to a GPU on a bridging interposer chip that provides greater memory capacity and higher memory bandwidth than the socket-connected DRAM used with x86 CPUs. It comes in generations designed for specific GPUs, with each generation generally providing higher bandwidth and more capacity than the preceding one. We are currently in the HBM3E (extended) generation, designed for Nvidia Blackwell GPUs, with up to a 12-Hi stack of DRAM layers offering 36 GB of capacity, a 9.2-9.8 Gbps data transfer speed per pin, and 1.2-1.28 TB/s overall bandwidth.

HBM4 has been designed for Nvidia’s Rubin GPU, which can have up to 288 GB of HBM4 attached. AMD’s MI455 can have more – 432 GB. JEDEC’s HBM4 standard supports 4-Hi, 8-Hi, 12-Hi, and 16-Hi stacks, with die capacities of either 24 Gb (3 GB) or 36 Gb (4 GB). Per-stack capacities then range from 12 GB to 64 GB. HBM4 has an 8 GB/s baseline per pin data transfer rate with support for scaling to higher levels.

Nvidia wanted 13 TB/s of overall HBM4 bandwidth when it first announced Rubin last year but is now targeting 22 TB/s from a 12-Hi HBM4 stack.

This caused vendors developing HBM4 such as Micron, Samsung, and SK hynix to test their data rates, with Samsung having to optimize its design to achieve good yields at the higher speed. Reuters and the South Korea Daily outlet report that Samsung met the new speed requirement and will begin shipping HBM4 chips to Nvidia in February. It will also be supplying AMD with HBM4 chips.

A Bloomberg report differs, saying Samsung is close to getting certification from Nvidia, but doesn’t have a shipping agreement yet.

HBM market leader SK hynix, which has delivered HBM4 samples to Nvidia, has not disclosed when its HBM4 chips will be shipping to GPU makers. It announces its Q4 earnings on January 29, as does Samsung, and we might hear then.

Micron, the third HBM4 manufacturer, has also delivered 16-Hi samples to Nvidia, and said its full 2026 HBM4 production capacity is sold out.

The HBM5 and HBM6 generations could feature 20-Hi and 24-Hi stacks respectively.