HBM at a Crossroads: Customization and 3D Stacking Take Center Stage
TREE NEWS reports: At this year’s Hot Chips conference, Samsung, Micron, SK Hynix, and Nvidia laid out competing roadmaps for High Bandwidth Memory (HBM), the critical component powering AI accelerators. The consensus: HBM is evolving from a standardized commodity into a customizable platform, with three-dimensional vertical stacking as the shared architectural direction. Thermal and power constraints now dominate design decisions.
What Happened
Samsung detailed a three-phase roadmap to transform the HBM base die from a simple communication layer into a customizable AI platform. Its ultimate vision, called ZHBM, involves stacking HBM directly atop the XPU, eliminating the traditional 2.5D interposer. Samsung claims this could cut total DRAM power by ~70% versus HBM5 (over 100W absolute) and boost bandwidth by 2.3x. Meanwhile, Nvidia revealed it has at least three RVA23 processors in its labs and completed the first public demonstration of CUDA running on RISC-V hardware, signaling a potential opening of its ecosystem to third-party custom CPUs via NVLink Fusion.
Market Impact: Supply Squeeze Persists
Storage analyst Jim Handy highlighted a structural mismatch: HBM yields only one-third the gigabytes per wafer compared to standard DDR, and the industry hasn’t built major new fabs in over a decade. Even aggressive timelines require ~2 years for new capacity. As a result, HBM spot prices have surged roughly 7x, and Samsung, SK Hynix, Micron, and NAND suppliers are seeing exceptional revenue growth.
This supply-demand imbalance is rippling across memory markets. AI infrastructure is also tightening NAND and HDD supply, with QLC SSDs filling new storage tiers and even replacing HDDs in some cases. Companies like Kioxia and SanDisk (SNDK) are direct beneficiaries.
Investor Takeaways
- Memory pricing power: Expect continued strength in HBM and broader memory pricing through 2025-2026, benefiting suppliers like Samsung, SK Hynix, Micron, and Western Digital/SanDisk.
- Architecture shifts: Custom base dies and hybrid bonding (fusion bonding) will become key differentiators. Companies with advanced packaging capabilities (TSMC, ASE) and equipment makers (Besi, ASML) stand to gain.
- Nvidia’s RISC-V pivot: If CUDA runs on RISC-V, it could loosen Nvidia’s grip on the AI accelerator ecosystem, creating opportunities for custom silicon startups (e.g., Tenstorrent, RISC-V IP vendors like SiFive) — but also risks fragmenting Nvidia’s software moat.
- Thermal management: With power density becoming the top constraint, companies specializing in advanced cooling (liquid cooling, thermal interface materials) will see rising demand.
Ultimately, the AI compute buildout is not slowing down. Any efficiency gains are being reinvested into more tokens processed, not reduced capex. For investors, the memory supply chain remains one of the most tightly held bottlenecks in the AI trade.
