AI’s Real Bottleneck Isn’t Chips: Veteran Chip Designer Says Packaging, Interconnect and Memory Now Rule
TREE NEWS reports: A veteran chip designer known in tech circles as “Mr. Bubble” has delivered a blunt reassessment of the AI compute supply chain: the constraint on artificial intelligence is no longer the logic die itself, but advanced packaging, interconnect latency and the memory hierarchy. In a wide-ranging interview, the hardware analyst argued that Moore’s Law has effectively gone bankrupt on an economic basis, and that the industry’s center of gravity has shifted from “designing faster chips” to “controlling the physical stack from silicon to rack.”
The Economics of Moore’s Law Have Broken Down
Mr. Bubble’s core argument is that shrinking transistors no longer pays for itself. Advancing to each new process node now costs tens of billions of dollars, yet yields only 15–20% density gains rather than the historic doubling. As a result, he says, advanced packaging has become “the new Moore’s Law” — the industry is expanding area and stitching multiple dies together to behave as one chip rather than relentlessly shrinking transistors per unit area.
That shift has real supply-chain consequences. He points to Nvidia’s roadmap, which he says is constrained less by Nvidia’s own design ambitions than by what memory and packaging suppliers can deliver. The originally planned four-die package for Rubin Ultra was reportedly downgraded to two dies per package, with two packages linked via PCB. In that world, PCB manufacturers — not GPU designers — become the new bottleneck.
The Unit of Compute Is Now the Rack
As model parameter counts have surged, the focus has moved from scale-out to scale-up. Mr. Bubble argues you can no longer design a chip in a vacuum: the design target is the entire rack, or multiple racks, for both training and inference. Interconnect throughput and latency now determine whether a cluster lives or dies.
He contrasts two architectures: Nvidia’s NVL72 system, which uses large NVSwitch networks that consume rack space otherwise available for GPUs, and Google’s TPU team, whose scale-up domain reaches roughly 9,600 chips with routing logic built directly into the silicon. That system-level design capability, he argues, will be the core moat for future AI hardware vendors.
Memory: Flash, Not Just HBM
On inference, Mr. Bubble challenges the consensus that ever more expensive HBM is the only answer. Model weights belong in HBM, he says, but if all context is not used simultaneously, there is little reason to store massive context there. He expects “flash offload” to grow, with cheaper, higher-capacity flash — including high-bandwidth flash (HBF) — handling context, provided the flash controller is designed for AI workloads.
His long-term bet is 3D DRAM: bonding DRAM directly above logic to cut latency, raise bandwidth and lower power through sheer physical proximity.
Investor Takeaways
- Own the picks and shovels. Mr. Bubble favors hardware-layer companies with decade-long moats over frontier AI labs he says will compete to the death as talent churn erodes alpha.
- Watch packaging and PCB names. As die counts rise and packages multiply, advanced packaging and PCB makers become structural chokepoints.
- Rethink the memory trade. HBM demand may be less inelastic than assumed; flash controllers, 3D DRAM and CXL-style memory expansion are underappreciated opportunities.
- Capex payback is plausible. He argues a handful of AI-discovered drugs could justify the entire data-center buildout — a rapid takeoff scenario, not a bubble.
The message for markets: the AI trade is migrating down the stack, from chip designers to the unglamorous physical infrastructure that actually moves bits.
