NVIDIA NVHBM Boosts XPU Performance 30% With Custom Memory Design

NVIDIA has introduced NVHBM, a custom high-bandwidth memory technology designed to reshape how AI accelerators and future GPUs handle data. This shift matters because it moves the memory controller into the HBM base die, which reduces the silicon footprint required on the main processor. Buyers and developers gain more usable silicon for compute hardware, which directly impacts the density and efficiency of next-generation AI chips.

Custom memory design shifts controller to base die

The technology targets partners developing AI accelerators using the NVLink Fusion architecture, with Amazon's Annapurna Labs confirmed as the first collaborator. Annapurna Labs is currently integrating NVHBM into its Trainium4 accelerator to leverage these new memory capabilities. NVIDIA also stated that this custom memory design will form the foundation for the memory architecture in its upcoming GPU generations, signaling a broader industry move away from standardized HBM modules.

NVHBM Specifications

• Bandwidth: Up to 30% higher than standard HBM4E
• Power Consumption: Up to 15% lower than standard HBM4E
• Compute Die Area: Up to 25% more available area
• PHY Area Reduction: Up to 67% smaller support area
• Overall XPU Performance: Up to 30% increase

NVHBM delivers up to 30% higher memory bandwidth per stack compared to standard HBM4E while reducing power consumption by up to 15%. The redesign shrinks the PHY and support area by up to 67%, freeing up significant space within the package. This efficiency allows for up to 25% more compute die area and provides up to 80% more usable silicon across the complete layout.

When combined with the NVLink Fusion architecture, NVIDIA claims these improvements can increase overall XPU performance by up to 30%. The company attributes this gain to the combination of extra bandwidth, additional die area, and lower power demands. Industry analyst Ian Cutress notes that this represents a strategic shift toward custom memory designs rather than relying on off-the-shelf HBM standards.

NVIDIA has not yet announced which specific future GPU generation will be the first to utilize NVHBM. The technology is currently positioned as a foundation for AI accelerators and future graphics hardware. This announcement marks a clear departure from previous standardized memory approaches in favor of tailored solutions for high-performance computing.