Nvidia GB300 Sets New MoE Record: 1648 TFLOPs Per GPU

Nvidia’s Blackwell GB300 has set a new world record for Mixture of Experts (MoE) pre-training, signaling a major leap in large language model efficiency. This milestone illustrates how hardware scaling can significantly reduce the time and cost required to train large AI models. Developers and buyers monitoring compute capabilities now have a concrete benchmark for next-generation infrastructure performance.

Blackwell chip achieves 50% throughput increase over prior benchmark

The record-breaking run utilized 256 Blackwell GB300 GPUs to train the DeepSeek-v3 671B model. Nvidia achieved this by optimizing the system to deliver 1648 TFLOPs of throughput per GPU. This performance metric marks a notable increase over previous industry standards and establishes a new baseline for high-performance computing clusters.

• GPU Count: 256
• Model: DeepSeek-v3 671B
• Throughput: 1648 TFLOPs per GPU
• Previous Throughput (Nov): 1088 TFLOPs
• Previous Gen Throughput (GB200): 606 TFLOPs

Compared to the previous record set in November, which stood at 1088 TFLOPs, the GB300 delivers a 50% performance improvement. The jump is even more pronounced when measuring against the previous generation GB200 architecture, which delivered 606 TFLOPs. This comparison highlights an approximate threefold performance increase over the prior generation hardware.

Nvidia reached these optimization levels by simulating over 250,000 configurations and conducting 1.4 million hours of GPU testing. We've been tracking Blackwell GB300 closely — see our earlier coverage on NVIDIA Blackwell GB300 Systems Dominate MLPerf. The company confirmed that the system successfully processed the complex 671B parameter model within this optimized framework.

The Blackwell GB300 now holds the official MoE pre-training world record with verified throughput metrics. The system demonstrates that architectural improvements can yield substantial gains in training efficiency for large-scale AI workloads.