AMD MI355X AI GPU Benchmarked: 288GB Memory Crushes Nvidia B200

AMD's MI355X AI accelerator has been benchmarked, showing it can compete with Nvidia's B200 in real-world single-node inference. The results come from Zyphra, a company that developed software optimizations for the AMD GPU.

Zyphra optimizations bridge interconnect gap

The MI355X features 288 GB of HBM memory, significantly more than the B200's 180 GB. However, its inter-chip bandwidth is 537.6 GB/s, lower than the B200's 900 GB/s via NVLink. To compensate for this bandwidth limitation, Zyphra developed algorithms called TSP and Tree Attention.

With these optimizations, the MI355X's throughput approaches that of the B200 in long-context scenarios. The software improvements help bridge the gap caused by the slower interconnect.

AMD has not confirmed pricing or availability for the MI355X. The company has not commented on the benchmark results from Zyphra.