Huawei Ascend AI Chip Memory Usage Drops 6x With Tsinghua Optimization

Huawei has introduced a new optimization system for its Ascend AI chips that drastically reduces memory usage. The technology, developed in collaboration with Tsinghua University and PwC Intelligence, achieves a sixfold reduction in memory consumption. This advancement targets the growing demand for efficient large language model training on domestic hardware.

New system cuts memory usage by six times using ternary quantization.

The core of this system is 1.58-bit ternary quantization, which compresses model weights into three distinct states: negative one, zero, and positive one. This compression method simplifies arithmetic operations by replacing high-energy floating-point multiplications with basic addition and subtraction. The result is lower power consumption alongside the significant memory savings.

Performance retention remains strong across various model sizes despite the heavy quantization. A 1 billion parameter version retains 97.1 percent of full precision performance, while a 3 billion parameter version keeps 97.2 percent. An 8 billion parameter model maintains 95.7 percent accuracy, and even the smallest 0.5 billion parameter variant preserves 90.1 percent of its original capabilities.

Throughput on the Ascend 910B chip experiences only a minimal drop with this optimization. Full precision operations deliver 155 TFLOP/S, while the ternary quantized version sustains 148 TFLOP/S. The system is open-source and fully reproducible, allowing users to deploy models of any size without restrictions.

The broader impact on high-bandwidth memory market prices and production capacity remains uncertain. Current speculation suggests that reduced memory demands could alleviate pressure on existing supply chains, but concrete data on these market effects has not been provided by the developers.