NVIDIA DeepSeek V4: 1.6T Parameter Model Hits 3,500 TPS on Blackwell

NVIDIA has introduced DeepSeek V4, a new large language model architecture. NVIDIA DeepSeek V4 The company claims the model achieves high throughput on its latest hardware. NVIDIA states the system pushes 3,500 tokens per second on 1.6T parameter models. This performance target relies on GB300 or Blackwell Ultra GPUs. The company notes this processing speed is preliminary. NVIDIA expects the throughput to increase with further optimization.

The architecture uses a mixture of experts design to manage its scale. DeepSeek V4 contains 1.6T total parameters. It activates only 49B parameters during inference. A flash variant exists with 284B total parameters. This flash version activates just 13B parameters. Both versions support a context length of 1 million tokens. The maximum output length reaches up to 384K tokens according to the DeepSeek API documentation.

NVIDIA reports significant efficiency gains for this model. The model utilizes only 27% of the single-token inference FLOPs and 10% of the KV cache. It also requires only 10% of the KV cache memory. These metrics apply to the processing of 3,500 tokens per GPU. The company demonstrates this capability on GB300 or Blackwell Ultra hardware. This efficiency is achieved through a sparse architecture that reduces computational requirements.

New architecture delivers high throughput with sparse design

The model demonstrates high throughput capabilities, pushing 3,500 tokens per second on 1.6T models. NVIDIA positions this architecture to handle heavy workloads on its current GPU lineup. The preliminary 3,500 TPS figure serves as a baseline for future improvements. The model supports extensive context windows for complex tasks. The flash variant offers a lighter footprint for specific use cases. The system demonstrates nearly 3,500 TPS processing per GPU, such as the GB300 or Blackwell Ultra.

Source: wccftech.com