Moonshot AI Unveils Kimi K2.5 Trillion-Parameter Large Language Model

Moonshot AI has released the Kimi K2.5 large language model, which contains one trillion parameters. A Reddit user recently demonstrated that this massive model runs on a single-GPU workstation using hybrid inference methods.

Reddit user demonstrates hybrid inference on single-GPU workstation with 768GB memory.

The system relies on 768GB of total memory capacity to handle the model weights. This configuration combines six 32GB Samsung DDR4-2666 ECC modules with six 128GB Intel Optane DCPMM persistent memory units. The hardware also features an Intel Xeon Gold 6246 processor and two ASUS GeForce RTX 3060 OC 12GB graphics cards.

The workstation achieves approximately four tokens per second during inference. It uses llama.cpp with a hybrid GPU and CPU routing approach enabled by the override-tensor flag. Additional components include a Taiwan S5630GMRE-CGN motherboard, a WD SN850X 2TB SSD, and an ASRock Steel Legend SL-850G 850W power supply.

Industry analysis suggests that Intel Optane modules serve as a cost-effective bridge between fast DRAM and slower storage for large language models. Market observers expect CXL standards to eventually enable more affordable byte-addressable memory solutions for AI workloads.

Reddit user APFrisco shared the technical details of this setup, but Moonshot AI has not confirmed the specific hardware requirements or official support status for running Kimi K2.5 on consumer-grade components.