Colibrì Runs 1.5-TB AI Model on 25GB RAM, Speed Hits 0.1 Tokens Per Second

Running a 1.5-terabyte AI model on a standard laptop used to require expensive enterprise GPUs, but a new proof-of-concept changes that dynamic. Colibrì allows users to load frontier-level language models directly onto consumer hardware with minimal RAM. This approach matters because it demonstrates that local AI is no longer strictly limited by expensive graphics cards, though speed remains a major hurdle.

Italian engineer demonstrates MoE architecture on consumer hardware

The tool was developed by Italian engineer Vincenzo, known online as JustVugg, to run the GLM-5.2 model. GLM-5.2 is a massive model containing 744 billion parameters and weighing in at 1.5 TB. Colibrì manages this enormous footprint by leveraging a Mixture-of-Experts (MoE) architecture. The software loads and unloads specific expert sub-models for each token generated, keeping the active memory footprint low.

The system operates on a CPU with only 25 GB of RAM and simulates a 1 GB/s virtual NVMe drive. This configuration allows the model to fit into memory that would typically be insufficient for such a large AI. However, the performance trade-off is severe. The setup generates only 0.05 to 0.1 tokens per second on average.

This speed is far below the 20 to 30 tokens per second required for real-time conversation. The primary bottlenecks are storage input/output and RAM bandwidth, which limit how quickly the system can swap expert data. Tom's Hardware noted that the tool allows cheap machines to use large models at a steep performance penalty. The current implementation serves as a proof-of-concept rather than a practical daily driver.

The project highlights the potential for running large models on modest hardware while exposing current physical limits. Future feasibility depends on overcoming these I/O and bandwidth constraints to achieve usable speeds. For now, the tool proves that the architecture works, even if the output rate is too slow for interactive use.