Cactus Compute released Needle 2, a tiny AI model that lets Raspberry Pi 5 boards run local instructions without extra hardware. This capability enables the development of smart home and industrial devices that respond instantly, even without an internet connection. The 14MB model size allows it to operate effectively on memory-constrained edge devices that typically face challenges with AI tasks.
Tiny model converts natural language to commands in milliseconds
Needle 2 is not a general chatbot. It is a specialized tool that translates natural language commands into specific device functions. The model runs entirely on the Raspberry Pi 5 CPU, bypassing the need for expensive AI accelerator expansion boards.
The model weighs in at just 14MB. It uses about 28MB of RAM for the model alone, and 46.4MB when running a full Python demo on an 8GB Raspberry Pi 5. This low memory footprint ensures the system stays responsive during operation.
The system demonstrates rapid response times for local processing tasks. The system takes only 78 milliseconds to recognize a command and execute an action like turning on an LED. Querying the CPU temperature takes 149 milliseconds, while unmatched queries return an empty value in 92 milliseconds.
Cactus Compute and the Raspberry Pi Blog state that the model converts natural language instructions into preset function calls in real time. The code is open-sourced on GitHub under the Apache 2.0 license. Model weights are available on Hugging Face for global download right now.
Discussion
0 comments
Log in to join the thread with a thoughtful take, question, or correction.