Google Gemma 4 AI Models Get Up to 3x Speed Boost with Multi-Token Prediction

Google has released Multi-Token Prediction (MTP) drafters for its Gemma 4 AI models. The new feature enables speculative decoding, which can significantly speed up inference. According to Google, MTP can boost performance up to 3x on Pixel smartphones and 2.5x on Apple M4 chips.

MTP drafters enable faster inference on Pixel and Apple M4

The MTP drafters are small models. For the Gemma 4 E2B variant, the drafter has 74 million parameters. On Pixel smartphones, the E2B model runs 2.8x faster, while the E4B model achieves a 3.1x speedup. The larger Gemma 4 31B model sees a 2.5x improvement on Apple's M4 chip. Google states that the technique should result in no quality degradation.

The MTP drafters are available now globally. They are released under the Apache 2.0 license, making them freely usable for developers and researchers.

The speed gains come from predicting multiple future tokens at once, rather than generating one token at a time. This approach is particularly beneficial for on-device AI, where hardware resources are limited. Google's claims highlight the potential for faster responses in applications like chatbots and real-time translation.

Ars Technica reported the release. Google has not disclosed any further details about future updates or additional model support.