Apple CoreAI Engine Delivers Mixed Results in On-Device LLM Benchmarks

Apple introduced CoreAI at WWDC 2026 as its new on-device language model engine. Independent benchmarks reveal that this framework delivers mixed results depending on hardware and workload size.

Independent benchmarks reveal mixed results depending on hardware and workload size.

CoreAI runs alongside Apple's existing MLX and CoreML engines on M4 Mac computers and the iPhone 17 Pro. The software targets local large language model tasks without requiring cloud connectivity.

• Qwen3-0.6B Decode speed (tok/s): Core AI (GPU, pipelined): 180 warm 🥇 | MLX (GPU): 115 | Core AI (ANE): 50 | CoreML-LLM (ANE): 39
• Qwen3 8b Decoding Performance: CoreAI is only 1.05x faster than MLX, offering near-parity decoding performance on M4 Max Mac
• Google LiteRT-LM Engine Performance on iPhone 17 Pro: Fastest engine at 55.4 tokens per second; used 4.5× less RAM than Apple's MLX framework (641 MB vs 2,900MB)
• Apple Foundation Models Energy Efficiency: 2× more energy-efficient per token than GPU-backed runtimes, 4× more than CoreML/ANE

Tests show significant speed advantages for tiny models. Decoding Qwen3-0.6B runs at 180 tokens per second using pipelined GPU acceleration, compared to 115 tokens per second on MLX. This creates a 2.47x performance lead over the older framework.

Performance narrows considerably as model sizes grow larger. Apple Foundation Models remain two times more energy-efficient than GPU-backed runtimes and four times more efficient than CoreML with the Neural Engine. Google's LiteRT-LM engine also outperforms both on iPhone 17 Pro, reaching 55.4 tokens per second while using only 641 MB of RAM compared to MLX's 2,900 MB.

GPU throttling affects sustained workloads differently than the Apple Neural Engine combination. Benchmarks indicate that CoreAI barely edges out MLX at realistic eight-billion parameter sizes on M4 Max Mac hardware.

An independent tester benchmarked these engines on iPhone 17 Pro and noted surprising results in decoding speed. The data confirms that performance varies significantly based on model size, platform architecture, and workload type.

Apple's new CoreAI engine delivers substantial speed gains for small models but offers near-parity with MLX at larger sizes. The framework prioritizes energy efficiency over raw throughput compared to GPU-backed alternatives.