NVIDIA RTX 5090 Gets 90% AI Boost With New llama.cpp Update

NVIDIA has released software optimizations that make running large local AI models significantly faster on its consumer and professional GPUs. This update matters to builders and developers because it unlocks higher throughput for generative AI tasks without requiring new hardware. Users with compatible graphics cards can now process text and code more efficiently using standard open-source tools.

New kernels accelerate local model inference on RTX and DGX platforms

The update targets the GeForce RTX 5090, the RTX PRO 6000 Blackwell, and the DGX Spark platforms. NVIDIA integrated new XQA attention kernels into FlashInfer and optimized backend processes for vLLM and llama.cpp. These changes streamline the local AI experience by reducing the technical friction usually associated with model deployment.

Performance gains vary by model and hardware configuration. The RTX 5090 delivers up to a 50% boost in token throughput for the Qwen3.6-27B model and a 90% acceleration for the Qwen3.6-35B variant using llama.cpp. The RTX PRO 6000 Blackwell sees up to a 20% speedup in vLLM across both models. DGX Spark platforms achieve a 1.4x increase in Qwen3.6-27B vLLM performance and a 20% increase in DeepSeek v4 Flash inferencing.

NVIDIA is simplifying the setup process for popular AI agents. One-click local model setup is coming soon for the Hermes Agent, OpenClaw, and Perplexity Computer. These features will be available for NVIDIA GPUs featuring 24 GB of memory or higher. Perplexity Portable Computer is currently available for RTX GPUs with 24GB+ VRAM on Linux and Windows.

The optimizations bring simplified local AI capabilities to NVIDIA's RTX and DGX platforms. vLLM and llama.cpp updates boost compute performance by up to 1.9x on specific configurations. The company confirmed that these enhancements are designed to make local AI more accessible to a broader range of users.