Apple Publishes Three AI Research Papers for Spatial Computing and Vision Pro

Apple has published three new AI research papers focused on spatial computing and the Vision Pro. The papers cover a benchmark for multimodal LLMs, sign language recognition, and 3D head modeling.

Benchmark tests multimodal LLMs

The first paper introduces SFI-Bench, a benchmark that tests multimodal LLMs on spatial arrangement and object functionality. It includes 134 indoor video scans and 1,555 expert-annotated questions. According to Apple's Machine Learning Blog, Google Gemini 3.1 Pro ranked first overall on SFI-Bench, followed by OpenAI GPT-5.4-High in second place and Gemini 3.1-Flashlight in third.

The second paper addresses sign language recognition, using AI to auto-generate annotations and reduce manual labor. The dataset includes over 300 hours of ASL STEM data and 7.5 hours of FLEURS-ASL data. A fingerspelling model achieved a 6.7% character error rate on FSBoard and 74% top-1 accuracy on the ASL Citizen dataset. The third paper, called HeadsUp, reconstructs high-quality 3D Gaussian head models from multi-view captures, using a dataset of over 10,000 subjects.

Apple's Greg Joswiak reaffirmed the company's commitment to spatial computing, stating that while he cannot predict when it will become mainstream, he is confident the trend is irreversible. The research papers underscore Apple's ongoing investment in AI and spatial computing technologies for the Vision Pro.