The Pulse
Apple Researchers Cut Multimodal Agents’ Reasoning Latency by 93%
Apple researchers report that a method for selecting prepared reasoning steps reduced per-turn reasoning latency by more than 90% in tests of multimodal search agents. The paper reports results from research benchmarks, not a feature announ
AI.info Team · Written with AI ·
Apple researchers report cutting multimodal agents’ per-turn reasoning latency by more than 90% by having models choose a reasoning step instead of generating one word by word. In their experiments, a 4-billion-parameter model using the approach took 0.061 seconds to reason per turn, down from 0.896 seconds for its freeform-reasoning counterpart. Total model inference time per question fell by 28% to 54%, depending on the training method.
The work, called Selection-Based Structured Reasoning (SSR), was submitted to arXiv on October 1 and revised October 8. Apple’s research page lists it as an October 2026 publication. The results come from research tests, not an announcement of a feature in Apple products.
Six choices replace open-ended reasoning
SSR gives an agent a fixed set of six short, manually written reasoning options. They cover identifying an image subject with image search, looking up a fact, inspecting an image detail, answering from gathered evidence, answering from the image, or relying on general knowledge. At each turn, the model scores the options against the conversation so far and selects one; it still generates the specific search query, image crop, or answer that follows.
That change targets a serial step in ordinary agent workflows: the model must generate each reasoning token before it can produce the next action. SSR instead scores the prepared options in parallel, using the model itself rather than a separate classification head. The selected text then guides the action-generation step.
For small, on-device models with limited reasoning capacity, lengthy texts and repeated deliberations may waste the generation cost without providing useful guidance for generating better actions.
Feiyu Gavin Zhu and co-authors, the paper’s authors, write in Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents
Qwen3-VL tests show faster turns
The researchers tested 2B- and 4B-parameter Qwen3-VL models on seven multimodal search benchmarks, using tools for reverse-image search, text search, and image cropping. Their 4B model trained with Group Relative Policy Optimization (GRPO) reached a 61.37% average success rate, compared with 58.65% for its freeform-reasoning counterpart. The paper also compares that result with 61.25% for a leading trained 4B search-agent baseline, TAPO with GSPO.
For the 4B GRPO comparison, mean model latency per question dropped from 5.456 seconds to 2.514 seconds. The paper reports that SSR reduced per-question inference latency across its tested training approaches, though the size of the reduction varied: the 4B GSPO result, for example, improved by 28.7%, while the GRPO result improved by 53.9%. The authors also report modest success-rate declines for SSR under GSPO and SAPO, rather than uniform gains in accuracy.
Benchmarks leave a deployment question
The latency measurements used an NVIDIA H100 GPU and an offline SGLang inference engine. Researchers profiled 100 questions per benchmark; their model-latency measure includes prefill, reasoning, and action generation across the trajectory, but excludes tool execution and evaluation. The paper’s timing results therefore do not establish how much a user would save in an end-to-end search workflow or on an Apple device.
A fixed library also sets a boundary on what the method can express: if a task calls for a high-level reasoning move missing from the six options, the agent cannot select it without changing the library. The researchers found that expanding the library from one generic entry to six specific candidates raised average success across the benchmarks from 37.24% to 61.37%. Whether the speed gains carry over to broader tasks, larger libraries, and real-world device workloads remains an open test.