The Pulse
OpenMobile-2 Narrows the Gap With Closed Smartphone Agents
OpenMobile-2 releases mobile-agent training data, environments and models, reporting that its 27B model raised its SPA-Bench score from 31.9 to 59.6 after training. The paper also reports that Gemini 3.1 Pro Preview generated instructions a
AI.info Team · Written with AI · fact-checked ·
OpenMobile-2’s authors are making a smartphone agent and its training stack available for inspection, but the examples that teach the system to use apps came from Google’s proprietary Gemini 3.1 Pro Preview. The tension sits inside the project’s claim to openness: researchers can access the resulting models, data and simulated environments, while one of the key sources of their training experience remains closed.
The preprint, submitted to arXiv on October 9, reports strong results on several mobile-agent tests, including a near-doubling of the 27-billion-parameter model’s SPA-Bench score on real Android devices. Yet the paper’s own comparison table still gives Gemini a higher SPA-Bench result. OpenMobile-2 offers a new open training route, not evidence that open agents have overtaken proprietary systems across the board.
More than 110 apps, with repeatable workflows
The team built MobileGym++ around 35 simulated commercial-style apps, then combined them with apps configured in Android emulators for a playground spanning more than 110 apps. The simulated clients preserve business details such as orders and itineraries, support workflows that cross between apps and can reset to a known state. That control lets researchers replay tasks and check whether an agent actually changed the relevant app data.
OpenMobile-2 also introduces app-native tools alongside familiar screen taps and swipes. More than 300 tools operate across 50-plus apps, giving an agent the option to call a function—such as an app action—rather than reach it through menus. The researchers’ MobileGym++ Bench contains 215 tasks, each designed for comparison between GUI-only and hybrid tool-and-GUI use.
The authors report nearly 12,000 demonstration trajectories for supervised fine-tuning and more than 2,000 executable reinforcement-learning tasks with automatic rewards. Their project page lists the environments, data, models and training recipes as released resources. The collection is meant to make a large, varied practice ground available beyond teams that can collect extensive interaction data from real commercial apps.
Real-device gains, but Gemini stays ahead
After supervised fine-tuning and reinforcement learning, the 27B model scored 59.6 on SPA-Bench’s real-device evaluation, up from 31.9 for its base model. The paper reports Gemini 3.1 Pro Preview at 70.2 on the same benchmark, so the open model’s substantial gain does not erase the remaining gap. On AndroidWorld, OpenMobile-2’s 27B model scored 79.9 after reinforcement learning; its supervised fine-tuning result was 80.8, a reminder that improvements varied by test and training stage.
The team also tested whether app variety mattered when the amount of training data stayed fixed. Across 2,000 emulator trajectories, expanding training coverage from 15 to 53 apps raised success on unseen applications from 60.7 to 67.5, while performance on familiar apps stayed broadly stable. The result supports the authors’ case for broader training environments, though it comes from their own controlled experiment rather than a claim that more app coverage will always help.
The open stack still has a closed teacher
Gemini 3.1 Pro Preview generated the user instructions and acted as the teacher agent, carrying out tasks whose successful runs became demonstrations. The researchers filtered trajectories to keep completed tasks that passed the relevant verifier, then released the resulting data. That process opens the output and much of the infrastructure, but it does not make the training pipeline independent of a proprietary model.
The distinction matters for anyone trying to reproduce the work or build on it: access to trajectories and recipes can aid research, while the authors’ reported results were produced using a closed teacher model. The paper also finds that reinforcement learning’s gains depend on how closely evaluation tasks resemble the training environment. Its conclusion states:
“We open-source environments, data, benchmarks, and trained models to support further open research on mobile agents.”
Chenyang Yan and co-authors, authors of the paper
OpenMobile-2’s clearest benchmark result is therefore both a step forward and a limit: its 27B agent reaches 59.6 on SPA-Bench, while the paper’s Gemini score is 70.2.