CUA-Master: Scaling Computer-use Agents through Open-World Recursive Improvements

AI summary

Overview

Research area: Computer-use agents (CUAs) — models that operate real software through graphical user interfaces (GUIs) and command-line interfaces (CLIs) — with a focus on synthetic data construction, recursive self-improvement, and post-training (supervised fine-tuning plus reinforcement learning).

Technical level: Advanced. The paper assumes familiarity with agent training pipelines, SFT/RL recipes, VM-based rollout infrastructure, and benchmark evaluation for GUI agents.

Scope: The paper proposes CUA-Master, a nested double-loop recursive improvement framework that couples open-world data discovery with composable task synthesis, and reports a 27B-parameter agent evaluated on GUI-only, GUI+CLI hybrid, and cross-platform benchmarks.

What This Paper Is About

Building capable computer-use agents is bottlenecked by data: high-quality human-annotated interaction trajectories are expensive and narrow, while existing synthetic data is homogeneous, hard to control, and unreliable to roll out in real environments. Existing "recursive improvement" methods also tend to be closed-world, recycling the model's own outputs and therefore reinforcing its existing biases with diminishing returns. The paper's goal is a system that keeps pulling in new knowledge from the open world, turns real workflows into controllable and verifiable synthetic tasks, and feeds that back into model training in a repeating loop.

Key Contributions

1. A nested double-loop recursive improvement framework. An outer "world-data-model" flywheel closes the loop between open-world exploration, data construction, and model optimization, using the evolving model's measured weaknesses (domain- and application-level failure cases, rollout success rates) to direct the next round of data acquisition. An inner data flywheel couples open-world seeding with autocatalytic propagation.

2. Open-world seeding from instructional video. The authors build a hierarchical taxonomy of 8 main categories, 36 subdomains, and over 650 applications, retrieve demonstration videos from YouTube, TikTok, and Bilibili via compositional queries, filter them with metadata gates, a noise filter, and a Whisper-Large-v3 transcript-based suitability score from 0 to 5, then segment each video into multiple timestamped seed tasks. A visual-semantic dual-path mechanism (semantic triggers from operation-bearing transcript clauses, visual triggers from frame-level appearance-change scores, matched within a temporal window T) reconstructs trajectories, followed by annotator-based action rechecking and verification.

3. Autocatalytic propagation via composable primitives. Each seed task and trajectory is abstracted into reusable five-tuple primitives (application, ability, preconditions, environment requirements, verifier specification) stored in an application-indexed primitive library. Tasks are then synthesized by constrained composition, with difficulty controlled explicitly by K (number of primitives) and N (number of applications), rather than by instruction rewriting.

4. A feedback-guided snapshot-rollback retry mechanism for trajectory collection: a detector estimates the earliest state-divergence step and produces execution-grounded feedback without revealing a privileged action solution; the environment is restored to the latest valid incremental snapshot before that step, the valid prefix is preserved, and only the successful and most efficient branch is kept as the final trajectory.

5. A 27B agent and training recipe. Initialized from Qwen-3.5-27B, trained with SFT over three iterative data expansions (63K, 38K, and 14K tasks, totaling about 115K at the last SFT iteration, each iteration starting from the same base checkpoint) followed by RL on about 12K high-quality tasks using a modified GRPO-style framework from ScaleCUA.

Main Findings

• Headline benchmark scores: CUA-Master-27B scores 80.2 on OSWorld Verified, 36.9 on OSWorld 2.0, 43.9 on WeaveBench, and 25.0 on Agents' Last Exam (the last reported as full-pass rate; the other three as average scores).

• Gains over a similarly sized specialist: Against UI-Mate-27B it improves by 2.6, 12.9, 25.6, and 11.8 points on the four benchmarks respectively.

• Competitive against much larger models: On GUI-only benchmarks it is competitive with Qwen-CUA-397B-A17B and outperforms it on the hybrid benchmarks (43.9 vs. 35.1 on WeaveBench; 25.0 vs. 21.1 on Agents' Last Exam). On Agents' Last Exam it trails only GPT-5.6-Sol (26.3) and Claude-Opus-5 (28.3), surpasses all other listed counterparts when full-pass rate ties are broken by average score, and matches Kimi-K3 and Qwen-3.8-Max at 25.0.

• Efficiency: On OSWorld Verified it reaches 80.2 in 8 minutes with 19.0 steps and 247K tokens per task. On OSWorld 2.0 it slightly beats Qwen-3.8-27B (36.9 vs. 36.5) with 24% less runtime. On WeaveBench it matches GPT-5.6-Luna (44.0) in 34 minutes with 28% less runtime, and on Agents' Last Exam it matches Kimi-K3 and Qwen-3.8-Max with 34–36% less time.

• Hybrid action space matters most on complex tasks: On WeaveBench, hybrid execution scores 43.9 versus 6.6 for GUI-only and 11.5 for CLI-only (gains of 37.3 and 32.4 points), with 23.8% fewer steps than GUI-only. On Agents' Last Exam, hybrid lifts the score from 5.9 (GUI-only) to 25.0 with 26.7% fewer steps; on OSWorld 2.0 it reduces steps by 43.8%.

• Cross-platform generalization: Scores of 56.2 on WindowsAgentArena, 43.0 on MacOSWorld, and 59.9 on MacAgentBench. UI-Mate-27B scores higher on WindowsAgentArena (63.2), while CUA-Master leads on MacOSWorld (43.0 vs. 37.1) and trails Qwen-CUA-397B-A17B on MacAgentBench (65.3).

• Rollout reliability under the retry mechanism: Success rates beat a baseline that re-rolls out from scratch with the same retry budget at every difficulty level — for example 95.4% vs. 74.0% at K/N = 5/1, 49.8% vs. 14.6% at 40/3, and 18.7% vs. 0.0% at the hardest tested setting, 50/8.

• Broader task coverage: A t-SNE visualization indicates that flywheel-generated tasks span multiple existing CUA datasets (AgentNet, VideoCUA, ScaleCUA, WildGUI, CUA-Gym, ProCUA-SFT) and extend into sparsely covered embedding regions.

Methodology in Plain English

The system is built as two nested loops that keep feeding each other.

The inner loop starts with the open internet. The authors catalogue software applications into a taxonomy and search video platforms for tutorials about each one, using operation-oriented keywords. Videos are deduplicated and filtered for duration, recency, and relevance; a transcription model turns the audio into timestamped text, and a strong annotator model scores each video for whether it actually shows a legible computer-use interaction that could be reconstructed. Each acceptable video is then split into separate task episodes with start and end timestamps, and its trajectory is rebuilt from two signals at once: spoken instructions that name actions ("click", "drag", "then", "next") and moments where the screen visibly changes. Where the two agree within a time window, an action is placed; a rechecking annotator then adds, removes, splits, or merges candidate actions, converts them into concrete CUA actions with coordinates and typed content, and a final verifier removes duplicates and discards broken sequences.

Those seed tasks are then broken down into reusable building blocks — a primitive records which application it uses, what ability it exercises, what must be true before it runs, what environment it needs, and how to check success. Blocks are stored in a library and combined into new tasks under explicit difficulty knobs: how many primitives (K) and how many applications (N). Incompatible blocks (conflicting preconditions or environments) are not allowed to be composed. Because each task is assembled from parts that already carry verification logic, every synthesized task arrives with an automatic checker.

Collecting real trajectories is where most pipelines break. Instead of throwing away a failed run, the system asks a detector model to look at the checker's complaint, the recorded trajectory, and the environment to guess the earliest point where things went wrong, then rolls the virtual machine back to a snapshot just before that point. The valid prefix is kept, the model is told what failed in execution-grounded terms (without being handed the answer), and it continues from there. Environments are assembled from a VM image catalog spanning Linux, Windows, and macOS, with the infrastructure supporting over 100 CUA environments and over 2,000 concurrently running VMs.

Finally, training happens inside the outer loop: early rounds use supervised fine-tuning on the verified trajectories accumulated so far, always restarting from the same base checkpoint so the data grows cumulatively without drifting; analysis of where the model still fails then steers the next round of data collection. The last round switches to reinforcement learning on about 12K curated synthetic tasks, using the task verifiers as automatic outcome rewards. Training ran on 16 nodes, each with 8 NVIDIA H200-141GB GPUs.

Why This Matters

Impact on research: The paper reframes recursive self-improvement for agents as an open-world process rather than a closed one. Rather than sampling more data from the model's own distribution, it injects external knowledge (human tutorial videos) and lets measured capability gaps steer what gets collected next. The primitive-composition view also turns task difficulty into an explicit, controllable variable (K and N) rather than something inferred after the fact from instruction text, which makes it possible to build difficulty-stratified training and evaluation sets deliberately.

Real-world applications:

• Desktop and OS automation: Delegating multi-step, cross-application workflows such as file processing, data entry, and report assembly to an agent that can choose between clicking through a UI and running a shell command.
• Software QA and regression testing: Synthesizing diverse, verifiable interaction tasks across many applications and OS images to exercise software before release.
• Back-office and enterprise tooling: Automating workflows that span several applications (for example, moving data between a spreadsheet, a browser, and a document tool) where both a GUI and a CLI are available.
• Training-data generation as a service: The seeding and propagation pipeline is itself a way to manufacture verified agent trajectories at scale, which is useful to anyone building GUI agents who lacks annotation budget.

Industry relevance: A 27B model scoring 80.2 on OSWorld Verified and 43.9 on WeaveBench, while using 8 minutes and 247K tokens per task on OSWorld Verified, is a meaningful cost/latency point for deployment. The paper's efficiency comparisons — 24% less runtime than Qwen-3.8-27B on OSWorld 2.0, 28% less than GPT-5.6-Luna on WeaveBench, 34–36% less than Kimi-K3 and Qwen-3.8-Max on Agents' Last Exam — speak directly to the operating economics of running agents at scale.

Future Directions

• Pushing past the hardest composition settings. The retry mechanism reaches 18.7% success at K/N = 50/8 versus 0.0% for the from-scratch baseline. The paper does not report behavior beyond that configuration, leaving open how far the snapshot-rollback approach scales as primitives and applications increase.

• Extending the open-world corpus. Seeding currently draws on YouTube, TikTok, and Bilibili with English-language transcription via Whisper-Large-v3 and operation-oriented keywords. Whether other video sources, languages, or non-video documentation would add coverage is not addressed in the reported content.

• Running more recursive iterations. The paper reports gains across iterations but stops at three SFT iterations plus a final RL round. How long the outer flywheel keeps yielding returns — and when saturation sets in — is not reported.

• Continuing the RL transition. RL is used only in the final iteration, on roughly 12K tasks. Whether RL applied across multiple iterations, or with denser reward shaping beyond outcome-based verifier rewards, would raise the ceiling is an open question.

• Cross-platform parity. CUA-Master leads UI-Mate-27B on MacOSWorld but trails it on WindowsAgentArena (56.2 vs. 63.2) and trails Qwen-CUA-397B-A17B on MacAgentBench (59.9 vs. 65.3), indicating room to improve platform-specific generalization.

Target Audience

Researchers and engineers working on GUI/computer-use agents, agentic post-training (SFT and RL for long-horizon tasks), and synthetic data generation for embodied or interactive models. It is also relevant to infrastructure teams building VM-based rollout and evaluation harnesses, and to practitioners deciding whether a mid-sized open model can replace a much larger or closed-source model for desktop automation. Readers without background in agent training will find the high-level framing accessible, but the methodology and results sections assume familiarity with benchmarks such as OSWorld, WeaveBench, and Agents' Last Exam.

Authors’ abstract

Despite recent advances in CUAs, their development remains constrained by the limited diversity, controllability, and reliability of interaction data. Moreover, conventional optimization pipelines lack iterative feedback, while existing recursive improvement approaches often rely on self-generated experiences, limiting ability expansion and yielding diminishing returns. To address these challenges, we introduce CUA-Master, a nested double-loop recursive improvement framework. At the system level, an outer world-data-model flywheel continuously incorporates open-world corpus and leverages evolving model weaknesses to guide subsequent data construction and optimization iterations. At the data level, an inner flywheel couples open-world seeding and autocatalytic propagation. The former mines authentic workflows as seed tasks from instructional videos, and employs visual-semantic dual-path trajectory reconstruction; The latter transforms these seeds into reusable primitives for controllable synthesis of diverse, complex, and verifiable tasks. To enable efficient and reliable trajectory collection, we further introduce a feedback-guided snapshot-rollback retry mechanism in various real environments. Extensive experiments demonstrate that CUA-Master-27B achieves competitive performance across GUI-only and GUI-CLI hybrid benchmarks, scoring 80.2 on OSWorld Verified, 36.9 on OSWorld 2.0, 43.9 on WeaveBench, and 25.0 on Agents' Last Exam. Our framework also substantially improves the quality and efficiency of data synthesis, and exhibits continuous gains across recursive iterations.