NorthCUA Technical Report: Turbocharging Computer-Use Agents through Experience at Scale

AI summary

Overview

Research area: Machine learning for computer-use agents (CUAs) — models that operate desktop software and web browsers through graphical user interfaces.

Technical level: Advanced. The report assumes familiarity with supervised fine-tuning, reinforcement learning (specifically GRPO), vision-language models, and agent scaffolding.

Scope (one sentence): The paper describes NorthCUA, an integrated system that pairs a task-generating "Experience Flywheel" with a long-horizon execution harness and a three-stage training recipe, producing a family of open models (Nano 9B, Mini 35B, Pro 397B) that the authors report as state-of-the-art among open-source models on GUI grounding, desktop execution, and web interaction.

What This Paper Is About

Computer-use agents can click, type, and navigate real software, but scaling them from single actions to complete multi-step workflows is hard: long tasks accumulate context, earlier steps change the state later steps depend on, and building enough varied, verifiable tasks is expensive. The paper's goal is to make software interaction itself a continuous source of training experience, so that a model can keep improving from its own attempts rather than only from static human demonstrations. It does this by co-designing three things that are usually built separately: how tasks are constructed, how agents execute and are judged, and how the resulting records become training data.

Key Contributions

1. The NorthCUA Harness — full-lifecycle infrastructure that runs desktop and web tasks in isolated environments, sustaining long interactions with bounded visual context, task-preserving context compaction, and an agent-writable "note" memory, then preserving each attempt's trajectory and workspace for inspection before release. A Batch Rollout Service coordinates parallel workers and overlaps lifecycle stages for high-concurrency collection.

2. The Experience Flywheel — a task-construction and curation pipeline operating at two levels: single-step grounding examples (screenshot–query–target pairs) and full agentic trajectories. Trajectories come from three routes (Scaling Intents, Scaling Environments, Scaling Verifiers), with a hard-task branch using recursive self-improvement via skill-guided exploration and milestone-guided search that preserves verified intermediate software states so later attempts build on earlier progress.

3. Staged Learning — a three-stage recipe: broad GUI mid-training (grounding data plus trajectories judged to complete their tasks, spanning over four million GUI interaction steps across dozens of applications), verified long-horizon supervised fine-tuning on workflows that pass executable verifiers, and joint desktop–web agentic reinforcement learning with GRPO over fresh attempts in the same harness.

4. An open model family plus a latency-optimized variant — Nano (9B), Mini (35B), and Pro (397B) built on Qwen3.5, all released along with the harness code; and NorthCUA-Ultrafast, which adapts pre-RL Nano with a "Jev-inspired choice head" to select finite actions directly while retaining generation for open-ended outputs.

Main Findings

• State-of-the-art among open-source models in all three domains. Across six benchmarks covering GUI grounding, desktop execution, and web interaction, the NorthCUA family is reported as best among open-source models under the paper's evaluation settings.

• Pro (397B) beats trillion-parameter models. Pro outperforms both Qwen3.8-Max and Kimi-K3 on the two grounding benchmarks, OSWorld-Verified, and WebArena-Verified.

• Mini (35B) is competitive with much larger models. The 35B Mini surpasses Kimi-K3 on OSWorld-Verified and Qwen3.8-Max on WebArena-Verified.

• Both later training stages help at every model size. Stage comparisons show desktop and web gains across all three sizes after long-horizon SFT and joint RL.

• A large latency win for the Ultrafast variant. The Head + LoRA variant of NorthCUA-Ultrafast completes matched successful WebArena-Verified tasks 2.3 times faster on average than full generation, described as a measured quality–latency tradeoff rather than a free gain.

• Exact benchmark scores are not reported in the provided content. The truncated text states rankings and comparisons but does not give the numeric per-benchmark results, so no score values can be quoted here.

Methodology in Plain English

The system is built as a loop with three moving parts.

Task construction. The Flywheel manufactures practice problems in three ways: deriving realistic requests from the software and content actually available, building or reconstructing the environments those requests need (including reference websites), and writing executable checkers so success can be judged automatically. For problems the model repeatedly fails, a "hard-task" branch saves the agent's progress at verified milestones and retries from those saved states, and reuses procedures that have already worked, so the agent explores alternatives without starting from scratch.

Execution and judgment. The harness runs each task in an isolated environment, giving the agent screenshots (plus DOM information for web) and mouse/keyboard or browser tools in a ReAct-style loop. To survive long workflows, it keeps only a bounded window of recent screenshots — older frames get overlays marking clicks, drags, and scrolls so the agent can still see what it just did — while the text history is compacted into a progress summary that deliberately drops stale coordinates and preserves goals, constraints, and pending work. The agent can also write its own notes to carry facts forward. When the run ends, an "agent-as-a-judge" evaluator reads the requirements, pulls selected recorded evidence, and can open the live workspace to check files or application state, producing a report that links every judgment to evidence.

Training. Experience flows into training in stages: broad mid-training to learn what interfaces look like and how they respond, supervised fine-tuning on complete workflows with verified outcomes, then reinforcement learning where the model generates its own attempts and GRPO compares outcomes within groups of attempts on the same task. Because the same harness and the same verifiers are used for both collection and RL, an improved model simply goes back into the harness and its new attempts become the next round of experience.

Why This Matters

Research impact. The paper argues that the bottleneck for computer-use agents is not model size but the supply and use of experience — and it shows a concrete architecture where task generation, execution, evaluation, and training share interfaces. That framing, plus the release of the harness code, gives other groups a reusable template rather than a one-off result. The finding that a 397B model can beat trillion-parameter models on grounding and execution benchmarks is also a meaningful data point about the value of data and post-training over raw scale for this capability class.

Real-world applications:

• Automating back-office workflows that live inside desktop applications — the paper's own example is compiling an invoice report in a spreadsheet and exporting a PDF that highlights overdue invoices.
• Web-based task automation such as form entry, navigation, and multi-tab work, where DOM-aware agents can act on page structure rather than pixels alone.
• Cross-application knowledge work where a value read from one file must survive into a document produced much later — the reason the paper invests in working memory and compaction.
• Latency-sensitive deployment, where the Ultrafast variant's 2.3 times speedup on matched successful WebArena-Verified tasks matters more than raw quality.

Industry relevance. The 9B / 35B / 397B ladder spans edge-deployable to frontier scale, and the release of models, harness code, and the report lowers the barrier to building on the stack. The harness's tool abstraction — local functions plus Model Context Protocol tools — is designed so new platforms (mobile is named as a natural extension) can be added without rebuilding the agent loop, which matters for organizations that need to support heterogeneous environments.

Future Directions

• Extending to new platforms. The report explicitly notes that mobile support would require only screen observations, touch operations, and application/session handling around the shared runtime; whether the same training recipe transfers is untested here.

• Better control of the quality–latency tradeoff. Ultrafast is presented as a measured tradeoff rather than a strict improvement. How far the choice-head approach can be pushed before it starts costing task success — and whether it can be trained post-RL instead of on pre-RL Nano — remains open.

• Scaling the self-improvement loop further. Recursive self-improvement currently targets tasks that stay unsolved in ordinary collection, using skills and verified milestones. How many rounds of return-to-harness improvement remain productive, and whether the loop can run without human curation at each turn, is not answered.

• Calibrating automated judgment. The evaluator is an LLM agent inspecting evidence, and the paper tracks "judgment uncertainty" as an explicit dimension, routing unresolved cases for further checking. Reducing that uncertainty — and understanding how often the judge disagrees with executable verifiers — is a clear next problem, especially for requirements that lack dedicated checks.

Target Audience

Machine learning researchers and engineers working on agents, tool use, and post-training at scale; teams building computer-use or browser-automation products who need long-horizon reliability rather than single-step accuracy; infrastructure engineers interested in the practical engineering of isolated, high-concurrency rollout environments; and data-curation specialists focused on synthetic task generation and automated evaluation. Readers without background in reinforcement learning or vision-language model training will find the systems sections accessible but the training sections dense.

Authors’ abstract

Scaling computer-use agents from individual actions to complex software work requires experience that connects decisions to complete outcomes. We present NorthCUA, an integrated system for turbocharging computer-use agents through experience at scale. Its Experience Flywheel supplies diverse executable tasks to the NorthCUA Harness, which sustains long interactions, records decisions, and checks task outcomes. The Flywheel uses these records to curate verified trajectories. For unsolved tasks, recursive self-improvement evolves reusable skills and preserves verified milestones so later attempts build on earlier progress. Staged Learning combines this experience with grounding data from the Flywheel to develop GUI competence through mid-training and long-horizon supervised fine-tuning. The policy then learns from fresh attempts in the same Harness through joint desktop-web reinforcement learning, closing the loop between experience production and policy improvement. The resulting Nano, Mini, and Pro family achieves state-of-the-art results among open-source models across GUI grounding, desktop execution, and Web interaction under the reported evaluation settings. For more responsive Web workflows, NorthCUA-Ultrafast pairs direct action selection with open-ended generation on pre-RL Nano, offering a measured quality-latency tradeoff. Together, these advances establish NorthCUA as a high-performing model family and a reusable foundation for agents that carry complex software work from intent to completion.