Cowork-Gym: Synthesizing Interactive Work Environments at Scale

AI summary

Overview

Research area: AI agents / agentic post-training and environment synthesis (LLM-based coworkers operating in shared workspaces). Technical level: Intermediate (assumes familiarity with LLM agents, supervised fine-tuning, and reinforcement learning, but the paper's structure is explained here in plain terms). Scope: Cowork-Gym is a framework that automatically generates large numbers of executable, multi-turn "cowork" task environments with simulated users, and the paper shows that post-training on the resulting trajectories improves agentic benchmark performance for three Qwen model sizes.

What This Paper Is About

Agents increasingly do multi-step work in a persistent workspace — editing files, running tools, producing deliverables — while a user may interrupt with new constraints or directly edit the shared files. Training and evaluating this behavior requires many environments, each of which must keep the task, the initial files, the tools, the simulated user, and the success checks mutually consistent; building them by hand is slow and error-prone.

The paper's goal is to synthesize such environments at scale and to show they are useful as training data. It introduces Cowork-Gym, which generates 64,888 environments across 14 application scenarios using a build–execute–revise loop, rolls them out with a stateful user simulator, and uses the filtered trajectories to post-train 9B, 27B, and 35B-A3B Qwen models.

Key Contributions

1. Cowork-Gym synthesis framework. A pipeline that combines taxonomy-guided generation (sampling from a six-dimensional taxonomy with 17 capability labels and 14 top-level application scenarios in a three-level hierarchy) with resource-grounded generation (deriving tasks from existing artifacts such as financial reports, legal documents, repositories, and paper collections). Every task is instantiated as an executable environment and repaired through a build–execute–revise loop that ends when it passes release criteria or exhausts the iteration budget.

2. Stateful user simulator. A simulated user that runs outside the acting agent's sandbox, holds private state (persona, latent goal, knowledge boundary, signal plan), and reacts to the agent's actual progress through both dialogue and direct edits to the shared workspace — including file patches that invalidate earlier observations and force the agent to revise dependent outputs.

3. Hybrid verification and trajectory filtering. A verifier combining deterministic code checks on the final workspace with LLM-based rubric judging of semantic correctness, plus a filtering operator that checks record integrity, task outcome, and whether success claims match observed outcomes. The paper also studies model-conditioned trajectory selection using average negative log-likelihood (NLL) and Rank–Surprisal Ratio (RSR).

4. Empirical demonstration across three model scales. SFT, then outcome-based RL (GRPO) on 2,873 environments, improving all three base checkpoints, with Ours-27B ranking second overall across six agentic benchmarks and achieving the highest scores on PawBench and ClawEval among all evaluated models, outperforming models with over 100B total parameters on both.

Main Findings

• Broad average gains, 17 of 18 comparisons. Averaged across six agentic benchmarks, Ours-9B, Ours-27B, and Ours-35B-A3B outperform their corresponding base checkpoints by 8.7, 6.4, and 4.1 points respectively. Improvement appears in 17 of the 18 matched model–benchmark comparisons.

• Ours-27B ranks second overall. Its six-benchmark average is 0.6119 versus 0.6157 for GLM-5.2, which places it second among evaluated models. It records the best PawBench score (0.8400, 41.82% pass rate) and the best ClawEval score (0.8508, 80.6% pass rate) of all evaluated models.

• Ours-35B-A3B leads its parameter group. Its average is 0.5665, ahead of its base Qwen3.6-35B-A3B (0.5251) and of Agents-A1 (0.4515), Nex-N2-mini (0.5014), and Macaron-V1 (0.4269).

• Gains target different weaknesses per backbone. The 9B model improves most on Workspace-Bench (+14.64 points) and ALE (+13.39), whereas the 27B model improves most on ClawEval (+11.97) and ClawMark (+10.17). The authors read this as different models being helped on different weaknesses rather than gains simply scaling with parameter count.

• A clear boundary on WildClawBench. The 9B model improves by only 1.36 points there and the 35B-A3B model declines by 0.52 points, and all three models remain substantially behind GLM-5.2 (0.5866 on WildClawBench).

• Synthesis yield and execution depth. From 77,729 raw trajectories, validity-based filtering retains 24,732 (31.82%), and model-conditioned selection retains 17,312 for SFT training. The evidence-valid set has a median of 10 tool-using assistant steps per trajectory, with the middle 50% spanning 6–16 steps. 3,437 trajectories (13.90%) exceed 20 tool-using steps and 303 (1.23%) exceed 40.

• Multi-turn interaction is a minority but is deeper. Single-turn trajectories make up 67.01% and multi-turn 32.99% of the raw corpus. Single-turn median is 10 steps (interquartile interval 6–17); multi-turn median is 15 (9–24). 32.69% of multi-turn trajectories exceed 20 tool-using steps versus 15.80% of single-turn.

• Adding multi-turn supervision helps. For 27B, single-turn-only training reaches ClawEval 84.13, ClawEval-MT 84.01, and ClawMark 69.27; adding multi-turn trajectories raises these to 85.08, 84.44, and 72.50. The largest additional gain is on ClawMark (+3.23 points over the single-turn-only model).

• Training transfers across harnesses. On PawBench with Qwen3.5-9B, training with QwenPaw only or OpenClaw only improves pass rates on all three evaluation harnesses, including the never-used Hermes harness (+7.2 and +9.0 points respectively). Training on OpenClaw only improves QwenPaw performance by +10.0 points, more than the +9.1 gained from QwenPaw-only training. Training on both sources reaches 31.8% on QwenPaw and 31.8% on OpenClaw.

• Model-conditioned selection helps most on ClawMark. For 9B, no filtering gives ClawMark 0.3267, NLL selection gives 0.4010, and RSR gives 0.3991. For 27B, no filtering gives 0.6557, NLL 0.6806, and RSR 0.6730. For 35B-A3B, no filtering gives 0.5394, NLL 0.5678, and RSR 0.5739. NLL and RSR each retain 70% of candidates.

• Training curves plateau unevenly. Tracking ClawEval from initial to final supervised checkpoint, the 9B model gains +4.66 in Score and +5.00 percentage points in pass@3; the 27B model reaches the highest final performance on all three metrics; the 35B-A3B model plateaus early on Score and pass@3 but continues improving on pass^3, with a +5.80 percentage point endpoint gain.

• Scenario coverage is broad but non-uniform. Office productivity and software engineering form the largest groups, while less frequent domains remain represented. Distribution percentages in the paper are computed over 64,888 environments.

Methodology in Plain English

Step 1 — Write the task specification. A task is generated in one of two ways. In the top-down route, conditions are sampled from a six-dimensional taxonomy (capabilities, application scenarios, task structure and difficulty, user and deliverable requirements, input/output modalities, and safety); incompatible label combinations are discarded through schema and cross-field checks. In the bottom-up route, the task is built around an existing resource, such as a financial report, legal document, or code repository, with complexity coming from cross-file dependencies, incomplete evidence, or required artifact transformations. Only tasks where the resource materially contributes to completion are kept.

Step 2 — Build, execute, and revise the environment. An environment is a five-part object: the task instruction and requirements, the initial workspace state, the interface set, the user policy, and the verifier. Three role-specialized agents on a shared coding runtime handle construction: the Builder creates the workspace, interfaces, and user policy; the Verifier Constructor independently builds the hybrid verifier in an isolated sandbox with write access restricted to tests/; and the Issue Detector runs the assembled environment, tests the verifier on a valid reference completion and on controlled failure states, and applies a rule audit covering package structure, permission allowlists, and leakage of answers, verifier logic, or private information. Defects are routed back by ownership — workspace and interface issues to the Builder, verifier issues to the Verifier Constructor — and the loop repeats until release criteria are met or the iteration budget runs out.

Step 3 — Roll out with a simulated user. Environments are packaged in the Harbor task format so the same environment can be run under different agent harnesses. During rollout, the user simulator sits outside the agent's sandbox with private context in a .user/ directory and turn-specific instructions and authorized workspace edits in .patch/. After each agent work segment it reviews progress and may send a message, revise a draft, or patch source data. The acting agent only sees public messages and the resulting workspace changes.

Step 4 — Filter and format trajectories. A trajectory is kept only if the event log is intact and free of leaked private information, the hybrid verifier confirms task outcome, and any success claims match observed behavior (recovered tool errors are allowed). Optionally, NLL or RSR scores rank trajectories within equal-frequency length bins and keep a 70% fraction, to prioritize data suited to the target student. Retained trajectories are normalized into the training format, with loss computed only on agent-generated tokens.

Step 5 — Post-train. Supervised fine-tuning on filtered trajectories is followed by GRPO using the environment's verifier as outcome reward. The reward is a weighted geometric mean of the code score and the LLM score, assigned zero whenever either component is zero; groups with zero reward variance are excluded, and MoE router weights are frozen. A second route, on-policy distillation, initializes the student from the base checkpoint and trains it to match a same-size SFT or SFT+RL teacher's next-token distribution on interaction histories the student itself generated, motivated by the concern that direct adaptation can weaken capabilities outside the training distribution.

Why This Matters

Research impact. The paper reframes agentic training data as a systems problem: correctness depends on five components staying consistent as the user changes requirements mid-task. Its build–execute–revise loop, hybrid code-plus-LLM verifier, and claim-versus-outcome filtering offer reusable machinery for anyone synthesizing interactive agent benchmarks. The demonstration that a 27B model can beat models with over 100B total parameters on PawBench and ClawEval is also a concrete data point on how much environment quality matters relative to raw scale.

Real-world applications:

• Training and evaluating enterprise "coworker" agents that must edit shared documents, spreadsheets, and code while a human changes the inputs.
• Regression testing for agent products: each synthesized environment ships with a verifier, so a task can be re-run when a model or harness changes.
• Domain onboarding in finance, legal, and healthcare-adjacent workflows, where resource-grounded generation can build tasks around real report and document structures.
• Harness portability checks: the cross-harness results suggest capability learned from one runtime transfers to another, which matters for teams shipping agents across multiple execution frameworks.

Industry relevance. The work comes from Alibaba Token Foundry and is released with a model collection and dataset on Hugging Face (agentscope-ai/qwenpaw-flash and agentscope-ai/Cowork-Gym), making the pipeline and the trained checkpoints directly usable rather than only described.

Future Directions

• Improving the weakest benchmark. WildClawBench is the clear failure mode: +1.36 points for 9B, a 0.52-point decline for 35B-A3B, and all three models far behind GLM-5.2 (0.5866). What task properties make an environment transfer poorly to that benchmark is unresolved.

• Resolving the agentic-gains versus general-capability trade-off. The paper poses this as Q4 and motivates on-policy distillation as the answer, but the excerpt does not report the OPD comparison results, so whether the second route actually preserves general capability better than direct SFT remains an open question in this text.

• Scaling yield and reducing rejection. Roughly two-thirds of raw trajectories fail validity filtering (24,732 of 77,729 retained) and unresolved environments are rejected after the iteration budget. Raising the acceptance rate of the Builder, Verifier Constructor, and Issue Detector is a direct lever on cost.

• Better selection criteria. NLL and RSR help, but their effect varies by benchmark and model size (RSR is best on ClawMark for 35B-A3B, NLL is best there for 9B and 27B). Whether stronger proxies exist for "which trajectory will teach this student most" is left open.

Target Audience

Researchers and engineers building agentic training pipelines who need scalable, verifiable environments rather than static prompt sets; teams post-training mid-sized models for tool use and long-horizon workspace tasks; and practitioners evaluating agent harnesses who want reproducible, verifier-backed tasks. Readers without background in reinforcement learning or agent runtimes will find the problem framing accessible, but the experimental sections assume familiarity with SFT, GRPO, and distillation objectives.

Authors’ abstract

Cowork agents navigate dependencies across tools and files while adapting to changing user requirements and edits to shared workspaces. Scaling such environments requires maintaining consistency among task objectives, executable resources, user behavior, and success criteria. We introduce Cowork-Gym, a framework that synthesizes Cowork environments through a build-execute-revise loop that uses execution evidence and review feedback to validate and repair generated environments. To capture realistic Cowork dynamics, a stateful user simulator responds to the agent's actions through dialogue and direct edits to the shared workspace. We synthesize 64.9K environments across 14 application scenarios and use them to post-train 9B, 27B, and 35B-A3B Qwen models with supervised fine-tuning, reinforcement learning, and on-policy distillation. Averaged across six agentic benchmarks, these models outperform their corresponding base checkpoints by 8.7, 6.4, and 4.1 points, respectively. Ours-27B ranks second overall among the evaluated models and achieves the highest average scores on PawBench and ClawEval, outperforming models with over 100B total parameters on both benchmarks.