InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training

AI summary

Overview

Research area: GUI (graphical user interface) agent training and automated web environment synthesis, submitted under Natural Language Processing (cs.CL) by authors at Microsoft Research Asia, Peking University, and Nanjing University.

Technical level: Intermediate. The paper assumes familiarity with reinforcement learning concepts such as GRPO and dense versus sparse rewards, plus general web development concepts (frontend, backend, localStorage), but the core argument is explained in accessible terms.

Scope: The paper describes InfiniteWeb, a system that automatically generates complete, functional multi-page websites together with tasks and automatic evaluators, and reports how training GUI agents on those generated environments affects performance on OSWorld, Online-Mind2Web, and MobileWorld.

What This Paper Is About

Training GUI agents requires environments where the agents can practice, but existing benchmarks such as MiniWoB++, WebArena, and OSWorld are manually constructed and therefore contain only tens to hundreds of applications. This scale is too small and too uniform for agents to learn generalizable interaction skills. The paper's goal is to remove that bottleneck by automatically generating realistic, functional web environments at scale, complete with tasks and reward evaluators that reinforcement learning can use.

Key Contributions

1. InfiniteWeb system. The authors propose InfiniteWeb, described as the first system specifically designed to generate functional web environments with verifiable evaluators for GUI agent training at scale.
2. Superior website construction. Experiments show the system surpasses advanced commercial coding agents in building realistic web environments on WebGen-Bench, achieving better visual and functional quality.
3. Measurable agent training gains. GUI agents trained on InfiniteWeb-generated environments improve by +6.9% on OSWorld, +5.7% on Online-Mind2Web, and +3.9% on MobileWorld, which the authors present as evidence of realism, quality, and cross-platform transferability.
4. Public release. The code and generated websites are released at https://github.com/microsoft/FIVE-UI-Evol.

Main Findings

• Functional correctness on WebGen-Bench. InfiniteWeb achieves the highest overall score of 85.6%, compared with Codex at 81.2%, Claude-Code at 74.3%, and Bolt.diy at 67.0%. Results are averaged over three runs.
• Category-level strengths. InfiniteWeb scores 80.9% in Functional Testing and 82.8% in Design Validation Testing, the two categories it describes as most challenging. It also scores 94.1% in Data Display Testing, below Codex's 96.2%.
• Statistical significance. Welch's t-tests report InfiniteWeb significantly outperforming Bolt.diy (t=14.81, p<0.001), Claude-Code (t=6.33, p<0.01), and Codex (t=6.57, p<0.05).
• Visual quality. In LLM-as-Judge pairwise comparisons over 200 generated websites, InfiniteWeb wins against all baselines with a 69–85% win rate. Human evaluation confirms 91% agreement with the automated assessments.
• Agent training results. Training UI-TARS-1.5-7B with GRPO on 600 generated tasks raises OSWorld performance from 24.5% to 31.4% (+6.9%), Online-Mind2Web from 23.0% to 28.7% (+5.7%), and MobileWorld from 6.4% to 10.3% (+3.9%).
• Scaling with task volume. On OSWorld, performance rises from 24.5% (baseline) to 27.3% with 200 tasks, 29.7% with 400 tasks, and 31.4% with 600 tasks.
• Environments are harder and more discriminative. Agents score 2–3× lower on InfiniteWeb than on OSWorld, and successful tasks take longer trajectories (UI-TARS: 10.3 steps on InfiniteWeb vs 9.0 on OSWorld). The gap between Agent S2 and UI-TARS is 6.7 percentage points on InfiniteWeb versus 2.8 on OSWorld.
• TCTDD ablation. Removing the task-centric test-driven development validation loop reduces the overall score by 5.0 points; without it, the method still reaches 80.6%, comparable to Codex.
• Backbone model ablation. Replacing GPT-5 with GPT-4.1 drops the score by 8.2 points (85.6 to 77.4), yet still outperforms Claude-Code using GPT-5 (75.8%).
• Dense reward impact. Running UI-TARS-1.5-7B on 4,000 generated tasks with 4 trajectories per task, dense reward enables GRPO learning from 767 discriminative tasks versus 174 with binary reward, a 4.4× increase.
• Generation cost. Each website consumes roughly 0.36M input tokens and 0.34M output tokens, costing approximately $1.93 at GPT-5 batch pricing ($0.625/M input, $5.00/M output). Median generation time is about 20 minutes per site.
• Transfer behaviors observed. Case studies attribute cross-domain gains to three capabilities learned from website training: exploration persistence, flow completeness, and loop avoidance.

Methodology in Plain English

InfiniteWeb takes two inputs: a short website description (a "website seed") and a reference design image. Both are extracted from Common Crawl, where pages are rendered in a headless browser for screenshots and an LLM writes the seed description while filtering out pages that violate robots.txt or contain illegal content.

The pipeline has four stages. First, the Unified Specification Stage generates realistic user tasks for that seed, then derives shared data models, a page architecture, and a common set of programming interfaces from those tasks, so every page uses identical signatures and data formats. Interfaces are made user-facing by wrapping system-managed parameters such as userId and sessionId so they are retrieved automatically from localStorage.

Second and third, two stages run in parallel. The Task-Centric Backend applies task-centric test-driven development: tests and implementation code are generated together from the same pre-generated data, tests are executed, and when they fail the LLM receives the failing test, the expected versus actual output, and the relevant code to produce a fix, repeating until all tests pass or a maximum of 8 iterations is reached. Most websites need 1–3 iterations. The Design-Guided Frontend uses a vision-language model to extract color systems, typography hierarchy, spacing rules, and component styling from the design image, then generates a shared page framework plus per-page HTML, CSS, and a JavaScript UI layer linked to the backend through data attributes. An initialization script writes data to localStorage so state persists across pages without a server.

Fourth, Evaluator Generation produces a JavaScript function per task that checks both naturally stored state variables and instrumentation checkpoints recorded in localStorage as task steps execute. Checkpoints are weighted, so an agent receives partial credit, for example 0.35 for attempting a newsletter subscription, 0.30 for a matching stored record, and 0.35 for on-site confirmation.

For evaluation, all generation methods use GPT-5 with reasoning effort set to "high" as the backbone for fair comparison. Website generation used temperature 0.7, maximum output tokens 32,000, 8–10 tasks per website, a maximum of 12 pages, and up to 8 TCTDD iterations.

Why This Matters

Impact on research. The paper reframes the GUI agent bottleneck as an environment scarcity problem rather than a model capability problem. By generating environments, tasks, and evaluators together, it offers an alternative to constructing benchmarks by hand and provides dense reward signals that the authors show expand the effective training signal by 4.4×.

Real-world applications:

• Web automation assistants that complete multi-step tasks like booking, shopping, or subscribing on real sites.
• Enterprise software agents that operate office suites, since training transferred to LibreOffice Calc, Impress, and Writer tasks.
• Mobile assistants, given the +3.9% transfer to MobileWorld without additional fine-tuning on mobile data.
• Software testing and QA, since the generated task evaluators and TCTDD loop already validate functional correctness of websites.

Industry relevance. The economics are notable: the authors report generating a website for about $1.93 and roughly 20 minutes, and contrast this with OSWorld requiring approximately 1,800 person-hours for 369 tasks. They also note generation accounts for only 6.9% of total training time because websites generate in parallel (600 websites in approximately 125 minutes). For companies training GUI agents, this suggests training data and environments can be produced as an engineering pipeline rather than a manual annotation effort.

Future Directions

1. Cross-website tasks. Current tasks operate within a single website. Scenarios such as comparing prices across shopping sites or aggregating information from multiple sources are not covered, though the authors observe cross-application improvement on OSWorld multi-app tasks from 3.8% to 9.7%.
2. Native mobile environments. The system only generates web environments. Generating native mobile application environments for mobile GUI agent training is left to future work.
3. Lowering generation cost and latency. Despite the low per-site price, further reducing per-website cost and latency remains an engineering direction, especially for higher-volume generation.
4. Scaling the environment count. The authors state that improvements scale with training data volume, indicating that generating more diverse environments could yield further gains.

Target Audience

Researchers and engineers working on GUI agents, reinforcement learning for embodied or interactive AI, and synthetic data generation will benefit most. The paper is also relevant to teams building web generation or code generation systems, since its unified specification and task-centric test-driven development ideas apply beyond GUI agents, and to practitioners deciding whether hand-built environments or automatically generated ones are the better investment for agent training.

Authors’ abstract

GUI agents that interact with graphical interfaces on behalf of users represent a promising direction for practical AI assistants. However, training such agents is hindered by the scarcity of suitable environments. We present InfiniteWeb, a system that automatically generates functional web environments at scale for GUI agent training. While LLMs perform well on generating a single webpage, building a realistic and functional website with many interconnected pages faces challenges. We address these challenges through unified specification, task-centric test-driven development, and a combination of website seed with reference design image to ensure diversity. Our system also generates verifiable task evaluators enabling dense reward signals for reinforcement learning. Experiments show that InfiniteWeb surpasses commercial coding agents at realistic website construction, and GUI agents trained on our generated environments achieve significant performance improvements on OSWorld and Online-Mind2Web, demonstrating the effectiveness of proposed system.