WebHarbor: Docking Real Websites for Evolving GUI Agent Environments

AI summary

Overview

Research area: AI agents — specifically GUI/web agents, their evaluation benchmarks, and reinforcement learning environments.

Technical level: Intermediate. Readers should be comfortable with the basics of LLM-based agents, web interaction, and reinforcement learning, but the paper's core idea (rebuilding websites as controllable sandboxes) is conceptually accessible.

Scope: WebHarbor is a suite of 153 executable, Dockerized replicas of real websites plus 3,518 tasks, built through task-driven human–agent collaboration to provide stable environments for evaluating and RL-training GUI web agents.

What This Paper Is About

GUI web agents need environments to practice and be tested in, but the two obvious options both fall short: live websites change, restrict access, and rarely expose authenticated workflows, while synthetic environments tend to be shallow and visually unrealistic. WebHarbor's goal is to reconstruct real web services as self-contained, deterministically controlled environments that look and behave like the real thing, then use them both to benchmark agents and to train them with reinforcement learning.

Key Contributions

1. A suite of 153 executable, Dockerized website environments. Each reconstructs the visual interface, content, and functional workflows of a real-world web service while running under deterministic control, so results do not drift between runs.

2. A task-driven human–agent construction process. Environments are not built once and frozen: new tasks expand an environment's functional scope, coding agents implement the required changes, and human contributors audit visual fidelity, functional correctness, and task validity.

3. Task coverage that includes authenticated workflows. The suite contains 3,518 tasks spanning multi-step and post-authentication workflows — an area the abstract says existing work often fails to cover.

4. Demonstration of the environments as RL training infrastructure. A Qwen 4B-level GUI agent trained with reinforcement learning inside WebHarbor is reported to generalize to unseen tasks and to perform competitively with leading open-source GUI web agents across four benchmarks.

Main Findings

• Real websites can be reconstructed as deterministic environments: The authors report 153 Dockerized environments that reproduce the interfaces, content, and functional workflows of real web services, giving stable behavior for evaluation and training.

• Coverage extends to logged-in, multi-step workflows: The 3,518 tasks include post-authentication and multi-step scenarios, which the abstract identifies as a gap in prior work.

• The construction pipeline is iterative and audited: Environments grow as tasks demand new functionality, with coding agents making changes and humans checking visual fidelity, functional correctness, and task validity.

• RL training in WebHarbor transfers to unseen tasks: A Qwen 4B-level GUI agent trained with RL in these environments is said to generalize beyond its training tasks.

• Competitive with leading open-source web agents: That same trained agent performs competitively across four benchmarks, per the abstract. The abstract does not report per-benchmark scores, baseline identities, or margins.

• Cost-efficient training data: The suite is presented as a high-fidelity synthetic alternative to training on live websites, avoiding the cost and instability of the real web.

Methodology in Plain English

Rather than pointing an agent at the live internet, the researchers rebuild the websites themselves as small self-contained applications that run locally in containers. Each rebuilt site mimics a real service's look, content, and the actions a user can actually perform.

The build process starts from tasks. When a new task requires functionality the replica does not yet have, the environment is extended: coding agents write the necessary changes, and people then review the result on three fronts — does it look right, does it work correctly, and is the task itself valid. This loop repeats, so the environment set grows in capability alongside the task set.

With stable, controllable environments in hand, the team used them for two purposes: evaluating agents reproducibly, and running reinforcement learning. To test whether training there is useful, they trained a relatively small (4B-level Qwen) GUI agent with RL and checked whether it handled tasks it had not seen and how it compared with established open-source web agents on four benchmarks.

Why This Matters

Impact on research: Live-web benchmarks are unstable — content changes, pages go down, logins break — which makes comparisons across papers unreliable and invites accidental contamination when test content leaks into training data. Dockerized, deterministic replicas give a fixed target that any lab can rerun, and they double as a training ground for RL, which needs many reproducible episodes rather than one-shot evaluation.

Real-world applications:

• Browser automation products: Agents that fill forms, book appointments, manage accounts, or navigate dashboards need somewhere safe to be trained and regression-tested before touching production sites.

• Enterprise authenticated workflows: Internal tools, CRMs, and admin consoles sit behind logins and are poorly covered by public benchmarks; replica environments let agents be developed without real credentials or customer data.

• Agent evaluation and QA pipelines: Teams shipping agent updates can run the same task suite repeatedly to detect regressions, which is impractical against a shifting live web.

• Cost and privacy: Training against local replicas avoids paying for or depending on live-site access and avoids exposing real user accounts.

Industry relevance: Computer-use and web agents are an active product category, and the bottleneck is increasingly environment quality rather than model size. A public suite of realistic, functionally deep, authenticated sandboxes addresses a concrete infrastructure need for anyone building or benchmarking these systems.

Future Directions

• Keeping replicas faithful over time: Real websites evolve, so environments will need a maintenance process to stay representative without reintroducing the instability that motivated the work.

• Quantifying the sim-to-real gap: How well does performance in a Dockerized replica predict performance on the corresponding live site? The abstract does not address this, and it is the central open question for any synthetic environment.

• Scale and diversity of coverage: Extending beyond 153 sites and 3,518 tasks to broader domains, locales, and interface patterns, and understanding how much environment diversity RL actually requires.

• Beyond the browser: Whether the same task-driven, human-audited construction loop generalizes to desktop applications, mobile apps, and other agent environments.

• Safety and adversarial testing: Using controllable replicas to probe risky agent behaviors (irreversible actions, prompt injection in page content) that would be unsafe to test on live services.

Target Audience

Researchers and engineers working on GUI and web agents — particularly those building benchmarks, RL training pipelines, or evaluation infrastructure. It is also relevant to practitioners shipping browser-automation products who need reproducible test environments, and to readers interested in human–AI collaborative dataset and environment construction. Readers looking for detailed empirical results should note that the abstract reports no numeric scores or ablations, so the full paper is needed to assess the benchmark comparisons.

Authors’ abstract

GUI web agents require realistic and interactive environments that remain stable during both evaluation and training. Live website evaluation suffers from changing content, access restrictions, and limited coverage of authenticated workflows, while existing synthetic environments for training often lack functional depth and visual fidelity. We propose WebHarbor, a suite of 153 executable and dockerized website environments that faithfully reconstruct the visual interfaces, content, and functional workflows of real-world web services under deterministic control. We build WebHarbor through task-driven human-agent collaboration, where environments evolve as new tasks expand their functional scope, coding agents implement the required changes, and human contributors audit visual fidelity, functional correctness, and task validity. The resulting suite comprises 3,518 tasks spanning multi-step and post-authentication workflows, where existing works often fail to cover. Beyond reliable evaluation, WebHarbor provides high-fidelity synthetic web environments for cost-efficient reinforcement learning (RL). We demonstrate that a Qwen 4B-level GUI agent trained with RL in WebHarbor generalizes to unseen tasks and performs competitively with leading open-source GUI web agents across four benchmarks. The code and data are publicly available at https://github.com/aiming-lab/WebHarbor.