SynthAgent: Adapting Web Agents with Synthetic Supervision

AI summary

Overview

Research area: LLM-based web agents, synthetic data generation, and supervised fine-tuning for environment adaptation.

Technical level: Intermediate. The core ideas are accessible, but the paper assumes familiarity with web agent architectures, accessibility-tree observations, and standard supervised fine-tuning.

Scope: This paper proposes SynthAgent, a fully synthetic supervision pipeline that adapts open-source web agents to previously unseen websites by jointly refining synthesized tasks during execution and synthesized trajectories after collection.

What This Paper Is About

Web agents trained on one set of websites perform poorly on new ones, and collecting environment-specific tasks and demonstrations by hand is expensive. Existing synthetic data pipelines generate both tasks and trajectories automatically, but produce low-quality data: tasks reference elements or states that do not exist, and trajectories contain redundant, misaligned, or wandering actions. SynthAgent's goal is to close this gap by improving the quality of synthetic supervision through coordinated refinement of both sides of the task-trajectory pair.

Key Contributions

1. A fully synthetic supervision framework (SynthAgent) that adapts web agents to new environments without test-task leakage or human annotation, consisting of four stages: categorized task synthesis, online task refinement, offline trajectory refinement, and supervised fine-tuning.

2. Identification of a central design tension in synthetic web agent data: task synthesis needs environment grounding to avoid hallucinations, yet grounding tasks aggressively during execution injects trajectory noise. The paper resolves this through a dual refinement strategy where each refinement compensates for the side effects of the other.

3. Categorized exploration, a function-aware exploration method that groups interactive elements by semantic role (for example Account Management, Search & Filters, Shopping Content) and samples across groups rather than uniformly, improving both task diversity and coverage of the target site.

4. Conflict-triggered task refinement and global-context trajectory refinement, including a lightweight conflict predicate that fires only when a task contradicts observed reality, lacks required arguments, or stalls, plus an offline edit model (Remove, Reorder, Drop, Keep) applied conservatively to clean trajectories.

Main Findings

• Large gains on WebArena: SynthAgent improves average success rate from 8.80 to 20.80 for Qwen2.5-VL-7B and from 8.85 to 17.26 for UI-TARS-1.5-7B, an average absolute gain of roughly +10.2 over base models and +5.1 over OS-Genesis, the strongest synthetic baseline.

• Baselines underperform or degrade: Self-Instruct gains little (11.50 overall) because it never grounds tasks in a real environment. Explorer performs worst of all (4.44 overall), actually degrading the base model, because its continuous task refinement produces overly long trajectories.

• Refinement frequency is the key difference: Explorer changes task intent an average of 8.6 times per episode versus 2.0 for SynthAgent, and 68.3% of Explorer trajectories exceed the step budget versus 6.3% for SynthAgent.

• Trajectory quality nearly doubles: GPT-5.1-rated trajectory quality reaches 82.6 (83.4 averaged across judges) for SynthAgent, versus 52.0 for OS-Genesis and 36.4 for Explorer. Trajectory completion rate is 96.5% versus Explorer's 30.5%, at roughly 60% of the API cost ($0.13 vs $0.22 per trajectory).

• Task diversity and human-likeness: SynthAgent achieves the highest task diversity score (72.6) with a t-SNE distribution resembling human-written test-set tasks, while Explorer's homepage-derived tasks cluster narrowly (diversity 54).

• Generalization to live websites: On Online-Mind2Web (136 live sites, no targeted synthesis or retraining), SynthAgent reaches 23.56 average success across three judges, competitive with GPT-4.1 at 24.56 and well above all synthetic baselines.

• Ablations confirm synergy: Removing categorized exploration costs 3.54%, removing task refinement costs 4.87%, and removing trajectory refinement costs 3.99%. Removing both refinements together behaves inconsistently across sites, supporting the claim that the two refinements are complementary rather than independently useful.

• Scaling behavior: Success rate rises from about 10.6 with 20 synthetic tasks per site (4% of the full set) to 20.8 with 500 tasks per site, with different websites peaking at different data volumes.

• Conservative reordering pays off: Reorder accounts for only 4.1% of trajectory edits, and a head-to-head evaluation of 100 reordered trajectories shows higher win rate (42% vs 27%) and quality score (68.9 vs 62.1) than the originals.

• Works across backbones: Qwen3-VL-8B also improves on both benchmarks when fine-tuned on SynthAgent data, though relative gains are smaller than for weaker base models, suggesting stronger models already internalize some of the demonstrated behaviors.

Methodology in Plain English

The pipeline treats a website as a partially observable environment where the agent receives a textual accessibility tree plus a screenshot, and emits actions such as click, type, and scroll. The method proceeds in four steps.

First, task synthesis via categorized exploration. Rather than clicking elements at random, the system asks an LLM to sort the interactive elements on a page into functional categories, then samples at most two unvisited elements from each category. This keeps one dense region of a page from dominating exploration and encourages broader coverage. Each sampled interaction produces a triplet (before page, action, after page), and an LLM proposes a high-level multi-step task for which that action is a plausible grounded first step.

Second, task refinement during execution. The agent starts executing each proposed task. A lightweight conflict check fires when the task references UI that does not exist, when the task is missing required parameters such as a username, or when the agent stalls through repeated no-ops or error loops. Only then does an LLM rewrite the task, following four principles: fill in missing details, align with what was actually observed, downscope when blocked, and stay in the same task category. This differs from prior work that refines continuously, which drifts from the original intent.

Third, trajectory refinement after collection. Because task edits leave early trajectory segments misaligned with the final task, and because agents wander even without edits, the system post-processes the full trajectory with knowledge of the final task. It can remove irrelevant or redundant steps, reorder locally commutable steps, drop the whole trajectory if too noisy, or keep it unchanged. Edits are applied conservatively offline with no further environment interaction, favoring precision over recall to avoid breaking hidden causal dependencies.

Fourth, agent fine-tuning. Refined task-trajectory pairs are split into per-step training examples where the model predicts the next action given the task and a three-step history window. The agent is fine-tuned with standard next-action supervised loss (learning rate 1e-5, batch size 32, three epochs, 500 pairs per site, 30-step episode budget).

Why This Matters

Impact on research. The paper reframes synthetic agent data from a generation problem into a data-quality problem, and shows that the naive fix for hallucinated tasks (aggressive on-the-fly grounding) creates an equally damaging failure mode in trajectory noise. The dual refinement framing gives future work a concrete template for balancing task feasibility against trajectory alignment, and the released pipeline is positioned as a source of large-scale diverse data for agentic reinforcement learning.

Real-world applications:

• Deploying browser or computer-use assistants on a company's internal tools, where no public demonstrations or labeled tasks exist.
• Adapting agents to newly launched or redesigned websites without a manual annotation campaign.
• Bootstrapping agent evaluation suites for sites that lack existing benchmark tasks.
• Generating domain-specific interaction data for verticals such as e-commerce checkout, content management, ticketing, and travel booking.

Industry relevance. The cost accounting matters commercially: SynthAgent produces cleaner trajectories than Explorer at roughly 60% of the per-trajectory API cost, and avoids the test-set leakage that undermines many prior agent training setups where tasks are pulled directly from evaluation benchmarks. That makes the approach more defensible as a genuine adaptation method rather than a benchmark-fitting trick.

Future Directions

• Live-site synthesis. The authors note that all experiments run on controlled self-hosted environments. Extending synthesis to live websites raises unsolved problems: constant site changes, CAPTCHAs, security mechanisms triggered by automated visits, and copyright or privacy barriers that prevent releasing the resulting data.

• Stronger training objectives. The current work uses only standard SFT. Direct Preference Optimization, online reinforcement learning, or richer supervision signals such as inner monologue annotations and structured reasoning traces could plausibly yield larger gains, but require reward models or verification mechanisms that this paper does not build.

• Synthesis quality optimization. The pipeline relies on GPT-4.1 with no hyperparameter tuning and does not explore alternative prompting strategies, stronger LLMs, or tuned execution parameters such as exploration depth and sampling budget.

• Combining synthetic and human supervision. OpenCUA-style results suggest human-annotated supervision substantially improves agent training, so integrating human signals into the refinement loop is a natural complementary direction.

Target Audience

Researchers and engineers working on web agents, GUI agents, or computer-use models who need to adapt agents to new environments without collecting human demonstrations. It is also relevant to practitioners building synthetic data pipelines for agentic training, and to readers interested in how data quality issues (hallucinated tasks, noisy trajectories, test-set leakage) propagate into downstream agent performance. Readers should have some familiarity with supervised fine-tuning and LLM-based agent architectures to get full value from the methodology section.

Authors’ abstract

Web agents struggle to adapt to new websites due to the scarcity of environment specific tasks and demonstrations. Recent works have explored synthetic data generation to address this challenge, however, they suffer from data quality issues where synthesized tasks contain hallucinations that cannot be executed, and collected trajectories are noisy with redundant or misaligned actions. In this paper, we propose SynthAgent, a fully synthetic supervision framework that aims at improving synthetic data quality via dual refinement of both tasks and trajectories. Our approach begins by synthesizing diverse tasks through categorized exploration of web elements, ensuring efficient coverage of the target environment. During trajectory collection, tasks are refined only when conflicts with observations are detected, which mitigates hallucinations while preserving task consistency. After collection, we conduct trajectory refinement with global context to mitigate potential noise or misalignments. Finally, we fine-tune open-source web agents on the refined synthetic data to adapt them to the target environment. Experimental results demonstrate that SynthAgent outperforms existing synthetic data methods, validating the importance of high-quality synthetic supervision. The code is publicly available at https://github.com/aiming-lab/SynthAgent.