History-Aware Reasoning for GUI Agents

AI summary

Overview

Research area: Multimodal large language model (MLLM) based graphical user interface (GUI) agents, specifically reinforcement-learning post-training for episodic (multi-step) reasoning.

Technical level: Advanced. The paper assumes familiarity with multimodal LLMs, supervised fine-tuning, GRPO-style reinforcement learning, Chain-of-Thought reasoning, and GUI grounding benchmarks.

Scope (one sentence): The paper introduces a training framework, HAR, and a resulting 3B-parameter model, HAR-GUI-3B, that changes a GUI agent's reasoning from ignoring past interactions to explicitly reasoning over the episode's interaction history.

What This Paper Is About

GUI agents that operate phones and desktops are usually trained with a "System-2" Chain-of-Thought style, reasoning explicitly before acting. The authors observe that these agents treat each screen as an isolated problem: their reasoning is history-agnostic, meaning they do not use the earlier interactions that connect one screen to the next in a long task. The paper's goal is to give such agents short-term memory — the ability to reflect on previous steps and on their own prior errors — so that decisions are made in the context of the whole episode rather than a single screenshot.

Key Contributions

1. The HAR framework. A History-Aware Reasoning framework that combines tailored correction strategies with reflective learning to move a GUI agent's reasoning from history-agnostic to history-aware, strengthening short-term memory for episodic reasoning.

2. HAR-GUI-3B. A native, end-to-end GUI model built with the framework, described as having reliable episodic reasoning and screen perception.

3. A three-part mechanism. HAR comprises (i) reflective learning scenario construction, (ii) synthesis of tailored correction guidelines for hard samples, and (iii) a hybrid reinforcement learning reward function that includes a Memory-Augmented Reward (MAR).

4. OOD generalization benchmark and results. The authors manually annotate a challenging Chinese mini-program benchmark (415 tasks from Alipay mini programs, CLICK-only action space, four categories) and show HAR-GUI-3B outperforming comparable-size methods and competing with much larger models.

Main Findings

• AITW improvement: HAR-GUI-3B reaches an overall step success rate (SSR) of 70.2, above MP-GUI (69.2), ShowUI (68.3), InfiGUI-R1-3B (67.7), and Qwen2.5-VL-3B (65.4). Its strongest sub-scores are Single (77.9) and G.Apps (69.5).

• Mind2Web improvement: HAR-GUI-3B scores Cross-Task Acc. 47.9 / F1 89.6 / SSR 42.2, Cross-Website 49.1 / 87.3 / 41.2, and Cross-Domain 47.3 / 88.3 / 44.0 — the leading values in the comparison table.

• GUI-Odyssey: HAR-GUI-3B achieves an overall SSR of 62.31, ahead of Qwen2.5-VL-7B (58.39) and InfiGUI-R1-3B (50.62), despite the 7B baseline having more parameters.

• Grounding benchmarks: On ScreenSpot, HAR-GUI-3B averages 83.3, with Mobile Text 94.5 and Mobile Icon 81.0. On ScreenSpot-V2 it averages 86.2, with Mobile Text 96.5 and Desktop Icon 76.5. The authors credit the task mixing training strategy (TMTS) in Round-2 RL, noting that mixing grounding with episodic reasoning improved both tasks relative to training them sequentially.

• Out-of-distribution results: On the in-house Chinese mini-program benchmark, HAR-GUI-3B records SSR/SR of 82.76/24.30 (Takeout), 77.69/11.76 (Repast), 76.50/35.60 (Finance), and 69.19/21.78 (Insurance). It leads on SSR in all four categories and on SR in three, while Qwen2.5-VL-72B reaches higher SSR on Repast (85.02) and Finance (86.91) with 57.43 SR on Finance.

• GUI understanding: Relative to the foundational Qwen2.5-VL-3B under the same training settings, HAR-GUI-3B shows an overall gain of 3.07 points. The paper states it surpasses MP-GUI (8B) on S2W, WS, QA and QAS, and UI-TARS-1.5-7B on S2W and TP. The HAR-GUI-3B row of that table is not present in the supplied content.

• Ablation insight: Training purely with episodic reasoning data weakens the agent's grounding capability, which motivated the task mixing strategy. The paper also reports that RL under an inference-format instruction mainly affects action-level prediction rather than changing the agent's inherent reasoning mode.

Methodology in Plain English

The authors split training into two stages.

Stage one, GUI scenario warm-up. The base model (Qwen2.5-VL-3B-Instruct) is given broad GUI knowledge through supervised fine-tuning. The data includes 4k GUI understanding instances from MP-GUI, 20k grounding instances from OS-Atlas, 58k instances with synthesized System-2 Chain-of-Thought, and 100k "Action-to-Summary" (Act2Sum) entries generated by a teacher model (Qwen2.5-VL-72B-Instruct) from AITW, Mind2Web and GUI-Odyssey. Act2Sum asks the teacher to describe what an action accomplished given the goal and screen, so the agent learns goal-oriented summaries of past steps.

Stage two, learning from failure. The warm-up model is run over episode data and its mistakes are collected as hard samples. For each incorrect case, the teacher model writes no more than three correction guidelines (at most three per instance). The agent is then placed in a "reflection scenario" where it sees the goal, the guidelines, the interaction history, its earlier wrong action and its earlier wrong reasoning, and must produce a corrected action plus reasoning plus a statement reflecting on the error.

Reinforcement learning with GRPO is used, because the authors want the agent to discover episodic reasoning knowledge on its own rather than copy it. Rewards combine three parts: a format reward (1 or 0), an action reward, and a model-based Memory-Augmented Reward (MAR). The final reward is r = r_format × (r_action + γ × r_memory) with γ = 0.2. Coordinates-based actions such as CLICK use a multi-scale distance reward, defined by thresholds τ_norm = 0.1, τ_abs¹ = 40 and τ_abs² = 200, so that near-misses are still rewarded and the agent is pushed toward fine-grained screen detail. The MAR uses Qwen3-235B-A22B as a judgment function to check whether the chain-of-thought actually references the interaction history; if it does, the reward is 1, else 0.

Round-1 RL uses this reflection prompt with 15k synthesized guidance templates. Round-2 RL switches back to the inference-format instruction (harder, and the format available at deployment), and uses a task mixing training strategy that interleaves 15k grounding samples from OS-Atlas with 15k hard samples.

Why This Matters

Impact on research: The paper reframes a specific failure mode of GUI agents — history-agnostic reasoning — as a short-term memory problem rather than a general reasoning problem, and shows that the reward design used in RL post-training can be extended beyond action correctness to shape the reasoning trace itself. This suggests that the format of training instructions, not just the reward, determines whether reasoning behavior actually changes.

Real-world applications:

• Accessibility: agents that operate end-user devices on behalf of users with impairments, a use case the paper names explicitly.
• Automated testing: scripted or exploratory testing of mobile and desktop applications.
• Cross-ecosystem device automation: the setup covers mobile, desktop and website platforms, and the OOD benchmark covers Chinese mini-program scenarios.
• Long-horizon assistant tasks: multi-step workflows such as ordering, booking or form-filling, where history awareness matters most. The in-house benchmark's Takeout and Repast categories average 10.11 and 9.49 steps per task, illustrating this length.

Industry relevance: The work is a collaboration between Zhejiang University and Ant Group, and the evaluation includes Alipay mini programs, indicating direct commercial interest in mobile agent deployment. A 3B-parameter model that competes with 7B and larger models matters for on-device or cost-sensitive deployment. Training ran on 8 NVIDIA A100 80GB GPUs.

Future Directions

• How the memory reward generalizes. The Memory-Augmented Reward depends on a separate large judge model (Qwen3-235B-A22B). Whether a lighter or rule-based verifier can provide the same signal is an open question.

• Reducing dependence on teacher-generated guidelines. Correction guidelines are synthesized by Qwen2.5-VL-72B-Instruct from hard samples. Whether agents can generate their own error analyses without an external teacher is not addressed.

• Why the foundation model is history-agnostic. The paper documents the behavior and one way to correct it, but the underlying cause within the pre-trained Chain-of-Thought is not established.

• Going beyond single-step reflection. Training operates on incorrectly predicted actions within episodes; whether the approach extends to correcting full multi-step trajectories, or to longer horizons than those tested, is not reported.

Target Audience

Researchers and engineers working on GUI agents, multimodal LLM post-training, and reinforcement learning for reasoning; practitioners building device automation or accessibility products who need small models with strong multi-step performance; and readers interested in how reward design can shape an agent's reasoning traces rather than only its final outputs. Familiarity with GRPO, Chain-of-Thought prompting and GUI benchmarks is helpful but not required to follow the main argument.

Authors’ abstract

Advances in Multimodal Large Language Models have significantly enhanced Graphical User Interface (GUI) automation. Equipping GUI agents with reliable episodic reasoning capabilities is essential for bridging the gap between users' concise task descriptions and the complexities of real-world execution. Current methods integrate Reinforcement Learning (RL) with System-2 Chain-of-Thought, yielding notable gains in reasoning enhancement. For long-horizon GUI tasks, historical interactions connect each screen to the goal-oriented episode chain, and effectively leveraging these clues is crucial for the current decision. However, existing native GUI agents exhibit weak short-term memory in their explicit reasoning, interpreting the chained interactions as discrete screen understanding, i.e., unawareness of the historical interactions within the episode. This history-agnostic reasoning challenges their performance in GUI automation. To alleviate this weakness, we propose a History-Aware Reasoning (HAR) framework, which encourages an agent to reflect on its own errors and acquire episodic reasoning knowledge from them via tailored strategies that enhance short-term memory in long-horizon interaction. The framework mainly comprises constructing a reflective learning scenario, synthesizing tailored correction guidelines, and designing a hybrid RL reward function. Using the HAR framework, we develop a native end-to-end model, HAR-GUI-3B, which alters the inherent reasoning mode from history-agnostic to history-aware, equipping the GUI agent with stable short-term memory and reliable perception of screen details. Comprehensive evaluations across a range of GUI-related benchmarks demonstrate the effectiveness and generalization of our method.