OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents

AI summary

Overview

Research area: Natural Language Processing, specifically LLM-based autonomous agents and agent execution harnesses. Technical level: Intermediate (readers should be comfortable with LLM agent concepts such as ReAct loops, tool calling, and context windows). Scope: The paper introduces OneDayAgent, a harness that structures long-horizon, cross-environment, multimodal everyday agent tasks through decomposition, execution memory, and verification/repair, evaluated on the AgentIF-OneDay benchmark.

What This Paper Is About

LLM agents increasingly receive open-ended everyday requests that span work, study, and life, and these requests are long-horizon, cross-environment, and multimodal. Over many steps, agents lose track of early constraints (goal drift), fail to pass intermediate state between environments (state loss), and run out of usable context. Prior work addresses these failure modes one at a time; this paper asks whether a single harness can manage them jointly and still work across different backend LLMs.

Key Contributions

1. A unified long-horizon harness (OneDayAgent). It jointly addresses task decomposition, execution memory, and deliverable verification/repair, operating over a unified action space covering web, computation, file, and multimodal tools.
2. A new state of the art on AgentIF-OneDay. With the GLM-5.2 backend, OneDayAgent reaches an overall score of 0.821 across 104 tasks, leading across all task types, domains, rubric dimensions, and input-attachment settings.
3. Cross-backend generalization without tuning. The same unchanged harness runs on five backend LLMs from three model families (GLM/Zhipu, Gemini/Google, Qwen/Alibaba), with overall scores from 0.613 to 0.821.
4. Open-sourced harness and trajectories for the broader community.

Main Findings

• Leading overall score. OneDayAgent with GLM-5.2 achieves 0.821 overall, outperforming official general-purpose agent baselines (Minimax-Agent 0.562, ChatGPT-Agent 0.626, Genspark 0.635, Manus 0.645, AutoClaw 0.799) and an additional Codex run using GPT-5.5 medium (0.664).
• Broad, not narrow, gains. The GLM-5.2 run leads on Open Workflow Execution (0.818), Latent Instruction Inference (0.821), Iterative Refinement (0.829), all three domains (work 0.823, life 0.731, study 0.784), all three rubric dimensions (instruction following 0.835, factuality 0.846, logic/functionality 0.782), and both attachment settings (with 0.847, without 0.782).
• Both modules help, with very different costs. A 2×2 ablation shows DIRECT (neither module) at 0.771, decomposition alone at 0.804 (+3.3 pp), verification alone at 0.804 (+3.3 pp), and FULL (both) at 0.821 (+5.0 pp). The text reports the decomposition-only and verification-only scores as 0.8039 and 0.8044 respectively. VERIFY adds only 2.2 minutes over DIRECT while matching DECOMP's score, whereas DECOMP adds 10.6 minutes and roughly 60% more tool calls. FULL has the best score but the lowest score-per-latency ratio (1.53 vs. 2.80 for DIRECT).
• Always-on modules are not uniformly optimal. VERIFY beats FULL on 17 tasks, DECOMP on 13, and DIRECT on 12, even though FULL produces the most perfect tasks (58).
• Decomposition changes execution shape. Most tasks are split into two to four subtasks, with only 16 of 104 tasks executed as a single subtask. One-subtask tasks average 20.6 minutes and 17 tool calls, while five-subtask tasks average 117.2 minutes and 156 tool calls.
• Repair is a rare, risk-focused mechanism. 95 of 104 tasks pass verification on the first attempt; 9 enter repair, of which 6 are recovered and 3 still fail. Repair concentrates in Iterative Refinement tasks, study-domain tasks, and long time-budget tasks.
• Context compression keeps long runs feasible. 35 of 104 tasks triggered compression, with the highest-pressure task accumulating roughly 350K context tokens. The correlation between compression count and score is near zero, indicating no systematic score degradation.
• Harness transfer is broad but behavior is backend-specific. All five backends complete 104/104 tasks. GLM-5.2 (744B) uses a high-cost profile at 53.6 minutes, 51.6 tool calls, and 585.7 KB context per task; Gemini-3.1-Pro-Preview is leaner at 21.4 minutes, 18.7 tool calls, and 118.1 KB context; Qwen3.6-27B triggers the highest repair rate at 56.7%.
• Scaling is a trend, not a law. Larger disclosed-scale models tend to do better (Qwen3.5-9B 0.624, Qwen3.5-397B-A17B 0.708, GLM-5.2 0.821), but ordering is not monotonic: Qwen3.6-27B does not dominate Qwen3.5-9B, and Gemini-3.1-Pro-Preview, widely believed to exceed 1T parameters, reaches only the second-best score (0.743).
• Conservative judging. The official release used Gemini-3-Pro-Preview as judge, which was unavailable in the June 2026 evaluation environment. The authors substituted Gemini-3.1-Pro-Preview and report a paired comparison showing it scores 3.12 percentage points lower.

Methodology in Plain English

The harness begins with the user's request plus any attachments and treats that original request as a fixed global intent. A planner breaks the request into an ordered list of bounded subtasks (up to 6). Each subtask runs inside its own ReAct loop, where the backend LLM reasons, calls tools, and observes results, with the harness keeping low-level traces out of later subtasks.

Execution memory works at three levels: tool observations from searches, page visits, and file reads are compressed into bounded evidence (long visit output above 20K characters becomes a summary plus a bounded raw prefix); subtask boundaries pass forward only a compact answer plus result-file handles rather than the full trace; and at the dialogue layer, when context exceeds 0.9× the budget, earlier rounds are compressed into an LLM-generated technical summary (maximum 8000 characters) while the system prompt, original task, and 3 recent rounds stay verbatim, with deterministic pruning at 0.95× as a fallback.

After all subtasks finish, a synthesizer combines results into a candidate deliverable. A global verification pass then checks that deliverable against the original request, the submitted subtask answers, and the declared attachments. If defects are found, a targeted repair loop (up to 3 attempts, 50 ReAct iterations each) patches only the inconsistent part rather than restarting subtasks, and the result is re-evaluated.

Tools are grouped into web access (search, visit), academic search (Google Scholar, OpenAlex), computation (Python, command execution), file workspace (read, write, edit), and multimodal processing (analyze_image, generate_image). Search uses Serper, visit uses Jina, summarization uses DeepSeek-V4-Pro, vision uses Qwen3-VL-235B-A22B-Instruct, and image generation uses Qwen-Image-2512-Lightning. Common settings across backend runs: temperature 1.0, top-p 0.95, 128K max tokens, 200 ReAct iterations, 7200-second timeout.

For scoring, AgentIF-OneDay uses binary instance-level rubrics with bonus and penalty criteria; positive criteria add points, penalties subtract, and task scores are clipped and normalized to [0,1] before averaging. The paper reports 104 tasks and 767 instance-level scoring points.

Why This Matters

Impact on research. The paper reframes the agent harness, not just the backend model, as a transferable layer worth studying. It provides an ablation-cost analysis showing that verification-only is the strongest cost-effective configuration, and it shows that backend differences manifest as distinct execution styles (latency, tool volume, repair rate) even under an identical workflow — evidence that model rankings alone do not predict agentic performance.

Real-world applications:

• Automated research-to-deliverable workflows: gathering web evidence and producing a report or slide deck, as in the "Language of Flowers" PPT case study.
• Personal and professional assistance: complex multi-step requests spanning file editing, web lookups, and artifact creation.
• Multimodal document work: revising decks and documents that mix text, images, tables, and attachments.
• Cost-aware agent deployment: choosing between full decomposition and verification-only based on whether maximum score or lower latency matters more.

Industry relevance. The cost asymmetry finding matters directly for production: decomposition adds substantial latency and roughly 60% more tool calls for the same score that verification achieves at near-DIRECT cost. The runtime accounting is also concrete — the GLM-5.2 run used approximately 9.0K backend calls (292.4M input, 10.7M output tokens), plus auxiliary services. The authors also flag that the current implementation runs without workspace isolation.

Future Directions

1. Causal isolation of context management. The paper reports near-zero correlation between compression count and score but explicitly leaves causal isolation to future work.
2. Validation beyond AgentIF-OneDay. The authors state that their findings are specific to this benchmark and that broader generalization requires additional benchmarks.
3. Adaptive module selection. Since simpler variants beat FULL on a non-trivial subset of tasks, deciding per-task whether to enable decomposition, verification, or both could improve the score-versus-cost tradeoff.
4. Workspace isolation and security. The current implementation runs without workspace isolation, which the authors flag as a security consideration in Appendix E.

Target Audience

Researchers and engineers working on LLM agents, agent harness design, long-horizon task execution, and context/memory management. It is also useful for practitioners deploying agents in production who need to weigh execution cost against deliverable quality, and for benchmark designers interested in transferability analysis across backend models.

Authors’ abstract

LLM agents are increasingly applied to open-ended everyday requests that span work, study, and life. These tasks are long-horizon, cross-environment, and multimodal, forcing the agent to preserve goals and constraints across many steps while navigating heterogeneous tools and attachments. While prior work has addressed individual failure modes such as goals drift, states loss, and context overflow, whether a single harness can manage them jointly and remain effective across backends has received less study. We present OneDayAgent, a long-horizon harness for autonomous agents. OneDayAgent turns an open-ended request into a managed execution process that decomposes tasks into bounded subtasks, maintains execution memory under context pressure, and verifies and repairs the final deliverable. We evaluate OneDayAgent on AgentIF-OneDay across 104 tasks. With the GLM-5.2 backend, OneDayAgent sets a new state of the art with an overall score of 0.821. The same harness runs across five backend LLMs from three model families, indicating the harness generalizes across backends without tuning, even as different models induce distinct execution styles under the same workflow.