ResearStudio: A Human-Intervenable Framework for Building Controllable Deep-Research Agents

AI summary

Overview

Research area: Autonomous LLM agents, specifically "deep research" (DR) agents, and human–AI collaborative interfaces for agentic systems. The work sits at the intersection of agent architecture design, multi-agent orchestration, and human-computer interaction.

Technical level: Advanced. The paper assumes familiarity with LLM agent architectures (planner–executor hierarchies, tool calling, MCP), benchmark evaluation on GAIA, and system-level concepts such as sandboxing, event-driven protocols, and frontend/backend streaming.

Scope in one sentence: The paper presents ResearStudio, an open-source framework that lets a user pause, edit, and resume a deep-research agent mid-execution while still achieving state-of-the-art autonomous results on the GAIA benchmark.

What This Paper Is About

Current deep-research agents operate in a "fire-and-forget" mode: once a task starts, the user becomes a passive observer with no channel to fix errors, redirect strategy, or inject domain knowledge while the agent runs. The authors argue this rigid one-directional pipeline causes errors, wasted compute, and diminished trust. Their goal is a framework where strong autonomous performance and fine-grained human control coexist, realized through a design they call the Collaborative Workshop.

Key Contributions

1. Formalizing the Collaborative Workshop. The paper defines a shared, persistent, interactive digital interface characterized by three properties: Transparency (all plans, intermediate artifacts, and actions are visible), Symmetrical Control (humans and AI hold equivalent authority to modify any element), and Dynamic Role Fluidity (control shifts between AI-led and human-led workflows).

2. Releasing ResearStudio as an open-source deep-research agent. It provides real-time bidirectional collaboration and live plan editing with the help of a search agent. Code, protocol, and evaluation scripts are released at github.com/ResearAI/ResearStudio, with a live demo at ai-researcher.net:3000, and the authors state they support development of DeepScientist.

3. Empirical evidence that collaboration does not cost capability. The authors report that in fully autonomous mode ResearStudio achieves state-of-the-art results on the GAIA benchmark, surpassing systems such as OpenAI's DeepResearch and Manus.

4. A documented intervenable tool substrate and safety posture. Every tool (document processing, search, code, terminal) is exposed through the same live interface that streams its inputs and outputs so users can override or refine any step. Appendix B adds a safety design based on per-task sandboxing, mediated tool calls through MCP, "Plan-as-Document" review checkpoints, and input classifiers that filter disallowed user requests.

Main Findings

• GAIA results as stated in the prose: The authors report a leading average score of 70.91% on the GAIA validation set, and state that ResearStudio achieves the highest scores on Level 2 (69.77%) and the highly challenging Level 3 (61.54%) tasks. The abstract additionally claims state-of-the-art on GAIA and says ResearStudio surpasses systems like OpenAI's DeepResearch and Manus.

• GAIA results as stated in Table 2: Table 2 lists ResearStudio at 75.47 (Level-1), 70.93 (Level-2), 53.85 (Level-3), with an average of 69.70. These numbers do not match the prose figures above, and the paper does not explain the difference. The table attributes this row to "ResearStudio (Zhou et al., 2025)."

• Baseline comparison (Table 2): ODR-smolagents averages 55.15, AutoAgent 55.15, OWL 69.09, A-World 69.70, and OpenAI-DeepResearch 67.36. Per level, OpenAI-DeepResearch is listed at Level-1 74.29, Level-2 69.06, Level-3 47.60; A-World at 86.79, 69.77, 34.62.

• Test set numbers are not reported numerically. Although the paper states results are achieved "across both validation and test sets," only validation figures appear in Table 2.

• Evaluation protocol: All experiments were run in fully autonomous mode with no human intervention from task initiation to completion. The metric is Exact Match (EM), where a prediction counts as correct only if it exactly matches the reference answer after normalizing for case, punctuation, and articles.

• Feature comparison (Table 1): ResearStudio is reported as the only system among the compared set with all four features — Online Search, OpenSource Framework, Pre-research Intervention, and Real-time Content Adjustment. OpenAI DeepResearch and Kimi-Researcher are marked as lacking an open-source framework, pre-research intervention, and real-time content adjustment; OpenAI/Google Canvas is marked as supporting none of the four explicitly, with "Text editing Only" for content adjustment; Google DeepResearch and Grok DeepSearch are marked as supporting Online Search and Pre-research Intervention only; Skywork Agent is marked "Slide/Doc Only" for content adjustment.

• Operational parameters (Table 3): Average task runtime on GAIA of roughly 20 minutes (simpler or specific successful cases under 10 minutes, and a complex GAIA Level-3 task roughly 10 minutes in Section 5), maximum concurrent workers 50, maximum interaction rounds 30, average steps per task ~25, and typical final workspace size ~100 MB. The prose in Section 5 instead says the framework supports "up to 50 interaction rounds," which differs from the table's value of 30.

• Case study — successful computational puzzle: A GAIA Level-3 task involving two 12-digit numbers, column transpositions, and a weighted-sum checksum validation was solved in approximately four minutes over 16 discrete steps, by writing and running a solve_puzzle.py script rather than reasoning in-prompt.

• Case study — failure on contextual ambiguity: On a GAIA Level-3 task asking for the volume of a Freon-12 container at the bottom of the Mariana Trench, the agent's web search returned both the trench's near-freezing ambient temperature (1–4°C) and hydrothermal vent temperatures (400°C). The agent selected 400°C, leading to a wrong density and an answer orders of magnitude off. The authors present this as the clearest demonstration of why the Collaborative Workshop matters: a user could pause, edit the TODO.md or a note file to specify "use ambient temperature of 4°C," and resume.

• Models used: Planner powered by gpt-4.1; Executor by o4-mini for datasets excluding GAIA; image processing by gpt-4o; video agent by gemini-2.5-pro; audio agent by Assembly AI; and o3 selected as the executor for the GAIA benchmark. Note that Section 4 describes the Executor as o4-mini while Section 3.3 specifies o3 for GAIA.

• Search and tooling stack: A self-hosted searxng metasearch combined with Crawl4AI page fetches, with results re-ranked by contextual similarity. Browser automation is deliberately omitted by default (re-enableable with a toggle) because the authors observed LLM planners overuse it, incurring latency and yielding little structured data.

Methodology in Plain English

The system is organized into three layers. L-1 is an MCP Toolbox: a set of standardized tool services the agent can call. L-2 is the Agent Core, split into a Planner that decides what to do and an Executor that carries out each step using L-1 tools. L-3 is the WebPage the human sees. A central communication protocol links all three.

The key design move is the "plan-as-document": the Planner writes every step into a live TODO.md file that the user can read and edit. A fast communication layer streams each action, file change, and tool call to the web interface, so the human sees what the agent is doing as it happens. At any moment the user can pause the run, edit the plan or code, run custom terminal commands, and resume — switching between AI-led, human-assisted and human-led, AI-assisted modes.

At the machine level, the Model-Context Protocol (implemented with fastmcp) turns the Executor's tool calls into reliable JSON-based function calls. At the human level, an event-driven protocol maintains a long-lived frontend–backend connection. User actions become API calls tagged with a task ID: changing a file is a POST with new content that updates the Workspace and notifies the Executor; pausing is a request that stalls all backend LLM calls until resumed. Large file contents are lazy-loaded on click.

The tool suite covers document processing across formats (jpg, png, mp3, pptx, xlsx, csv, zip, txt, json, xml, docx, mov, pdf, …) via modality-specific extractors (VLM captions for images, ASR transcripts for audio, slide-wise markdown for .pptx, row-wise CSV for spreadsheets); search with human accept/reject/deeper-crawl decisions; and a sandboxed code toolkit where every script is rendered in the UI before execution and a diff viewer enables one-click rollback or branching.

Evaluation was done on the GAIA benchmark in fully autonomous mode, using Exact Match as the metric, with the model assignments described above.

Why This Matters

Impact on research. The paper reframes human control and autonomous capability as complementary rather than competing, and it supplies an open-source artifact (code, protocol, evaluation scripts) that other groups can build on. The failure case study gives a concrete, mechanistic account of how a deep-research agent fails — conflicting web evidence, plausible-but-wrong selection, cascading error — and argues that such failures are correctable rather than fatal when the right interface exists.

Real-world applications.

• Expert-supervised research workflows where a domain specialist audits and corrects an agent's sourcing and assumptions as it works.
• Multi-step data analysis and report generation where the agent produces scripts and artifacts the user can inspect and fix rather than restart.
• Scientific or engineering tasks with domain conventions (units, ambient conditions, naming) that an LLM lacks the common sense to disambiguate.
• Deployment contexts demanding auditability, where the "Conversation & Activities" log and "Session History" turn each run into a reusable, inspectable template.

Industry relevance. The comparison table positions the work directly against OpenAI DeepResearch, Google DeepResearch, Kimi-Researcher, Grok DeepSearch, Skywork Agent, and OpenAI/Google Canvas, arguing that commercial systems either lack open-source availability, allow no pre-research intervention, or restrict real-time adjustment to single-file text, slide, or document editing. For teams building agent products, the paper's claim is that a controllable, open workshop can be competitive on raw benchmark performance rather than trading capability for oversight.

Future Directions

• AI-powered alerts for semi-autonomous intervention. The authors plan mechanisms that flag potential errors or logical inconsistencies for human review, to reduce the cognitive load of continuous monitoring.
• Formal HCI studies. They call for studies measuring task completion times with and without intervention, error correction rates, and user satisfaction scores, to quantify the practical benefit of the human-in-the-loop features. The paper explicitly states that its current experiments do not yet formally quantify these benefits.
• Adversarial stress testing of safety measures. The authors describe current safety as primarily architectural (sandboxing, mediated tool calls, Plan-as-Document review, input classifiers) and state it has not been rigorously stress-tested against active adversarial attacks.
• Broadening accessibility beyond experts. The paper acknowledges that effectiveness in collaborative mode depends heavily on user domain expertise, positioning the system as a tool for experts rather than novices, and identifies reducing this dependency as future work.

Target Audience

Agent framework developers and systems engineers who want to build or extend intervenable agent architectures; HCI and human-AI collaboration researchers interested in control, transparency, and role fluidity in agentic workflows; and practitioners deploying deep-research agents in expert domains where mid-run correction matters. Readers need comfort with LLM agent terminology, tool-calling protocols, and benchmark evaluation to get the most from the architecture sections, though the problem framing and case studies are accessible to a broader technical audience.

Authors’ abstract

Current deep-research agents run in a ''fire-and-forget'' mode: once started, they give users no way to fix errors or add expert knowledge during execution. We present ResearStudio, the first open-source framework that places real-time human control at its core. The system follows a Collaborative Workshop design. A hierarchical Planner-Executor writes every step to a live ''plan-as-document,'' a fast communication layer streams each action, file change, and tool call to a web interface. At any moment, the user can pause the run, edit the plan or code, run custom commands, and resume -- switching smoothly between AI-led, human-assisted and human-led, AI-assisted modes. In fully autonomous mode, ResearStudio achieves state-of-the-art results on the GAIA benchmark, surpassing systems like OpenAI's DeepResearch and Manus. These results show that strong automated performance and fine-grained human control can coexist. The full code, protocol, and evaluation scripts are available at https://github.com/ResearAI/ResearStudio. We will continue to update the repository to encourage further work on safe and controllable research agents. Our live demo is publicly accessible at http://ai-researcher.net:3000/. We support the development of DeepScientist, which can be accessed at https://github.com/ResearAI/DeepScientist.