An Open and Reproducible Deep Research Agent for Long-Form Question Answering

AI summary

Overview

Research area: Natural Language Processing — retrieval-augmented generation (RAG), deep research agents, and long-form question answering.

Technical level: Advanced. The paper assumes familiarity with large language models, preference optimization, retrieval pipelines, and LLM-as-a-judge evaluation.

Scope: The paper reports a winning system from the text-to-text track of the MMU-RAG competition at NeurIPS 2025, describing its architecture, synthetic preference-data pipeline, DPO tuning, and LLM-based evaluation results.

What This Paper Is About

Commercial "deep research" systems can iteratively search the web, reason, and synthesize long-form answers, but they are proprietary and hard to study or reproduce. The authors build an open, reproducible deep research agent by pairing an open-source LLM with an open web search API, then improve its answers through preference tuning judged by an LLM on clarity, insightfulness, and factuality.

Key Contributions

1. An open and reproducible deep research system for long-form question answering, built on an open-source LLM and an open web search API, released at https://github.com/efficient-deep-research/efficient-deep-research.
2. A search tool that extends a search API with a reranking module (Qwen3-Reranker-0.6B) and a summarization module (Qwen3-Next-80B-A3B-Thinking) that produces citation-marked summaries of the top reranked documents.
3. A synthetic preference-data pipeline that collects 1,000 training questions (500 from Researchy Questions, 200 from Natural Questions, 300 from the competition's official validation set), generates 20 sampled answers per question, scores them with OpenAI's o3-mini across clarity, insightfulness, and factuality, and builds DPO pairs by score gap and number of search queries.
4. Empirical evidence that DPO tuning of the research agent improves all three LLM-judged metrics over an untuned baseline, and the resulting system received the Best Static Evaluation award in the open-source category of the text-to-text track.

Main Findings

• Consistent improvement across all metrics: The tuned model (θ = 0.3) outperformed the untuned Vanilla model on clarity (8.18 vs. 6.71), insightfulness (7.50 vs. 6.52), and factuality (44.3 vs. 43.4).
• Largest gains on clarity and insightfulness: Improvements were +1.47 for clarity and +0.98 for insightfulness, while factuality improved only marginally at +0.41. The authors attribute the smaller factuality gain partly to reliance on noisy user-click-based training signals available only for a subset of the training data.
• Behavioral changes from tuning: The tuned model issued more searches on average (search count 1.20 vs. 1.03, a +0.17 change) and showed a slight increase in citation error rate (0.09 vs. 0.06, a +0.03 change).
• Threshold choice mattered little: Models tuned with θ = 0.5 (clarity 8.17, insightfulness 7.29, factuality 44.1, search count 1.05, citation error rate 0.13) and θ = 0.7 (clarity 8.30, insightfulness 7.33, factuality 44.2, search count 1.23, citation error rate 0.11) were all consistently better than Vanilla, with only marginal differences among them. The θ = 0.3 model was selected for competition submission.
• Preference datasets: Setting the score-gap threshold to θ ∈ {0.3, 0.5, 0.7} yielded 983, 828, and 341 preference pairs respectively, corresponding to 56, 47, and 20 training steps for one epoch each.

Methodology in Plain English

The system has two parts: a search tool and a research agent. The search tool takes a query, retrieves the top-K documents from an open search API built on ClueWeb22 (the ClueWeb22-A category covering two billion pages), reranks them with Qwen3-Reranker-0.6B, keeps the top-N, and summarizes them with Qwen3-Next-80B-A3B-Thinking while inserting citation markers pointing to source documents. The research agent, also based on Qwen3-Next-80B-A3B-Thinking, reads the question plus prior summaries, reasons about whether more searching is needed, issues new queries, and finally writes a long-form cited answer; it can perform up to 5 searches.

To train the agent, the authors first collected 1,000 questions from three sources and generated 20 sampled answers per question with the untuned model. They then used OpenAI's o3-mini as a judge to score answers on clarity, insightfulness, and factuality (factuality only on Researchy Questions, since Natural Questions and the validation set lack the user-clicked URLs needed for key points). Each metric's normalized score was summed into a final preference score. For each question, the highest-scoring answer became the "chosen" response and the lowest became the "rejected" one; ties were broken by preferring the answer that used more search queries. Answers with formatting errors were filtered out, and a minimum score-gap threshold θ kept only pairs with a meaningful difference.

DPO training used LoRA with α = 16 and rank = 16, and DPO β = 0.5, for one epoch on each dataset, optimizing over both the reasoning chain and the final answer while masking the tokens returned by the search tool. Evaluation sampled 100 queries from Researchy Questions not used in training, generated one answer per query, and scored them with the same LLM-as-a-judge procedure. Generation hyperparameters were temperature 0.6, top-P 0.95, top-K 20, and max tokens 20,480 for the agent; search API top-K 100 (raised to 300 for the competition evaluation), reranker top-N 10, and summarizer temperature 0.6, top-P 0.95, max tokens 8,192.

Why This Matters

Impact on research: The work shows that long-form deep research capability can be reproduced with openly available components — an open-source LLM, an open search API over ClueWeb22, and synthetic LLM-judged preference data — rather than remaining locked inside proprietary systems. It also provides a concrete, measurable recipe for extending DPO to multi-aspect, long-form answer quality.

Real-world applications:

• Research assistants that compile cited, in-depth reports on open-domain questions.
• Enterprise knowledge work, where analysts need sourced multi-step answers rather than short snippets.
• Educational or journalistic drafting tools that require traceable citations back to source web pages.
• Any retrieval-augmented product where answer clarity, depth, and factuality must be traded off and measured.

Industry relevance: The system won the text-to-text track of the MMU-RAG competition at NeurIPS 2025 and received the Best Static Evaluation award in the open-source category, indicating that its combination of reranking, summarization, and preference tuning is competitive under real-world evaluation conditions, at least in the static evaluation setting reported here.

Future Directions

• Reducing the citation error rate, which rose from 0.06 to 0.09 after tuning even as answer quality improved.
• Strengthening factuality training, which the authors describe as harder because it depends on noisy user-click-based signals available only for part of the training data; the observed gain was only +0.41.
• Testing whether factuality improves when key points can be extracted for all training questions rather than only those from Researchy Questions.
• Exploring whether larger or better-filtered preference datasets, or thresholds beyond the tested θ ∈ {0.3, 0.5, 0.7}, yield further gains beyond the marginal differences observed among the three tuned models.

Target Audience

Researchers and engineers building retrieval-augmented or agentic LLM systems for open-domain long-form question answering, particularly those interested in reproducible deep research pipelines, synthetic preference-data construction, and multi-aspect LLM-as-a-judge evaluation. Readers who want to replicate or extend a competition-winning system using open models and open search infrastructure will benefit most.

Authors’ abstract

We present an open deep research system for long-form question answering, selected as a winning system in the text-to-text track of the MMU-RAG competition at NeurIPS 2025. The system combines an open-source large language model (LLM) with an open web search API to perform iterative retrieval, reasoning, and synthesis in real-world open-domain settings. To enhance reasoning quality, we apply preference tuning based on LLM-as-a-judge feedback that evaluates multiple aspects, including clarity, insightfulness, and factuality. Our experimental results show that the proposed method consistently improves answer quality across all three aspects. Our source code is publicly available at https://github.com/efficient-deep-research/efficient-deep-research.