HybridResearcher: Bridging the Structured-Unstructured Boundary for Deep Research Agents

AI summary

Overview

Research area: LLM tool-using agents for "deep research" — specifically, reasoning that must cross between structured data (SQL databases) and unstructured data (the open web).

Technical level: Advanced. The paper assumes familiarity with supervised fine-tuning, reinforcement learning for LLMs (CISPO), agent tool-calling harnesses, and retrieval corpora. The prose is clear, but the methodology assumes a research background.

Scope: A complete training recipe — data synthesis, supervised fine-tuning, and reinforcement learning — that produces a 4B-parameter agent, HybridResearcher-4B, capable of carrying evidence from a database into a web search and vice versa.

What This Paper Is About

LLM agents have gotten good at gathering evidence from one kind of source at a time (either querying a database or searching the web), but real analytical questions often require both in a single reasoning chain — for example, computing a value from an internal database and then using that value to find something on the web. Existing benchmarks such as HybridDeepResearch show that even frontier agents fail at this, yet no prior work provides training data, a training framework, or a trained model for the capability. The paper's goal is to teach a small model to perform this cross-source "bridge" reasoning without losing its existing single-source skills.

Key Contributions

1. A synthesis framework that extends validated single-source tasks across the structured/unstructured boundary. Rather than inventing hybrid questions from scratch, each training instance starts from an existing text-to-SQL task or web-search task, and the missing source is fabricated around a "bridge entity" drawn from the seed's own output, so the seed's difficulty is inherited rather than re-established.

2. Construction gates that enforce genuine cross-source interdependence. Items are retained only when deterministic and model-based checks prove that neither source alone suffices — for example, the bridge entity must appear in the SQL output but not in the question, and the constraint intersection must be a singleton.

3. A full training recipe (SFT then RL) with two stabilization mechanisms. Reinforcement learning uses a five-vote LLM-judge majority for reward and "hard-gating" rules that terminate and zero-reward degenerate rollouts (no tool call, malformed arguments, nonexistent tools, duplicated queries).

4. HybridResearcher-4B, a 4B model that improves over its Qwen3.5-4B base by 30.0 points pass@8 on HybridDeepResearch, surpasses Qwen3.5-397B-A17B by 8.3 points, and also improves on held-out single-source benchmarks.

Main Findings

• Single-source data does not compose into hybrid skill. Fine-tuning Qwen3.5-4B on SQL-only, search-only, or the union of both failed to help on the 120-question hard subset of HybridDeepResearch: SQL-only reached 31.67 pass@8 versus the base model's 30.83, while union and search-only fell to 29.17 and 15.83.

• Naive hybrid synthesis invites shortcuts. Among teacher trajectories accepted by standard correctness filters, 79.5% never executed a substantive SQL query — the model answered from parametric memory and skipped the cross-source hop entirely.

• Hybrid training produces large gains on the hybrid benchmark. Pass@8 rose from 30.83% (base) to 59.17% (SFT) to 60.83% (RL). Avg@8 rose from 7.71% to 23.75% to 27.19%. The largest RL gain was on the strictly directional S2SQL tasks, where avg@8 went from 26.88% to 31.56%.

• The 4B model is competitive with much larger systems. On HybridDeepResearch it reports 60.83% pass@8 and 27.19% avg@8, versus Qwen3.5-397B-A17B-FP8 at 52.50% / 26.77%, GLM-5.2-FP4 at 63.33% / 38.54%, and Claude Sonnet 4.6 at 65.83% / 37.29%.

• No catastrophic forgetting; single-source skills improve. BrowseComp pass@8 went from 54.03% to 62.24% (RL) and BrowseComp-Plus from 60.84% to 73.13%; Spider 2.0-lite execution accuracy went from 59.26% to 65.19% after SFT and 62.96% after RL.

• SFT teaches reading, RL teaches precision. The base model issued 38.6 searches per rollout but made only 0.29 page visits per search; after SFT this became 22.1 searches and 0.50 visits per search, and after RL 18.7 searches and 0.88 visits per search. RL also cut exact query repeats by 63%, raised quoted-phrase use to 48.8%, and doubled operator use, while slightly reducing turns (44.7 to 42.0).

• Curated RL data beats scale. The filtered 0.7K dataset reached 27.2% overall avg@8 versus 21.0% for the full 21K candidate pool; 71% of rollout groups in the curated set provided sufficient reward contrast, versus 38% in the uncurated pool.

• Judge voting and hard-gating each help. Five-vote majority judging raised avg@8 from 25.2% to 27.2%; a single judge disagreed with the consensus in 8.2% of cases (12.2% for S2SQL, 1.2% for SQL2S). Removing hard-gating lowered avg@8 from 27.2% to 25.6%, raised average tool calls from 55 to 76 and repeated-call frequency from 11% to 49%, and raised the share of questions with no valid answer from 0.0% to 10.7%.

• Remaining bottlenecks. On S2SQL, the required bridge entity still failed to surface in 22.2% of rollouts, and 82.0% of RL's wrong answers (versus 48.7% of its correct ones) cited time budget, effort already spent, or a guess.

Methodology in Plain English

The team started from the observation that you cannot simply mix SQL training data and web-search training data and expect the model to learn to hand off results between them — their own experiment with SQL-only, search-only, and union training confirmed this. They also found that hybrid questions generated naively get "solved" by models that recall the answer from memory instead of actually using both tools.

Their fix is a two-sided construction process that always starts with a task that is already known to be hard:

• From a SQL task: they pick a cell from the query's output that does not appear in the question, so the model must run the SQL to discover it. They then invent fictional facts about that entity (using made-up but pronounceable names) and plant fake documents describing them. The agent must run the SQL, search for each invented fact, and find the single "hub" that satisfies all constraints; every near-miss distractor satisfies all but one.

• From a web-search task: they build a synthetic SQLite database around an entity from the search question, designing the schema so that the answer cannot be pre-aggregated and requires real SQL operations such as grouped aggregation or ranking. Depending on direction, the entity is the SQL answer the agent must discover before searching (SQL2S), or the search answer becomes a required parameter in the SQL query (S2SQL).

Every item passes a battery of gates — deterministic ones such as "the answer is not in the question" and "the constraint intersection is a singleton," plus model-based ones such as a mask test proving the answer cannot be recovered from the distractor documents. Failing items are discarded.

Training then happens in two stages. Supervised fine-tuning uses correct trajectories sampled from large teachers (Qwen3.5-397B-A17B-FP8 or DeepSeek-V4-Flash) plus directly used existing data. Reinforcement learning uses the 21K synthesized tasks that remained unsolved, filtered down to 0.7K high-signal items balanced 1:1 across source and direction, optimized with CISPO and a five-vote LLM judge for the reward. Evaluation runs in a reproducible offline environment built around a frozen local corpus and retriever.

Why This Matters

Impact on research: The paper reframes the hybrid-source problem as a training-data problem rather than a scale problem. Its central claim — that the bridge itself must be explicitly taught, because component capabilities do not automatically compose — is a general lesson for agent training beyond SQL and web search. The negative result that naive union training does not help, and that 79.5% of "correct" trajectories are shortcuts, is directly actionable for anyone synthesizing agent data.

Real-world applications:

• Financial auditing and compliance: crossing internal transaction databases with public filings, as the paper cites.
• Supply-chain risk assessment: merging inventory data in SQL with global news coverage.
• Enterprise analytics: answering a single business question that requires both a computed database result and external web evidence.
• Any small-model deployment where cost matters: the result shows a 4B model can handle workflows previously assumed to need frontier-scale systems.

Industry relevance: The method reduces reliance on very large models at inference time, and the RL findings (curated 0.7K beats an unfiltered 21K pool; hard-gating cuts tool calls from 76 to 55 per question) translate directly into cheaper and more reliable production agent behavior. The structural recipe is stated to generalize to other heterogeneous pairings such as internal documents, code repositories, or domain-specific databases.

Future Directions

• Extending beyond the SQL–web pairing. The authors frame their recipe as a blueprint that needs only a validated single-source task to seed from and an entity that can bridge across sources; testing it on documents, code repositories, or other domain databases is an open step.

• Teaching the agent when to search more versus commit. 82.0% of RL's wrong answers cited time budget, effort, or a guess, versus 48.7% of correct ones, suggesting that learning to refrain from guessing under self-acknowledged uncertainty is a promising direction.

• Improving retrieval so the bridge entity reliably surfaces. On S2SQL the required entity still failed to appear in 22.2% of rollouts, which bounds achievable accuracy regardless of reasoning quality.

• Reducing judge noise further. With a single judge disagreeing with the consensus in 8.2% of cases and 12.2% for S2SQL, better reward estimation for semantically nuanced answers remains an open problem.

Target Audience

Researchers and engineers working on LLM agents, agentic reinforcement learning, or synthetic training-data construction — particularly those building deep-research or tool-use systems and those interested in transferring frontier capabilities to small, deployable models. It is also useful for practitioners evaluating whether hybrid reasoning requires scale or can be taught, and for anyone designing evaluation or data-quality gates for multi-tool agent trajectories. Readers without a background in RL fine-tuning will find the problem framing accessible but the training section dense.

Authors’ abstract

While LLM agents are increasingly capable of gathering and synthesizing evidence from a single homogeneous source, such as structured tables or unstructured data, real-world analytical questions frequently span heterogeneous sources. For example, answering one enterprise question may require both a result computed over an internal database and evidence found on the web. Recent evaluations on benchmarks like HybridDeepResearch show that even frontier agents struggle when evidence must cross this boundary. While these benchmarks have exposed the limitation, methodologies to explicitly teach cross-source reasoning remain absent. To address this, we introduce HybridResearcher, a 4B agent trained to bridge evidence across the structured and unstructured boundary. To build it, we synthesize cross-source bridge trajectories by starting with validated single-source tasks-such as existing text-to-SQL or web search queries-and fabricating the missing source around a bridge entity drawn from the seed's own output. We retain a sample only when strict construction gates confirm that neither side alone suffices, ensuring the data requires true cross-source interdependence. Finally, we train on this curated dataset using supervised fine-tuning followed by reinforcement learning. Evaluated on the existing HybridDeepResearch benchmark, HybridResearcher improves over our base model, Qwen3.5-4B, by 30.0 points pass@8 and surpasses the far larger Qwen3.5-397B-A17B by 8.3 points, while also improving over its base model on established, held-out single-source benchmarks (BrowseComp and BrowseComp-Plus), demonstrating that this complex cross-source reasoning can be taught directly to small models without triggering catastrophic forgetting.