Research Agents and Evidence Synthesis

Answer-first research, Evidence-first research, and a Parallel research team differ in when the conclusion is allowed to exist. Answer-first writes it and then looks for support. Evidence-first is not allowed to write it yet. The risk that separates them is one sentence long. Citations can decorate a claim without supporting it. A reference collected after the conclusion was already written sits beside the sentence without ever having tested it.

That risk has a measured rate. In 2023 human evaluators went through the output of four commercial generative search engines — Bing Chat, NeevaAI, perplexity.ai and YouChat. On average only 51.5% of generated sentences were fully supported by their citations. Only 74.5% of citations supported the sentence they were attached to. Roughly one generated sentence in two was not fully supported. One citation in four did not support the sentence beside it. None of those systems was choosing between the three strategies above in any visible way. The output looked the same either way.

So the check on all three is the same. Take a claim out of the report and ask whether its source actually says that, on a stated date and under a stated definition.

A Research plan, an Acquisition stage, an Evidence ledger, and a Synthesis stand between the question and the report. Synthesis and Review should not share an owner, and should not share a test. The ledger is the part teams skip: one row per claim, with the source and the date beside it.

This is not a diagram invented for agents. Medicine has run it as a numbered reporting standard since 29 March 2021, the day the PRISMA 2020 statement was published. The statement describes its own shape in its summary points: “The PRISMA 2020 statement consists of a 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and revised flow diagrams for original and updated reviews.”

Twenty-seven items. The ones agents lose first are the ledger columns: the eligibility criteria, the full search strategy for every database, and the date each source was last searched. A research agent that cannot produce those three has not skipped a formality. It has skipped the part of the record that lets a second person redo the search and find out whether the answer still holds.

Research agents combine question decomposition, search, retrieval, browsing, data extraction, calculation, synthesis, and citation. That list is one pipeline: break the question up, go and find things, read them, pull the numbers out, do the arithmetic, write it up, and say where each claim came from. Their value comes from widening and organizing evidence. Not from making generated prose sound authoritative.

The task contract should define source classes, time boundaries, claim units, uncertainty, and what counts as sufficient support. It also needs a name for the way a cited claim fails, and there is a published one. A preregistered set of over 200 legal queries went to three RAG-based commercial legal research tools and to GPT-4. The researchers who ran them, Magesh and colleagues, named the state that sits between right and wrong: “A response is misgrounded if key factual propositions are cited but misinterpret the source or reference an inapplicable source.” Misgrounded is exactly what a bibliography conceals. The citation is present, the source is real, and it does not carry the proposition.

The rates they measured appeared in the Journal of Empirical Legal Studies in 2025. Lexis+ AI, Westlaw AI-Assisted Research and Ask Practical Law AI each hallucinate between 17% and 33% of the time. Lexis+ AI was accurate on 65% of queries, Westlaw AI-Assisted Research on 42%. These are retrieval-augmented products sold into a profession where the citation is the deliverable.

High-stakes conclusions still need qualified review. Qualified means somebody who could have written the report themselves.

Thirty short medical papers, each with at least three references: that was the order given to ChatGPT-3.5 in 2023. The four researchers who placed it then looked up every one of the 115 references in Medline, Google Scholar and the DOAJ. Their result, in the abstract: “Among these references, 47% were fabricated, 46% were authentic but inaccurate, and only 7% were authentic and accurate.”

Of the 115 references, 7% were both authentic and accurate. The 46% that were authentic but inaccurate are the dangerous half. The paper exists. The journal exists. A reader who recognises the name nods and moves on, and the paper does not say the thing it was cited for. The papers also carried an incorrect PMID in 93% of cases — an identifier whose only function is to be looked up, wrong in more than nine papers out of ten. Read as prose, those bibliographies looked like coverage. Counted, they were a long list resting on almost nothing.

Grade a research agent on claims rather than on reports, and build that evaluation for one real research workflow. For each sampled claim, open the cited source and mark whether it states that claim, states something weaker, or does not mention it at all. The graded set is your evidence that material claims have source-linked evidence with dates and definitions. Grade twenty claims, not twenty reports.

The two numbers that grading produces already have names. ALCE, the first automatic benchmark for evaluating LLM citations, scores systems on fluency, correctness and citation quality. It splits that last one into citation recall and citation precision — the claim-level pair your ledger is implicitly computing. The team that built it reported in 2023: “Our experiments with state-of-the-art LLMs and novel prompting strategies show that current systems have considerable room for improvement—For example, on the ELI5 dataset, even the best models lack complete citation support 50% of the time.”

Make the claim units short enough to check against a reference answer, the way OpenAI's BrowseComp does. Published in 2025, it is 1,266 questions requiring persistent web navigation. Human trainers solved only 29.2% of the 1,255 questions they attempted, and gave up after two hours on 70.8% of them. GPT-4o scored 0.6%. GPT-4o with browsing, 1.9%. OpenAI Deep Research, 51.5%.

Then read the calibration column. It is the one a research agent lives or dies by. The paper states it plainly: “As shown in Table 3, we found models with browsing capabilities such as GPT-4o w/ browsing and Deep Research exhibit higher calibration error, suggesting that access to web tools may increase the model’s confidence in incorrect answers.” Deep Research posted the worst calibration error in that table, 91%. The strongest browser in the benchmark was also the most confidently wrong one. Injecting contradictory evidence and auditing calculations are how you find that out before the report is written, not after.

A research agent should be allowed to conclude that the evidence is insufficient. Forced completeness is a major source of fabricated certainty.

The call belongs to whoever reads the report and has to act on it. They decide whether the references under a claim are load-bearing or ornamental. A research agent earns that trust when every claim that matters carries a link to the source it came from, the date that source speaks to, and the definition it uses for the terms in question.

Grounding is defined, and somebody has to keep it up to date. Retrieval-augmented generation pairs parametric with non-parametric memory; Lewis and colleagues set it out in 2020. AWS describes the operational half: the model consults an authoritative knowledge base outside its training data before it answers. That same documentation says the documents and their embeddings must be updated asynchronously.

Staleness has been benchmarked, and so has the related failure of accepting a question's premise. FreshQA is a dynamic benchmark of 600 questions that mixes fast-changing items with false-premise items, and its builders collected more than 50,000 human judgments. Their result, reported in 2024: “Through human evaluations involving more than 50K judgments, we shed light on limitations of these models and demonstrate significant room for improvement: for instance, all models (regardless of model size) struggle on questions that involve fast-changing knowledge and false premises.”

Regardless of model size is the part to keep. A larger model does not refresh an index, and it does not debunk a premise it was handed. Both failures produce the same artefact: an answer built from a snapshot, delivered at the confidence of a fact. The refresh is a job somebody owns, on a schedule somebody wrote down.