Real PMIDs, Wrong Papers: Harness Design for Biomedical Claim-Verification Agents

AI summary

Overview

Research area: AI for scientific literature search — evaluation of language-model "harnesses" (the scaffolding around a model: tools, prompts, loop structure) for biomedical claim verification.

Technical level: Intermediate. The paper is short and readable, but understanding the paired statistical tests, bootstrap confidence intervals, and the retrieval-versus-reasoning error decomposition benefits from some familiarity with LLM agent evaluation.

Scope: A controlled ablation of four harness designs for biomedical fact-checking over 100 SciFact claims using the live PubMed API, with two Gemini backbones, plus a citation audit of what closed-book models actually cite.

What This Paper Is About

Language agents that search the biomedical literature are becoming common components of scientific assistants, but there is almost no controlled evidence about which parts of their design actually matter. The author holds the model, prompt format, and output schema fixed and varies only the harness — closed-book answering, one-shot retrieval, a ReAct-style agent that writes its own queries, or an oracle given the gold abstracts — to isolate where performance is won and lost. A second thread asks whether the papers such agents cite are faithful to what they actually read.

Key Contributions

1. A small, fully reproducible harness-ablation protocol for literature agents, built on a public benchmark (SciFact) and a public API (live PubMed E-utilities), with code, prompts, raw model outputs, and cached tool calls released.
2. An empirical comparison of four harnesses across two backbones, reported with 95% bootstrap confidence intervals and paired (McNemar) analyses, including a decomposition into retrieval failures and reasoning failures.
3. A citation audit that distinguishes nonexistent citations from existing-but-irrelevant ones, showing that closed-book models produce PMIDs that resolve in PubMed but are almost never topically related to the claim.
4. Evidence that retrieval, not reasoning, is the dominant bottleneck, quantified as a significant gap between the best realistic harness and the oracle-abstract condition.

Main Findings

• Naive one-shot retrieval does not help, and changes how the model fails. Using the claim itself as a PubMed query retrieved a gold abstract for only 6% of evidence-bearing claims (4/64 for both backbones). Both models then abstained heavily — 59% and 70% of predictions were NEI, against 36% NEI in the gold labels — and accuracy was unchanged or lower than closed-book (53.0 vs. 55.0 and 49.0 vs. 59.0; McNemar p = 0.89 and p = 0.24). On the 4 claims where a gold abstract was retrieved, both models were correct.
• Letting the model write its own queries triples gold recall and gives the best verdicts among realistic harnesses. Agentic search reached 19–20% gold recall and the highest accuracy of the non-oracle conditions (62.0 for both backbones), with macro-F1 of 60.3 and 61.1. Its queries were short, entity-centric reformulations, and it could repair claim misspellings ("Chenodeosycholic" to "chenodeoxycholic") that silently break literal retrieval. These gains are not statistically significant at N = 100: agent vs. single-shot, 24 vs. 15 discordant claims (p = 0.20) and 29 vs. 16 (p = 0.07); agent vs. closed-book, 13 vs. 6 (p = 0.17) and 11 vs. 8 (p = 0.65). The direction is consistent across both backbones.
• Agents under-use their search budget. With a budget of 4 searches, agents issued 1.42 and 1.70 queries per claim, answering after a single search on 71% and 55% of claims; only 3% and 6% used the full budget. Among evidence-bearing claims where no gold abstract was ever seen, the agent still answered Support/Contradict correctly 75% of the time — but it abstained less on true NEI claims (NEI recall 33% and 42%, versus 72% and 81% for single-shot).
• Retrieval, not reasoning, is the main bottleneck. Given gold abstracts, the same models reached 86.0% and 79.0% accuracy (macro-F1 85.8 and 78.6), 24 and 17 points above the best realistic harness. Unlike the gaps among realistic harnesses, this one is highly significant (agent vs. oracle, 5 vs. 29 discordant claims, p < 10⁻⁴; 9 vs. 26, p = 0.006). The oracle also restored calibrated abstention (NEI recall 78% and 86%).
• Closed-book citations are real papers that are almost never relevant. Closed-book models cited PMIDs for 97 and 80 of the 100 claims. Almost all cited PMIDs exist in PubMed (186/190 and 141/145; 327/335 overall), so existence checks would pass them. None matched a SciFact gold paper, and the cross-family judge rated only 1 of 186 and 2 of 141 as relevant (3 of 327 overall). Cited examples include "[The proper eyeglasses for children]" for a claim about TNF-α/IL-1 and IL-6/IL-10 regulation. The judge was validated on controls: 21/25 gold abstracts labelled relevant, 0/25 mismatched gold abstracts labelled relevant.
• Grounding sharply changes citation quality. The same judge rated 66% (141/215) and 70% (128/184) of agent-cited PMIDs as relevant, and 77% and 84% of agent verdicts that cite anything cite at least one relevant paper. Agents almost never cite outside what they retrieved (0% and 2.6% of citing claims).
• Closed-book models are overconfident. They predicted NEI for only 4% and 9% of claims, recovering 6% and 17% of true NEI claims, so their competitive accuracy rests entirely on Support/Contradict claims.

Methodology in Plain English

The task is biomedical claim verification on SciFact: each claim is labelled Support, Contradict, or Not Enough Info with respect to its cited abstracts. SciFact test labels are hidden, so the author uses the labelled dev split (300 claims) and, for computational budget reasons, evaluates a fixed random sample of N = 100 claims (seed 0) shared by all conditions.

Four harnesses are compared, all sharing the same instructions, temperature 0, and a required JSON output of {label, cited_pmids, rationale}:

• Closed-book: the model answers from its own knowledge and may cite PMIDs it believes are relevant.
• Single-shot RAG: the claim, with stopwords removed, is issued as one PubMed query (relevance sort, top-5). Titles and abstracts truncated to 1,500 characters are shown to the model.
• Agentic search: a ReAct-style loop where the model either issues its own PubMed query or answers, with a budget of 4 searches of top-5 results each, deduplicated, and instructions about PubMed's keyword semantics.
• Oracle: the model receives the SciFact-cited gold abstracts directly, upper-bounding performance under perfect retrieval.

Two efficient frontier backbones from one provider are used: gemini-3.1-flash-lite and gemini-3.5-flash-lite. Because SciFact documents carry Semantic Scholar IDs rather than PMIDs, retrieved PubMed records are mapped to gold documents by normalized title match (exact or ratio > 0.9). Gold recall counts the fraction of Support/Contradict claims for which at least one gold abstract was retrieved. For retrieval conditions, a citation is deemed unfaithful if its PMID was never returned by the tool. For closed-book citations the author checks existence in PubMed, gold match, and topical relevance judged by a model from a different family (qwen3.8-27b), with the same judge applied to agent citations for comparison. Accuracy and macro-F1 are reported with 95% bootstrap confidence intervals from 2,000 resamples.

Why This Matters

The paper's central practical claim is that validating that a PMID exists is not a sufficient safeguard: an assistant can return real, resolvable PubMed IDs that have nothing to do with the claim, and existence-based checks will wave them through. It also reframes where engineering effort should go in literature agents — toward retrieval quality, not verdict prompting — and shows that a plausible-looking pipeline (claim-as-query retrieval) can actively hurt by converting confident errors into unhelpful abstentions.

Real-world applications:

• Scientific assistants and literature-review tools that surface citations to researchers: relevance checking against retrieved text is needed, not just identifier resolution.
• Biomedical fact-checking and evidence-synthesis workflows, including screening claims against the published literature where an incorrect citation is worse than an abstention.
• Agent evaluation and QA pipelines for vendors and labs building tool-using LLM systems, where harness choices (query formulation, search budget) are shown to be measurable design variables.
• Clinical or regulatory-adjacent information products, where the distinction between "the paper exists" and "the paper supports this claim" is the whole point of the citation.

Industry relevance: the ablation is cheap and reproducible from a released cache, which makes it directly reusable as a regression test or design checklist by teams building retrieval-augmented scientific products, and its finding that agents stop after roughly one to two of four allowed searches points to a concrete product-level tuning knob.

Future Directions

• Scaling up the evaluation. N = 100 claims per condition gives wide confidence intervals, and none of the differences among realistic harnesses is significant; the author plans to run the full dev set.
• Broadening the backbone set. Both models are small, efficient models from a single provider, so adding open-weight and biomedical-specialist backbones is an explicit next step.
• Budget-forcing strategies for agentic search. Agents used only 1.42 and 1.70 of 4 allowed searches and abstained less on true NEI claims; a minimum-evidence criterion before answering is suggested as a harness improvement.
• Better retrieval rather than better prompting. The 17–24 point gap to the oracle motivates iterative search that is actually used, citation chasing, and domain-specific retrievers. The relevance judge itself was validated on only 50 controls and comes from a single LLM, so judge robustness is also open.

Target Audience

Researchers and engineers building LLM agents over scientific literature; evaluation and benchmarking teams interested in controlled ablations of agent scaffolding; biomedical NLP researchers working on claim verification and retrieval-augmented generation; and product or policy stakeholders who need to know why "the citation resolves" is a weak guarantee for a scientific assistant. Readers wanting a compact, reproducible template for harness ablation will get the most from it; readers seeking definitive rankings among harnesses will find the sample too small, as the author states.

Authors’ abstract

Literature-searching language agents are becoming a standard component of scientific assistants for the life sciences, yet there is little controlled evidence on which parts of their harness matter. We run a controlled ablation on biomedical claim verification (SciFact) with the live PubMed API as the only tool. Holding the backbone and output format fixed, we compare four harnesses: closed-book answering, single-shot retrieval using the claim as the query, a ReAct-style agent that writes its own PubMed queries, and an oracle given the gold abstracts, each with two efficient frontier backbones. Naive single-shot retrieval finds the gold paper for only 6% of claims and turns overconfident errors into abstentions without improving accuracy. Letting the model formulate its own queries triples gold recall and yields the best verdicts (62% accuracy vs. 49-59% for the other realistic harnesses), although these differences are not significant at our sample size and agents use fewer than half of their search budget. With gold abstracts, the same models reach 79-86% accuracy, a significant 17-24 point gap showing that retrieval, not reasoning, is the main bottleneck. Finally, a citation audit shows that closed-book models cite PubMed IDs that almost always exist (327/335) but are almost never relevant: a cross-family LLM judge rates only 3 of 327 existing citations as relevant to the claim, versus 67% for citations produced by the search agent. Existence checks are therefore insufficient safeguards for scientific agents. We release code, prompts, and all cached tool calls at https://github.com/saqlain2204/real-pmids-wrong-papers.