Search and Retrieval Tools for Agents

Search tools discover candidates. Retrieval tools fetch content. Ranking chooses what enters context. None of these stages proves that a claim is current, independent, authoritative, or supported by the selected passage.

That gap has been measured on products that were sold as having closed it. Two commercial legal research tools were run through over 200 preregistered legal queries, the first preregistered evaluation of retrieval-augmented legal research. LexisNexis's Lexis+ AI answered 65 percent of the queries accurately. Thomson Reuters's Westlaw AI-Assisted Research answered 42 percent. Both hallucinated between 17 and 33 percent of the time. The study appeared in the Journal of Empirical Legal Studies in 2025.

The number is only usable because of the definition sitting behind it. A response counted as hallucinated if it was incorrect, or if it was misgrounded. Misgrounding is the retrieval failure this lesson is about, and Magesh and colleagues put it in one sentence: “A response is misgrounded if key factual propositions are cited but misinterpret the source or reference an inapplicable source.” The citation is present. The source exists and is real. It does not say what the sentence says. Nothing in the rank order tells that case apart from a correct one.

Agents should retain query, source, retrieval time, passage boundaries, and citation links so later checks can reproduce how evidence entered the run.

The reflex when an answer looks thin is to retrieve more documents. Someone measured what more actually buys. Hold the information constant. Hide one relevant document inside a retrieved stack, then move it around. Multi-document QA accuracy traces a U-shaped curve: highest when the relevant passage sits near the beginning or the end of the input, lowest when it sits in the middle. The result was published in 2024, under a title that says it — Lost in the Middle.

The two ends of the scale are the useful part. Given only the single relevant document, the oracle setting, GPT-3.5-Turbo reached 88.3 percent. Given nothing at all, it scored 56.1 percent closed-book. Retrieval is supposed to live between those two numbers. It did not always: “GPT-3.5-Turbo’s multi-document QA performance can drop by more than 20%—in the worst case, performance in 20- and 30-document settings is lower than performance without any input documents (i.e., closed-book performance; 56.1%).”

A pipeline that retrieved twenty documents, one of them correct, and placed the correct one in the middle would have been better off retrieving nothing. Retrieval count was the metric that looked like progress. Passage position was the variable that decided the answer.

Open-web search, a controlled corpus, and a structured system query differ mainly in who decides what counts as a source. On the open web the model decides while it works: high coverage, variable quality, adversarial content. In a curated corpus, whoever built it decided in advance. That buys stronger provenance, at the price of freshness and of somebody having to do the curating. In a structured query the schema decided long before either: precise fields, schema dependent, and it may omit the context that would have changed the reading. Whichever you pick, the test afterwards is the same. Every material claim maps to a retrievable evidence item, and the reader can open that item without asking the agent to explain itself.

"Adversarial content" is the bullet that sounds like a caveat and is not. Text planted in a page can act on the system that later retrieves it. That was demonstrated in 2023 against deployed systems, including Bing's GPT-4-powered Chat and code-completion engines. Greshake and colleagues named the technique Indirect Prompt Injection, and stated the mechanism: “We reveal new attack vectors, using Indirect Prompt Injection, that enable adversaries to remotely (without a direct interface) exploit LLM-integrated applications by strategically injecting prompts into data likely to be retrieved.”

That is the real difference between the three columns. A retrieved passage is not inert text that the agent reads. It is text that can act on the agent. On the open web the attacker gets to choose what is in it. A controlled corpus does not make a source true. It does decide in advance who is allowed to write into the channel the model reads from.

A query plan produces candidates. Source filtering discards most of them. Passage selection decides what actually enters context. The stages run in that order. Break the information need into claims, entities, dates, and source types. Retrieve candidates with lexical, semantic, structured, or domain-specific search. Filter on authority, recency, independence, rights, and access. Select text that directly supports or contradicts the target claim. Then record source, quote location, retrieval time, and unresolved conflicts in an evidence ledger. Passage selection and the evidence ledger need separate owners and separate tests.

The retrieval step is where teams assume the newest method wins. BEIR is the measurement that says otherwise. It assembled 18 publicly available retrieval datasets across diverse tasks and domains, then evaluated 10 state-of-the-art retrieval systems on them: lexical, sparse, dense, late-interaction and re-ranking. That was in 2021. Thakur and colleagues state the outcome in the abstract: “Our results show BM25 is a robust baseline and re-ranking and late-interaction based models on average achieve the best zero-shot performances, however, at high computational costs.” Dense and sparse retrieval models, though computationally cheaper, often underperformed other approaches out of domain.

Two consequences follow for the map above. First, the choice at candidate retrieval is an empirical question per domain, not a default. A lexical baseline you can debug may beat an embedding model you cannot, on data neither was tuned for. Second, semantic similarity ranks passages by how much they resemble the query. It does not rank them by whether the source is authoritative. It does not rank them by whether the passage contains the claim. That is the job of source filtering and passage selection, and no amount of similarity does it for you.

Set up a run whose object is evidence rather than an answer. Use a question your team has already researched by hand. Because you already know the answer, you can watch what the retrieval did to reach it. Write claim units: split the task into statements that can be supported independently. Choose source classes — primary records, official documentation, datasets, expert analysis. Diversify queries with dates, alternative terms, and contradiction-seeking searches. Record provenance: source identity, passage, timestamp, access path. Then grade each claim as supported, contradicted, uncertain, or outside scope.

Three checks turn this from a demo into a test. Trace each sentence the agent produced back to the document it came from, and ask the misgrounding question of every one: do the cited words carry the proposition, or do they merely sit near it? Count how many of those documents are independent of one another rather than copies of a common original. Then re-run the same question with the supporting passage moved from the middle of the context to the front. A pipeline whose answer changes when only the position changed has told you it is reading placement, not evidence.

The run has earned its keep the moment two confident citations turn out to lead back to the same place.

Search agents need explicit source-quality and conflict policies. Without them a capable model can produce a polished synthesis of poor evidence. A synthesis of many agreeing pages reads exactly like a synthesis of one page quoted by many sites.

The largest audit of that failure was run by people who do sourcing for a living. Journalists from 22 public service media organisations, across 18 countries and 14 languages, evaluated AI assistant answers about the news. Of 2,709 evaluated core responses, 45 percent had at least one significant issue. The dominant cause was not style, reasoning or tone. The report states: “Sourcing was the biggest cause of problems, with 31% of all responses having significant issues with sourcing – this includes information in the response not supported by the cited source, providing no sources at all, or making incorrect or unverifiable sourcing claims.” The European Broadcasting Union and the BBC published it as "News Integrity in AI Assistants" on 21 October 2025.

The spread between systems is the argument for writing the policy down rather than trusting the model. Gemini was an outlier: 72 percent of its responses carried a significant sourcing issue, against 24 percent for ChatGPT and 15 percent for Perplexity and Copilot. Same task, same evaluators, 15 percent at one end and 72 percent at the other. What separates them is not raw capability. It is what each system does when the retrieved evidence is thin, absent or contradictory. That is a rule about sources, not a better model. Count independent origins rather than documents, and make the output say which claim rests on which origin.