SearchInject: Retrieval Exposure Control for Indirect Prompt Injection in Language-Model Search

AI summary

Overview

Research area: Security of retrieval-augmented and search-augmented large language models, specifically indirect prompt injection and adversarial information retrieval.

Technical level: Advanced. The paper assumes familiarity with retrieval metrics (NDCG@10, MRR@10, Recall@10), logistic scoring, calibration constraints, and analytical confidence bounds.

Scope: The paper proposes and evaluates SearchInject, an admission-stage filter that scores retrieved candidate pages before they enter a language model's context, and measures how much it reduces exposure to crafted instruction-carrying pages across three lightweight retrievers while tracking the cost to clean retrieval quality.

What This Paper Is About

A search-augmented language model can only be manipulated by a hidden instruction on a web page if that page is first retrieved and placed into the model's context. The authors treat this as an "admission problem" rather than a generation problem, and build a scoring rule that decides which candidates to quarantine before context assembly. The goal is to cut how often crafted target pages reach the model while keeping clean search results essentially unchanged, and to report honestly what the evidence does and does not establish.

Key Contributions

1. A relevance-conditioned risk score. SearchInject combines four signals in a fixed feature order — normalized retrieval relevance r, search-abuse cues a (title coverage, body term density, repetition, source overlap, markup indicators), instruction likelihood i (IDF-weighted word one- and two-grams plus character three- to five-grams), and two interaction terms a·r and i·r — through a class-balanced logistic regression whose output is used as a risk score.

2. A clean-cost-constrained quarantine policy with refill. The threshold is chosen on a calibration split to maximize crafted-page recall subject to an empirical clean-candidate false-quarantine rate of at most 0.005, yielding τ = 0.973853 with calibration recall 0.804 and calibration false quarantine 0.00498. At inference the policy scans the depth-100 ranking, skips candidates scoring above τ, and refills to ten accepted pages from the same list without reranking or fetching new candidates.

3. A controlled evaluation that separates exposure from hijacking. Using 300 held-out SciFact queries, 13 held-out payload categories, and four construction families (title stuffing, doorway, near duplicate, hidden metadata) all seen during development, the paper reports target exposure as a distinct endpoint from downstream attack success, so a change in one is not credited as a change in the other.

4. An explicit accounting of statistical and reproducibility limits. The authors replace naive per-case binomial reasoning with a query-level paired analysis, report that the frozen threshold transfers above its calibration constraint on test, and state that the interaction terms, matched-cost component comparisons, and attack-success reduction remain unestablished.

Main Findings

• Large exposure reduction across all three retrievers. Over 1,200 query–family outcomes per retriever, top-10 target exposure falls from 0.9408 to 0.1583 for BM25, from 0.9158 to 0.1508 for the normalized hybrid, and from 0.8450 to 0.1375 for latent semantic analysis. Exact counts move from 1,129 to 190 (BM25), 1,099 to 181 (hybrid), and 1,014 to 165 (LSA).

• Relative reductions of 83.2–83.7%. The absolute reductions are 0.7825 for BM25, 0.7650 for the hybrid, and 0.7075 for LSA.

• Small measured clean utility cost. Clean NDCG@10 changes by −0.0018 (BM25), −0.0013 (hybrid), and −0.0013 (LSA); the largest absolute change is 0.0018. Paired bootstrap intervals are [−0.0053, 0], [−0.0039, 0.0001], and [−0.0039, 0]. MRR@10 changes by −0.0017, 0, and 0, and Recall@10 decreases by 0.0017 for each retriever. The authors note these near-zero estimates do not constitute a formal equivalence test.

• Analytical intervals are wide. Using Hoeffding's inequality with 300 independent query units gives a ±0.15682 marginal 95% interval, yielding [0.626, 0.939] for BM25, [0.608, 0.922] for the hybrid, and [0.551, 0.864] for LSA.

• The calibration constraint does not transfer exactly. Observed clean test false-quarantine rates are 0.590% (BM25), 0.620% (hybrid), and 0.650% (LSA), each above the 0.5% calibration constraint by 0.09, 0.12, and 0.15 percentage points. Calibration placement counts are unreported, so a sampling-error estimate is not possible.

• Doorway and near-duplicate pages resist quarantine most. Doorway pages retain roughly 0.21–0.23 post-quarantine exposure under every retriever, and near duplicates retain exposure under all three, with lower post-quarantine values for LSA (0.133) than for BM25 (0.197) or the hybrid (0.163). Title stuffing and hidden metadata drop to roughly 0.080–0.130.

• Component policies trade off differently but are not matched fairly. Abuse-only screening reaches exposure 0.1436 at a clean NDCG cost of 0.1113; instruction-only screening leaves exposure at 0.4169 with a cost of 0.0071; joint quarantine combines exposure 0.1489 with a mean clean change of −0.0014. Because the policies were not matched at a common false-quarantine rate, the benefit of the interaction terms cannot be attributed from these numbers.

• Continuous reranking is retriever-dependent. For the hybrid, reranking reaches exposure 0.083 with a clean NDCG decrease of 0.062; under BM25 exposure stays at 0.878. The two-stage penalty shifts exposure by only 0.1–0.5 percentage points relative to quarantine while incurring a larger clean loss.

• Some payload categories remain hard. The instruction detector's held-out-category recall is 0.754 with calibration false quarantine 0.00076. Under hybrid retrieval, question-like sentiment requests reach post-quarantine exposure 0.500, conversational requests 0.476, research assistance 0.346, task automation 0.298, and business intelligence 0.269.

• The generation follow-up is inconclusive by design. All 104 selected targets enter the undefended hybrid top five; after quarantine, 21 remain and 83 contexts no longer contain the instruction. Across Qwen3-4B and Phi-4-mini-instruct under conventional and hardened prompts, every cell records 0/4 confirmed hijacks and 4/4 unresolved among 104 attacked cases, with 1,192 responses and 1,248 judgments overall. Because no undefended hijack was confirmed under a post hoc, non-blinded endpoint, the study cannot estimate a reduction in attack success. Clean accuracy is 40.0%/37.8% for Qwen3-4B under conventional/hardened prompts and 52.2% for Phi-4-mini under either prompt.

Methodology in Plain English

The authors start from a simple observation: an injected instruction is harmless to a search assistant until the page carrying it is pulled into the model's context. So instead of trying to detect instructions inside the model or train the model to ignore them, they put a gate on the retrieval pipeline.

They build a controlled test bed. Clean material comes from SciFact scientific abstracts in BEIR format: 5,183 documents, 809 training queries (655 for fitting, 154 for calibration) and a held-out 300-query test split. Attack material comes from a filtered subset of BIPIA text attacks — 9 fitting categories, 3 calibration categories, and 13 test categories — with malware, fraud, propaganda, and political-persuasion payloads excluded.

They then construct four kinds of malicious pages by pairing each SciFact query with a chosen source document: title stuffing (repeating key terms in the title and lead sentence), doorway pages (a query-specific summary followed by the payload), near duplicates (most of a relevant abstract plus an appended instruction), and hidden metadata (query terms placed in indexed fields while the visible body stays natural).

Each candidate is scored with a logistic model over relevance, abuse cues, instruction likelihood, and two interactions between relevance and the other signals. The threshold is picked on the calibration split to maximize catch rate while keeping false quarantine of clean candidates at or below 0.5%. Then the model and threshold are frozen and run on the held-out queries and payload categories.

At retrieval time, the system walks down the original depth-100 ranking, skips pages above the threshold, and keeps filling the top ten from the remaining list — so a blocked page is replaced rather than leaving a gap. Three cheap rankers are tested: BM25, latent semantic analysis, and a normalized hybrid averaging the two.

Finally, they check two things separately: did the crafted page reach context (exposure), and did a model actually act on the instruction (hijacking). A small 104-case generation study with Qwen3-4B and Phi-4-mini-instruct at four-bit precision and temperature-zero decoding probes the second question, using prompts capped at five sources of 450 characters and 128-token answers labeled SUPPORTS, CONTRADICTS, or NOT_ENOUGH_INFO.

Why This Matters

Impact on research. Most prompt-injection defense work operates after retrieval — detecting contaminated spans, training models to respect instruction/data boundaries, or checking tool calls. This paper isolates the earlier admission decision and quantifies it, and it is unusually explicit that exposure reduction is not the same as attack-success reduction. It also demonstrates a rigorous way to report a security gain alongside its clean-utility cost and its calibration-transfer failure.

Real-world applications (as implications of the approach):

• Enterprise search and internal RAG assistants that retrieve from wikis, ticketing systems, or shared drives where any contributor can plant text.
• Web answer engines and AI-summarized search results that ingest third-party pages with no publisher trust relationship.
• Customer-support bots that pull from public documentation and forums susceptible to doorway or near-duplicate content.
• Agentic pipelines where retrieved text triggers tool calls, making page admission a precondition for downstream authorization failures.

Industry relevance. The authors are affiliated with Google LLC (with one co-author at Harvard University), and the threat model is squarely aimed at deployed search-augmented products: adversarial SEO techniques such as query-term repetition, title manipulation, doorway text, and copied passages are already familiar in web search, and this work connects them directly to model-context contamination. The finding that cheap lexical and latent-semantic retrievers show consistent reductions suggests the gate is inexpensive enough to sit in front of an existing ranking stack.

Future Directions

• Matched-cost component ablation. Refit [r, a, i] against [r, a, i, a·r, i·r] on identical splits with separately selected thresholds under the same calibration constraint, to determine whether the interaction terms earn their place. The paper states this comparison remains unreported.
• Defense-aware and construction-level holdout evaluation. All pages were fixed before final evaluation, and no defense-aware adaptive attack was run; the authors note that static attack suites can overstate protection and that a construction-level holdout was not evaluated.
• Threshold sensitivity and calibration methodology. A prespecified grid of calibration constraints would show how test exposure, false quarantine, NDCG, and refill behavior move together, since only one threshold (0.973853) is reported and calibration placement counts are missing.
• Broader transfer and downstream endpoints. Dense retrievers, independent construction families, source-level monitoring for repeated false quarantine of a single publisher, and a prospectively defined generation endpoint with confirmed undefended attacks are all identified as needed. Exact replay also requires original candidate records and trained artifacts, which the current revision does not provide.

Target Audience

Researchers and engineers working on RAG security, prompt-injection defense, and adversarial information retrieval; applied scientists designing guardrails for search-augmented assistants; and evaluation-methodology readers interested in how to report security gains without conflating retrieval exposure with attack success. Readers looking for a deployable, fully reproducible defense will find the paper deliberately conservative about what its evidence supports — it reports 31,500 query-level rows and 315,000 top-10 rows across 105 scenario–retriever–policy groups, but states that the released package cannot replay the experiments or supply the missing component and threshold analyses.

Authors’ abstract

An indirect prompt injection can affect a search-augmented large language model only after its carrier page reaches model context. SearchInject controls this admission decision with a joint score combining retrieval relevance, search-abuse cues, instruction likelihood, and two relevance interactions. A quarantine threshold is selected under an empirical 0.5% clean-candidate false-quarantine constraint on a calibration split. Evaluation uses 300 held-out SciFact queries and held-out payload categories within four construction families already represented during development. Across 1,200 query-family outcomes per retriever, top-10 target exposure falls from 0.9408 to 0.1583 for BM25, from 0.9158 to 0.1508 for a normalized hybrid, and from 0.8450 to 0.1375 for latent semantic analysis. These are relative reductions of 83.2-83.7%; the largest absolute change in clean normalized discounted cumulative gain at rank 10 is 0.0018. The frozen threshold yields clean test false-quarantine rates of 0.59-0.65%, above the empirical calibration constraint. A query-level analytical reanalysis accounts for the repeated construction outcomes under an independent-query assumption. A separate 104-case generation follow-up records context removal but no confirmed undefended hijacks under a post hoc endpoint, so it cannot estimate a reduction in attack success. The evidence supports retrieval exposure control within the evaluated construction families; the benefit of the interaction terms at matched clean cost remains unresolved.