Retrieval Sensitivity to Identity Signals in Queries

AI summary

Overview

• Research area: Information retrieval and NLP bias evaluation, specifically how dense (embedding-based) retrievers respond to sociodemographic and ideological signals carried in search queries.
• Technical level: Intermediate. It assumes familiarity with dense retrieval, embedding models, MRR, and basic bias-evaluation methodology, but its design is explained clearly enough for readers outside IR.
• Scope: The paper evaluates five dense retrievers plus a sparse BM25 baseline across four query datasets in political news and consumer health, measuring whether retrieval shifts toward a query's signaled political lean and whether retrieval quality drops for African American Language (AAL) versus White Mainstream English (WME) health questions.

What This Paper Is About

Retrievers decide which documents reach users and the language models built on top of them, but they are typically tested with neutral queries that do not signal anything about the person asking. The authors ask whether the identity signals real users express, here political ideology and dialect, change what a retriever returns. They build paired synthetic evaluations that vary only the identity signal alongside naturalistic query sets, and find that every retriever tested prefers documents matching the questioner's own political lean and ranks answers lower for AAL health questions than for WME ones.

Key Contributions

1. Two query-based identity-bias evaluations for retrievers, one for political ideology and one for AAL/WME sociolect, each combining a synthetic topic-matched (paired) query set that isolates the identity signal with an unpaired naturalistic query set drawn from real users.
2. A consistent finding across six retrieval systems (four open-weight dense encoders, one closed dense encoder, and BM25): every retriever favors documents aligned with the questioner's own political lean, and every retriever scores lower for AAL health queries than WME queries, in both the controlled and the naturalistic settings.
3. A lexical-versus-representational decomposition. Using MCQ log-odds lexical asymmetry and conditional linear probing, the authors show that on content-controlled paired data the bias gap survives partialling out surface vocabulary, and that lean and dialect are linearly decodable from retriever query embeddings beyond token-level features.

Main Findings

• Own-lean tendency is universal. On both AllSides_synth (3,723 left and 3,723 right-leaning paired queries) and Reddit_nat (1,327 liberal and 1,225 conservative queries), every retriever rejects the null of no lean preference at p < 0.05. The tendency holds at retrieval depths k ∈ {5, 10, 20, 50, 100}.
• Gap sizes vary by model and dataset. On AllSides_synth, BM25 and OpenAI te-3-large have roughly 2× the own-lean gap of the four open-weight models. On Reddit_nat, BM25 and Llama-Nemotron-8B have the largest gaps (about 2× those of BGE, Qwen3, and Octen), with OpenAI in-between.
• Dense gaps do not require token overlap. BM25 has among the highest own-lean gaps on both political datasets, but has no gap on Reddit_nat without the gold filter because its unfiltered top-10 is dominated by off-topic documents. Dense retrievers' gaps remain significant, indicating their tendency does not depend on exact token matching.
• WME beats AAL in health retrieval. On both HCMagic_synth and HCMagic_nat, every retriever shows MRR_WME > MRR_AAL. On HCMagic_synth, paired Wilcoxon tests reject on all six retrievers (p ≈ 0). On HCMagic_nat, five of six reject at p < 0.05, with BGE marginal at p = 0.10.
• Naturalistic dialect gaps are larger. The HCMagic_nat gap tends to be about 1.7× larger than the synthetic one for every retriever, suggesting that in real deployments, where AAL and WME queries differ in content as well as dialect, the disadvantage may be larger than the content-controlled estimate.
• Lexical controls do not explain synthetic gaps. On AllSides_synth and HCMagic_synth, partialling out MCQ lexical asymmetry leaves the residual close to the raw gap and significant on every retriever. BM25 carries one of the largest AllSides_synth residuals, so under content control the lexical channel is at least as biased as the dense retrievers.
• Naturalistic results split by retriever. On Reddit_nat, partialling out lexical asymmetry shrinks every gap, but only changes the conclusion for the two dense retrievers with the smallest raw gaps (BGE and Qwen3), whose residuals become statistically insignificant. Octen, OpenAI, and Nemotron retain significant residuals. On HCMagic_nat, no retriever's gap shrinks into insignificance, but the authors attribute this to a confound: the "AAL-distinctive" tokens MCQ picks up are dominated by patient-topic words, not dialect surface form.
• Identity is decodable from embeddings. Linear probes recover lean and dialect from query embeddings with F1 ≈ 85% and 99%, and conditional probing shows most layers add identity information beyond the non-contextual embedding baseline, with positive conditional gains everywhere except BGE on AllSides.
• Back-translation does not fix it. A simple deployment mitigation that back-translates AAL queries into WME does not close the gap and only helps weaker retrievers.

Methodology in Plain English

The authors run two parallel studies, each with a controlled dataset and a naturalistic one.

For politics, they use the Qbias AllSides news corpus, restricted to a subset of about 17k articles whose editorial roundups contain equal numbers of left- and right-leaning documents, so corpus imbalance cannot produce the effect. For the controlled set, they sample 1,000 roundups, use gpt-5-mini to label the frames present in each roundup's center article, and generate one left-leaning and one right-leaning query per frame. Because both queries share a frame, the pair differs in political signaling rather than in which aspect of the story the asker cares about. For the naturalistic set, they collect 3,340 posts from r/AskALiberal and r/AskConservatives and keep only those where two independent LLM classifiers agree on the writer's ideology.

For health, they use the ChatDoctor HealthCareMagic-100k corpus and index all 112,165 doctor responses as documents; the task is to surface the response that answers a given patient question. The controlled set starts from 4,999 questions classified as WME and augments each with AAL morphosyntactic and phonological/orthographic features, three times, producing 14,997 AAL queries that hold content fixed. The naturalistic set uses 656 AAL queries (647 after filtering) identified by a demographic-alignment classifier plus a similar number of real WME queries.

They then run the same six retrievers over all four datasets. Politics is scored with a lean metric: the difference between the fraction of right- and left-leaning articles in the top-k, with gold filtering to keep only on-topic retrievals. Health is scored with mean reciprocal rank of the gold doctor response. Significance is assessed with cluster bootstraps, permutation tests, paired Wilcoxon, and Mann-Whitney U tests. Finally, to test whether the gaps are about vocabulary or representation, they compute a per-query lexical asymmetry score using MCQ log-odds and regress the bias gap on it, and they train linear probes on each dense retriever's layer-wise query embeddings to see whether lean and dialect are encoded beyond the input token features.

Why This Matters

• Research impact: The paper shifts bias evaluation from neutral queries and document-level representation to the identity signals inside the query itself, and it supplies a reusable paired/unpaired design plus a lexical-versus-representational decomposition that other bias studies can adopt.
• Search and recommendation: If a retriever systematically skews which documents it surfaces based on who is asking, users of search and recommendation systems receive systematically different information for the same underlying need.
• Retrieval-augmented generation: Skewed retrieval is passed directly to a large language model, so retrieval bias propagates into generated answers that users increasingly treat as authoritative.
• Health information access: The AAL/WME gap means questioners using AAL get lower-ranked answers to the same health question, which the authors frame as an allocational harm and a reinforcement of existing health disparities.
• Polarization: Retrieval that confirms a questioner's existing political lean can entrench echo chambers rather than inform.

Industry relevance: Any organization deploying dense retrieval for search, recommendations, or RAG pipelines uses models of exactly the families tested here, including BGE, Qwen3, Llama-Nemotron, Octen, and OpenAI's text-embedding-3-large. The finding that the bias is consistent across architectures and is not closed by a simple back-translation mitigation suggests that teams shipping retrieval systems cannot treat this as a single-model bug or a surface-level preprocessing problem.

Future Directions

• Representation-level mitigation. The authors state that because the gaps are consistent across architectures and not closed by back-translation, they likely require mitigation at the level of learned representations, which they leave to future work.
• Finer-grained identity. The binary contrasts (left versus right; AAL versus WME) collapse centrists and ignore intersectional or non-binary identities, and the labels used as proxies carry their own error and bias.
• Beyond gold-document evaluation. The health metric scores only the rank of each question's original doctor response; it does not assess the medical usefulness or safety of the other retrieved documents, and the authors propose LLM-judging full retrieved sets as a complementary evaluation.
• Closing the paired/unpaired trade-off. Synthetic queries are content-controlled but potentially artifact-prone, while naturalistic queries are ecologically valid but confound dialect or lean with content and topic; neither setting alone is decisive.

Target Audience

This paper is most useful to IR and NLP researchers working on retrieval bias, fairness, and evaluation design; to practitioners building or auditing search, recommendation, and RAG pipelines; and to researchers studying dialect and sociolinguistic variation in language technology. Readers interested in health equity and political polarization will also find the two application domains directly relevant. A working knowledge of dense retrieval and standard retrieval metrics helps, though the experimental design is described concretely enough for a motivated non-specialist to follow.

Authors’ abstract

Dense retrievers decide which documents reach users and the language models that use them, yet they are typically evaluated with neutral queries. We ask whether the identity signals that real users express in their queries---political ideology and dialect---bias what a retriever returns. We design evaluations in two domains, political news and consumer-health questions, each pairing a controlled synthetic set that varies only the identity signal with naturalistic queries. Across five dense retrievers and a sparse baseline, every retriever (i) retrieves articles that align with the query's own political lean and (ii) performs worse for questions written in African American Language (AAL) than in White Mainstream English (WME). Two analyses tie these gaps to queries' identity signals beyond surface vocabulary: partialling out an aggregate lexical-asymmetry score leaves the synthetic gaps largely intact, and linear probes recover lean and dialect from the retrievers' query embeddings beyond token-level features. Left unaddressed, such retrieval biases risk contributing to polarization and reinforcing the health disparities already faced by AAL speakers. Code is available at https://github.com/Andrewtcr/bias-ret.