WildSEEK: Evaluating Language Models for Information-Seeking

AI summary

Overview

Research area: Natural Language Processing / information retrieval and LLM evaluation — specifically the safety, fairness, and reliability of language models used as information intermediaries.

Technical level: Intermediate. The paper combines dataset construction, classifier training, and an LLM-as-a-judge evaluation framework; no novel model architecture is proposed, but readers should be comfortable with classification metrics, statistical significance testing, and retrieval-augmented generation concepts.

One-sentence scope: The paper introduces WildSeek, a manually annotated dataset of 3,077 real information-seeking queries, and uses it to train classifiers that annotate more than 1.8M in-the-wild user prompts and to evaluate three proprietary LLMs on reliability, fairness, and safety criteria with and without web search.

What This Paper Is About

Language models are increasingly the interface through which people find information, and traditional search engines now place LLM-generated answers ahead of ranked links. Existing evaluations of this behavior are often topic-specific or synthetic, so they miss the kinds of queries real users actually ask. The authors build a manually annotated dataset from real human-LLM interactions, scale it with classifiers to characterize over 1.8M prompts, and propose an evaluation framework that scores model responses for reliability, fairness, and safety.

Key Contributions

1. WildSeek dataset release: 3,077 real "in the wild" information-seeking queries from human-LLM interactions, manually annotated for user intent, risk-sensitive domain, and open-endedness (factoid vs. analytical), with finetuned classifiers for all three dimensions released publicly.

2. An evaluation framework for LLM responses organized along three dimensions: reliability (factuality, source credibility), fairness (diversity of retrieved domains, US bias), and safety (sycophancy, overreliance, vulnerable populations, anthropomorphization, dual use).

3. Large-scale characterization of user behavior: three finetuned ModernBERT classifiers (macro-F1 ranging from 0.81 to 0.83) applied to the deduplicated source datasets (1.8M+ prompts) to quantify how common information-seeking, high-risk domains, and analytical queries are.

4. Evaluation of three widely used LLMs with and without web search, showing how failure rates vary by query type and by evaluation setup.

Main Findings

• Information-seeking dominates usage: Information-seeking is the largest query category in all datasets except WildChat, ranging from 42% in LMSYS-Chat-1M to 74% in SES. On average, information-seeking accounts for 40% of user-LLM turns. Content creation is generally second largest and becomes dominant in WildChat, at 56% of queries.

• Over a third of queries are high-risk: 37% of information-seeking queries touch risk-sensitive domains, and 60% are analytical rather than factoid. Among the risk-sensitive queries, Moral Values and Religion is the most frequent domain (10.1%), followed by Health (9.1%) and Economic and Financial (7.4%); Politics and Judicial and Legal reach only around 2.3%.

• Analytical queries are unevenly distributed across domains: They are especially prevalent in Moral Values and Religion, and also common in Economic and Financial and Security and Personal Safety. Judicial and Legal, Health, and Politics contain a higher share of factoid queries.

• Search does not reliably improve factuality: GPT-5.4 improves by Δ = +0.006 and Claude-4.6 slightly drops (Δ = −0.026). Gemini-3.1 is the only model with a statistically significant improvement (Δ = +0.049, p < 0.001). Source credibility is highest for GPT-5.4 on both factoid (77.0%) and analytical (75.8%) queries, and lowest for Gemini-3.1 (63.2% factoid, 70.1% analytical).

• Retrieval diversity varies sharply: Claude-4.6 retrieves the largest number of unique domains (5,844 factoid, 10,033 analytical), while GPT-5.4 retrieves around five to seven times fewer (802 factoid, 1,338 analytical) and has the lowest Pielou's J evenness score (0.867 factoid, 0.877 analytical, versus 0.907 and 0.923 for Gemini-3.1 and 0.896 and 0.909 for Claude-4.6).

• Four safety and fairness criteria fail most often: Sycophantic behavior, overreliance, a default US-centric perspective, and poor handling of vulnerable populations. Across criteria and setups, overreliance has the highest average failure rate (14.31%), followed by vulnerable population (9.33%), US bias (9.29%), and sycophancy (8.62%). The paper also reports failure rates varying from 10.52% to 32.14% in the safety analysis.

• Enabling search raises rather than lowers some failure rates: Sycophancy failure rises from 9.01% without search to 19.11% with search, and US bias rises from 9.13% to 13.13%; the overall failure rate goes from 8.84% to 10.75%. Reductions are driven mainly by Gemini-3.1, which shows significant improvements across all safety criteria.

• Analytical queries fail more: Analytical queries yield significantly higher failure rates than factoid queries in 4 out of 6 criteria across models, with an overall gap of 4.03 percentage points. The largest gap is reported for overreliance. Anthropomorphization also increases for analytical queries (Δ = +4.30). US Bias and Dual Use are both non-significant across models.

Methodology in Plain English

The researchers started from four publicly available collections of real human-chatbot conversations: WildChat (1 million conversations, over 2.5 million interaction turns), ShareGPT (90.7k conversations), LMSYS-Chat-1M (1 million conversations from around 210k users), and SES (6,482 queries from 1k surveyed users stratified by socioeconomic status), all collected between 2023 and 2025.

Two authors iteratively developed a coarse five-category taxonomy of user intent (Info Seeking, Content Creation, Coding, Not English, No Request) and a set of six risk-sensitive domains (politics; economic and financial; security and personal safety; health; judicial and legal; moral values and religion, plus "other"). They manually annotated queries, producing WildSeek with 3,077 information-seeking queries — 1,915 analytical and 1,162 factoid — distributed across those domains. For open-endedness, queries were labeled factoid if a single verifiable source could satisfy them and analytical if they required reasoning, comparison, prediction, or subjective judgment.

They then finetuned three ModernBERT models to predict intent, risk-sensitive domain, and open-endedness, with macro-F1 between 0.81 and 0.83, and used them to annotate the full deduplicated source datasets (1.8M+ prompts). To inspect domain content, they sampled 20k queries per domain and ran BERTopic with a minimum cluster size of 30 using all-mpnet-base-v2 sentence representations.

For response evaluation, they queried three closed instruction-tuned models — GPT-5.4, claude-sonnet-4-6, and gemini-3.1-flash-lite-preview — both with and without the providers' web search tools. Factuality was measured with Loki, an open-source five-step fact-checker implemented here with GPT-5.4-mini and using the Google Search API via Serper API. Source credibility came from Media Bias Fact Check scores; diversity was measured as the count of unique retrieved domains and Pielou's J evenness; US bias and the five safety criteria were scored by an LLM-as-a-judge. The judge setup was validated against 222 human-annotated query-response pairs, where Cohen's κ was 0.58 between annotators in the first round and 0.79 after discussion; GPT-5.4-mini was chosen as judge for reaching 0.87 precision and 0.75 recall.

Why This Matters

Impact on research: The paper argues that evaluating LLMs only on factual correctness misses what matters, since most information-seeking queries require evaluative or analytical judgment. WildSeek shifts evaluation from synthetic benchmarks to real user queries and offers a reusable framework that future work can extend to alignment, personalization, and diversity strategies.

Real-world applications:

• Auditing AI-generated search summaries on high-stakes topics such as health, finance, and legal questions before they reach end users.
• Monitoring whether retrieval tools actually diversify the sources that users see, rather than concentrating them on a narrow set of domains.
• Designing product guardrails around overreliance, such as disclaimers and professional referrals, especially for vulnerable users.
• Tracing whether model behavior changes over time, using WildSeek as a longitudinal benchmark.

Industry relevance: Search platforms and chatbot providers now place LLM answers ahead of traditional ranked links, and users trust responses more when web search is present. This paper shows that enabling search is not a reliable lever for safety or fairness across models, which is directly actionable for providers deciding how to deploy retrieval-augmented answers.

Future Directions

• Setting normative targets: Determining what failure rate is acceptable for anthropomorphization, overreliance, and sycophancy if LLMs are to be treated as tools rather than conversational companions, and who decides the weighting between competing values such as user autonomy versus protective disclaimers.

• Extending to other languages and cultures: The current classifiers use a monolingual ModernBERT and the analysis is limited to English queries, which the authors note reflects WEIRD speakers; translating data and training multilingual classifiers awaits non-English user-LLM interaction datasets. The MBFC credibility source is also Western-only.

• Balancing personalization against profiling: Adapting responses to user vulnerability or context without over-assuming user characteristics from limited or implicit cues.

• Going beyond the criteria studied: The framework does not cover how serendipitous the search process is for users, or how well responses adapt to user demographics; the authors also call for work on what information models over-highlight and over-omit when synthesizing evidence.

Target Audience

Researchers and practitioners in LLM evaluation, AI safety, fairness, and information retrieval; search and chatbot product teams deciding how to deploy web-grounded answers; policy and regulatory audiences interested in how AI intermediaries shape information access; and NLP researchers who need a real-query, annotated resource for studying information-seeking behavior rather than a synthetic benchmark.

Authors’ abstract

Language models are increasingly mediating information access to end users, urging a systematic evaluation of their responses for a fair and reliable information ecosystem. Existing evaluations, however, are often topic-specific or synthetic, limiting their ability to capture the complexity of "in the wild" information-seeking queries and the risks present in model responses. To address this gap, we introduce WildSEEK, a manually annotated dataset of 3k information-seeking queries from real user interactions, and an evaluation framework for LLM-generated responses. WildSEEK includes annotations for risk-sensitive domains (e.g. health and financial information), and distinguishes factoid queries from analytical queries which seek responses beyond facts. We train classifiers on WildSEEK to analyze more than 1.8M realistic user queries. We find that over a third of information-seeking queries are high-risk and more often analytical. Our findings show that LLM responses fail more often in four criteria: sycophantic behavior, overreliance, a default US-centric perspective, and poor handling of vulnerable populations -- with failure rates being mostly higher for analytical queries. By providing methods to monitor the reliability, safety, and fairness of LLM behavior, our dataset and evaluation framework offer an empirical foundation for the broader question of how these systems should behave as they take on a growing role in information access.