ConsistencyAI: A Benchmark to Assess LLMs' Factual Consistency When Responding to Different Demographic Groups

AI summary

Overview

Research area: Natural Language Processing / LLM evaluation — specifically fairness, bias, and factual-consistency auditing of large language models, with a policy-facing framing.

Technical level: Intermediate. The pipeline (persona sampling, prompt templates, sentence embeddings, cosine similarity, named-entity overlap) is described without heavy mathematical notation, but readers will benefit from knowing what embeddings and Jaccard similarity are.

Scope: A single paper introducing ConsistencyAI, a provider-independent benchmark that measures whether 25 LLMs present the same facts to users of different demographic backgrounds asking identical questions.

What This Paper Is About

LLMs are increasingly used for information retrieval, and there is a risk that a model presents substantively different facts to different users depending on who it thinks is asking. The paper builds a benchmark that tests this directly: it asks the same factual questions under many different, population-representative personas and measures how much the model's responses overlap across those personas.

The authors are explicit that they are not fact-checking LLMs. Their target is factual consistency — whether the model maintains a stable set of claims across audiences — which they argue matters even when the underlying information is true, because selectively framing true facts can still reinforce a user's existing convictions.

Key Contributions

1. A provider-independent benchmark. ConsistencyAI was designed and run without involvement from LLM providers, described as an impartial mechanism for evaluation and accountability, with the full pipeline and an interactive demo released.

2. A three-scale, high-repetition experimental design. The benchmark was run at three persona scales (2, 8, and 100 personas) across 15 topics, with each query repeated 100 times and persona descriptions drawn from NVIDIA's Nemotron dataset of over 100,000 U.S. Census-grounded synthetic personas. The 100-persona run is the main analysis, chosen because it was the maximum the Nemotron API allowed in one call.

3. A control condition separating persona effects from model noise. Every one of the 25 models was queried 100 times with the same persona and the same topic ("What are the health impacts of GMOs?"), establishing a within-persona variability baseline against which cross-persona divergence is compared.

4. A published benchmark threshold. The authors adopt the across-model mean similarity of 0.8555 as an interpretable industry baseline that providers can seek to outperform.

5. Two complementary metrics. SBERT cosine similarity captures semantic overlap, and spaCy-based named-entity Jaccard similarity captures overlap in which entities (organizations, agencies, scientific bodies) are cited.

Main Findings

• Wide but high-scoring range across models. Across 25 models on 100 personas, factual consistency scores range from 0.9137 (x-ai/grok-4) to 0.6719 (google/gemini-2.5-pro), with a mean of 0.8555 and a median of 0.8653 (deepseek-v3.2-exp). The gap between highest and lowest is 0.2418, and the standard deviation across models is 0.04623.

• The four most consistent models come from three providers. x-ai/grok-4 (0.9137), x-ai/grok-3 (0.9122), openai/gpt-5-chat-latest (0.8949), and mistralai/mistral-saba (0.8833).

• Nine models fell below the benchmark. These include deepseek-r1-0528 (0.8344), anthropic/claude-sonnet-4.5 (0.8318), and qwen/qwen3-next-80b-a3b-thinking (0.7878).

• Persona conditioning changes semantic content, not just style. In the control comparison, 20 of 25 models show lower SBERT cosine similarity in the cross-persona condition than in the fixed-persona control, meaning the persona change drove divergence beyond intrinsic model stochasticity. Named-entity results were more mixed: only 14 of 25 models showed lower entity overlap cross-persona, while the rest showed comparable or higher overlap.

• Topic matters as much as model. Topic-level consistency ranges from 0.8208 ("How are housing costs changing in America?") to 0.9245 ("Give me statistics on the U.S. trade deficit"), a range of 0.1037 with a standard deviation of 0.0240. Other low-scoring topics include U.S. domestic personal information collection (0.8245), changes in abortion access (0.8392), and the causes of the Israeli-Palestinian conflict (0.8409).

• Volatility across topics varies sharply by model. x-ai/grok-3 has the lowest standard deviation across topics (0.0196); mistralai/mistral-medium-3.1 has the highest (0.1007). The mean standard deviation across topics averaged over all LLMs is 0.0409.

• Benchmark outperformance is uneven. Of 375 model-topic specific scores, 230 exceed the 0.8555 benchmark, which the paper puts at roughly 61.33% of trials. Only three models — openai/gpt-5-chat-latest, x-ai/grok-3, and x-ai/grok-4 — beat the benchmark on all 15 topics. google/gemini-2.5-pro beat it on only 1.

• Models cluster users along demographic lines. On "Is the U.S.-Mexico border secure?", qwen3-max produced two response clusters: an enforcement-framed group that averaged age 39 and was 51% female, versus a geography-framed group that averaged age 21 and was 64% female. Claude-Sonnet-4.5 on the same topic split into a small out-group consisting entirely of minors under 21, who received more simplified responses using analogies (comparing border documentation to showing a library card).

• Inconsistency is systemic. Nearly all models showed variance across at least some topics, and even the most consistent models showed reduced alignment on sensitive issues, which the authors read as a challenge shared across LLM architectures rather than a single provider's design choice.

Methodology in Plain English

The benchmark was first prototyped at the 2025 Society-Centered AI Hackathon at Duke University with 14 deliberately contrasting personas (an oil executive, a climate activist, a Pennsylvania coal miner, and others), then scaled up.

In the scaled system, the researchers pull personas at random from NVIDIA's Nemotron persona dataset, which contains over 100,000 synthetically generated personas grounded in U.S. Census and demographic distributions. Persona descriptions include information on persona fields (sex, age, marital status, level of education, occupation) and 16 contextual demographic and socioeconomic fields. Those descriptions are attached to a fixed topic prompt that asks the model to present five facts. Prompts are held constant across all models and runs so results stay comparable, and the temperature parameter was left unchanged to simulate default consumer settings.

Responses are embedded with a Sentence-BERT model and compared pairwise with cosine similarity to measure semantic overlap. Separately, spaCy extracts named entities from each response and Jaccard similarity measures how much the sets of entities overlap. High cross-persona similarity means the model kept a stable factual core regardless of audience.

The control experiment reverses the design: one persona, one topic, 100 repetitions per model, so that within-model randomness can be measured and subtracted conceptually from cross-persona divergence.

The public web app follows an abridged version of the pipeline — users pick 2 to 10 personas, up to 21 models across four providers, and 10 topics — and it substitutes TF-IDF embeddings for SBERT because of SBERT's limitations in a web application. The site is built with Vercel's V0, Python, Next.js, TypeScript, and Tailwind CSS.

Why This Matters

The paper argues that factual consistency is a measurable proxy for whether LLMs are engaging in subtle "social engineering" — presenting different facts to different groups based on inferred demographics rather than stated ideology. It ties the work directly to America's AI Action Plan (whose second priority states AI systems "must be free from ideological bias") and the Executive Order on Preventing Woke AI in the Federal Government, which sets out principles of ideological neutrality and truth-seeking. It also cites civil-society and standards-body concern from the ACLU, EPIC, CDT, the Algorithmic Justice League, NIST's AI Risk Management Framework, and NTIA, alongside provider commitments from Google, OpenAI, and Anthropic to reduce hallucinations.

Scale gives the concern weight: ChatGPT usage devoted to seeking information rose from 14% to 24% between July 2024 and July 2025; ChatGPT had been adopted by roughly 10% of the world's adult population in 2025; Claude had nearly 19 million users; and Google's AI Overviews reach about 2 billion monthly users.

Real-world applications:

• Model procurement and auditing. Governments and enterprises adopting LLMs in the federal government could use above/below-baseline factual consistency scores as an assurance check, consistent with the Action Plan's requirement that adopted systems be objective and free from top-down ideological bias.

• Information-integrity and journalism. Journalists and researchers can run the released pipeline to document whether a model serves different fact sets to different audiences on contested topics such as abortion access, immigration, or housing costs.

• Consumer and civil-society transparency. The interactive demo lets non-specialists generate personas, run a similarity analysis, and view a 3D PCA embedding, a cosine similarity heatmap, and a summary without writing code.

• Model development feedback. Because the benchmark identifies specific topics and providers where consistency drops, developers can target persona-invariant response strategies rather than optimizing only for aggregate accuracy.

Industry relevance: The authors deliberately chose the arithmetic mean of current scores (0.8555) as the benchmark so that it functions as a moving target providers can compete to beat, rather than cherry-picking one model. They note the benchmark is model-flexible and can be extended to new models by running the public code.

Future Directions

• Lowering factual inconsistency on sensitive topics. The lowest-scoring topics (housing costs, domestic surveillance policy, abortion access, the Israeli-Palestinian conflict) and the models that beat the benchmark on only a handful of topics are the obvious targets for improvement.

• Combining external benchmarking with internal model inspection. The authors suggest integrating ConsistencyAI with Anthropic's persona-vector framework, which identifies linear directions in activation space corresponding to traits like sycophancy and hallucination, to pair external output testing with internal mitigation.

• Deciding when personalization is permissible. The clustering results raise a normative question the paper poses but does not resolve: under what contexts, and according to which demographics, is response personalization acceptable? The provided text ends mid-sentence here.

• Refining the benchmark metric itself. The authors acknowledge alternatives to the mean, such as medians or percentile cutoffs, as possible thresholds, and note that the control condition's mixed NER results leave open how much entity-level variation is persona-driven versus intrinsic to the model.

Note: the provided paper content is truncated partway through Section 6 (Discussion). Sections covering limitations, the remainder of the discussion, any conclusion, and any appendix persona breakdown are not included, so their contents cannot be reported here.

Target Audience

• AI policy analysts and regulators who need an operational metric for the neutrality and truth-seeking principles stated in the AI Action Plan and the Executive Order on Preventing Woke AI.
• LLM evaluation and fairness researchers looking for a benchmark that targets cross-persona factual divergence rather than truth verification alone.
• Model developers and provider safety teams who want a reproducible, model-flexible pipeline and a published threshold to test against.
• Journalists, civil-society organizations, and technically literate general readers who want to inspect how a given model answers the same question differently depending on who is asking.

Authors’ abstract

Is an LLM telling you different facts than it's telling me? This paper introduces ConsistencyAI, an independent benchmark for measuring the factual consistency of large language models (LLMs) for different personas. ConsistencyAI tests whether, when users of different demographics ask identical questions, the model responds with factually inconsistent answers. Designed without involvement from LLM providers, this benchmark offers impartial evaluation and accountability. In our experiment, we queried 19 LLMs with prompts that requested 5 facts for each of 15 topics. We repeated this query 100 times for each LLM, each time adding prompt context from a different persona selected from a subset of personas modeling the general population. We processed the responses into sentence embeddings, computed cross-persona cosine similarity, and computed the weighted average of cross-persona cosine similarity to calculate factual consistency scores. In 100-persona experiments, scores ranged from 0.9065 to 0.7896, and the mean was 0.8656, which we adopt as a benchmark threshold. xAI's Grok-3 is most consistent, while several lightweight models rank lowest. Consistency varies by topic: the job market is least consistent, G7 world leaders most consistent, and issues like vaccines or the Israeli-Palestinian conflict diverge by provider. These results show that both the provider and the topic shape the factual consistency. We release our code and interactive demo to support reproducible evaluation and encourage persona-invariant prompting strategies.