Wrong Together: Shared Bias in User Simulation

AI summary

Overview

Research area: Natural language processing and computational social science, specifically LLM-based "silicon sampling" of survey respondents and the evaluation of simulated population estimates.

Technical level: Intermediate. The paper uses equivalence testing, AUC-based risk ranking, and variance decomposition, but the central idea is a simple one: simulators that agree may simply be wrong in the same way.

Scope: The paper tests whether disagreement across six language-model simulators can flag unreliable estimates of a single population's response proportion ("level") versus the difference between two populations ("contrast"), using 60 GlobalOpinionQA questions across 12 countries.

What This Paper Is About

Researchers increasingly use language models to stand in for human survey respondents, but a simulated estimate is only useful if it is close to what real respondents would have said. The paper asks whether the amount of disagreement among several simulators can serve as a warning sign that a simulated estimate is invalid, assessed without access to the target population's human responses. The key twist is that simulators can agree with each other while being collectively wrong, so the paper examines when and why this shared error is visible to the disagreement signal and when it is not.

Key Contributions

1. A level-versus-contrast comparison of the disagreement signal. The authors show that cross-simulator disagreement predicts invalidity substantially better for between-group differences than for single-group response proportions (AUC 0.87 versus 0.71), and that this ordering holds on held-out questions, restricted country pairs, and separate survey sources.

2. An error decomposition that explains the gap. Simulator error is split into a component shared across simulators and constant across countries, a component shared across simulators but varying by country, and a simulator-specific residual. Because a contrast cancels the country-constant component, the shared error that disagreement cannot detect is partly removed before the prediction task.

3. A cross-country bias-correction test. A per-question offset estimated on 7 calibration countries is subtracted from level estimates for 5 held-out test countries, reducing mean absolute error from 0.194 to 0.129 while leaving disagreement scores unchanged.

4. A retrospective selective-referral evaluation. Referring the 20% of contrasts with the greatest disagreement for human assessment reduces invalidity among resolved, retained contrasts from 63.7% to 54.8%, showing that disagreement can order human checks even though it cannot certify the estimates it retains.

Main Findings

• Contrasts are more predictable than levels. Disagreement yields AUC 0.87 (95% CI [0.85, 0.90]) for contrasts and 0.71 ([0.67, 0.75]) for levels, a paired difference of 0.16 (question-clustered CI [0.13, 0.19]). On 24 separate test questions the values are 0.86 and 0.73 (difference 0.14), and on the 5 test populations only they are 0.88 and 0.72.

• Most error variance is shared across simulators. Of total error variance, 58% (95% CI [53, 65]) is shared across simulators, and 65% ([56, 72]) of that shared component is constant across countries and cancels in a contrast. The country-varying shared component accounts for 21% of original error variance.

• Shared error changes the residual error levels. After cancellation, mean absolute error is 0.199 for levels and 0.166 for contrasts, so contrasts remain more accurate while still carrying a shared component that disagreement cannot measure.

• Bias correction works, but not through the disagreement signal. Subtracting the calibration-country offset lowers test-country level error from 0.194 to 0.129 (CI on the reduction [0.044, 0.087]); a placebo that shuffles offsets across unrelated questions raises median error to 0.246. AUC against corrected labels moves only from 0.72 to 0.71 (difference CI [-0.05, 0.05]), consistent with removing an error component invisible to disagreement.

• Correction benefits the questions it should. Grouping questions into thirds by the share of calibration error that is constant across countries gives shares of 0.26, 0.62, and 0.86, with corresponding test-country error reductions of 0.019, 0.052, and 0.123.

• Referral lowers risk but leaves much unresolved. Invalidity among resolved contrasts falls from 63.7% at full coverage to 54.8% at 80% coverage, an 8.9-point reduction (CI [7.5, 10.3]); restricting to test-country pairs gives 8.8 points. Across all 14388 contrasts, the equivalence test supports 20%, contradicts 32%, and leaves 48% unresolved, and 51% of the retained 80% remain unresolved.

• Several uncertainty signals perform similarly. Cross-model disagreement (0.87), pooled within-model sampling spread (0.88), and leave-one-model-out instability (0.86) are within ±0.02 of each other and each beats ensemble-distribution entropy (0.65) and plurality margin (0.63). The pooled-spread result depends on pooling: for five of six individual simulators, that simulator's own spread gives AUC 0.51 to 0.58, versus 0.75 to 0.80 for disagreement on the same estimates.

• The signal does not require multiple variants per family. An ensemble with one model per pretraining family gives contrast AUC 0.88, close to the full ensemble's 0.87.

• The ordering survives robustness checks. Using only human sampling uncertainty gives level AUC 0.63 and contrast AUC 0.78; scaling all margins from 0.5× to 2× keeps contrast AUC in [0.81, 0.90]. Pew gives 0.87 versus 0.71 and the World Values Survey 0.87 versus 0.73, with paired differences of 0.16 ([0.13, 0.20]) and 0.14 ([0.02, 0.23]).

• Ordinal estimands show the same pattern, more weakly. Ordinal contrast AUC is 0.72 ([0.64, 0.79]), direction 0.67, absolute mean 0.51, and full distribution 0.47; pooling all families gives 0.59. Referral on ordinal contrasts lowers invalidity from 40.8% to 32.9%, versus 55.2% to 50.9% across all families.

• Invalidity differs sharply by estimand. Direction 0.391, ordinal contrast 0.507, mean 0.651, distribution 0.942. On a standardized error scale, contrast error is 0.262 lower than mean error (SE 0.043, p < 0.001).

• Agreement can still mean shared wrongness. Among low-disagreement estimates, invalidity is 0.456, versus 0.673 among high-disagreement estimates; for distribution estimates disagreement and error are weakly negatively related.

• Simulator provenance matters less than expected. gpt-4o and mistral-large show 63.2% invalidity versus 63.6% for the smaller models, a comparison the authors explicitly say does not isolate parameter count or training.

• Uncertainty accounting is material. Median simulator standard error is approximately 0.025 versus approximately 0.016 for humans, and including the simulator term moves about a fifth of otherwise-resolved contrast estimates to unresolved.

Methodology in Plain English

The study treats each simulated survey estimate as a claim to be checked. For every question, country, and answer option, the authors average the outputs of six simulators (gemma4-31b, mistral-small, gpt-4o-mini, qwen3-next-80b, gpt-4o, and mistral-large, spanning four model families) and compare the ensemble estimate with the real human response distribution from GlobalOpinionQA.

Whether an estimate counts as "valid" is decided by an equivalence test rather than a point comparison. The authors fix a scientifically meaningful error margin in advance (0.10, i.e., ten percentage points, for the primary proportions) and build a 95% confidence interval around the observed error, combining human sampling uncertainty with simulator Monte-Carlo uncertainty. If the whole interval sits inside the margin the estimate is "supported"; if it sits outside it is "contradicted"; if it straddles the margin it is "unresolved." Only resolved estimates enter the invalidity rates, and the unresolved share is reported separately.

The predictor is deliberately simple: disagreement is the mean absolute deviation of the six simulator estimates, computed from simulator outputs alone with no human data. Its quality is measured by AUC, the probability that an invalid estimate gets a higher disagreement score than a valid one, with inference clustered by question. Because the authors also want to test the shared-error explanation, they split the 12 countries into 7 calibration and 5 test countries, estimate a per-question bias on the calibration set, and apply it to the test set — a manipulation that shifts every simulator equally and therefore leaves disagreement and its ranking untouched. A placebo version shuffles offsets onto unrelated questions to confirm the reduction is not an artifact.

Finally, they evaluate a referral rule retrospectively: rank contrasts by disagreement, send the top 20% to humans, and check whether invalidity among the resolved estimates that remain is lower. Throughout, all checks use existing human survey responses, and the paper is explicit that this is not a preregistered prospective prediction.

Why This Matters

Impact on research. Silicon sampling is often justified by plausibility rather than calibration. This paper gives researchers a cheap, label-free triage signal and, more importantly, tells them where that signal fails: it is informative for between-group comparisons and largely blind to errors that all simulators share within a single group. The finding that 58% of error variance is shared across simulators is a direct caution against treating multi-model agreement as evidence of correctness.

Real-world applications:

• Survey pretesting. Teams can use disagreement to decide which simulated contrasts to check with real respondents first, rather than treating all simulated estimates as equally trustworthy.
• A/B and product-decision simulation. When simulated users are used to estimate the difference between two conditions, the paper's result suggests the comparison is the safer quantity to report, with the caveat that the cancellation argument must be re-tested for that setting.
• Benchmark and evaluation design. The level-versus-contrast distinction argues for reporting these estimands separately rather than as a single fidelity score, since they behave very differently.
• Interactive system evaluation. The paper's framing of shared bias applies to simulated users of search, recommender, and dialogue systems, where a bias constant across systems would cancel in system-level comparisons.

Industry relevance. Organizations substituting LLM panels for human respondents need to know which simulated numbers can support a decision and which require a human check. This work provides both a triage rule and a cost model — with assigned error and referral costs, the threshold rule yields a maximum modeled saving of 17%, and 13% at a cost ratio of 0.3 where 35% of contrasts are retained.

Future Directions

1. A prospective, preregistered test. The authors note that existing survey responses predate the study design and that the 300 population splits assess sensitivity to country choice rather than transfer to a new human study. They plan a time-stamped protocol and a blind collection application.

2. Extending to interactive tasks. The primary evidence covers global opinion with six simulators from four model families; transfer to search, recommender, and dialogue simulation requires separate evidence, and an exploratory persuasion-dialogue analysis in the appendix did not permit scoring the disagreement predictor.

3. Testing whether bias is stable across systems. The cancellation argument only helps if a simulator's bias is sufficiently similar across the systems being compared; the paper states this must be tested rather than assumed, and cites evidence that a conversational search simulator overstates retrieval effectiveness on turns where real users struggle.

4. Improving on disagreement as a signal. Because disagreement is provably blind to the country-constant shared component, the natural open question is what additional evidence — calibration countries, auxiliary human data, or design changes to the simulator ensemble — could detect the error component that survives disagreement-based referral.

Target Audience

Researchers and practitioners who use language models as stand-ins for human participants — survey methodologists, computational social scientists, and NLP evaluation researchers — along with teams building user simulators for search, recommendation, and dialogue systems who need to know which simulated quantities can be trusted. Readers should be comfortable with basic statistics (confidence intervals, AUC, variance decomposition); the conceptual message about shared bias is accessible without that background.

Authors’ abstract

User simulators generate responses in place of human participants, allowing researchers to estimate the result of a study. To use these estimates before collecting human responses, researchers need a way to assess their reliability. We test whether disagreement among simulators can identify unreliable estimates of a group's response proportions and of differences between groups. We evaluate this signal using 6 language-model simulators and existing GlobalOpinionQA responses for 60 questions in 12 countries. Disagreement predicts invalidity more accurately for differences between two groups than for a single group's response proportions (AUC 0.87 versus 0.71). Errors shared by the simulators remain invisible to this signal. When such an error affects both groups equally, it cancels in the comparison, helping explain the stronger result for group differences. We test this explanation by estimating the shared bias using one set of groups and subtracting it from predictions for a separate set. This correction reduces prediction error without changing disagreement among the simulators. In a retrospective analysis, we simulate referring the 20% of group comparisons with the greatest disagreement for human assessment. Among the remaining comparisons with a conclusive validity judgment, 54.8% are judged invalid, compared with 63.7% before referral.