When Models Defer to Wrong Answers: A Robustness Audit of Source-Attributed Cues in Multiple-Choice QA

AI summary

Overview

Research area: Natural Language Processing — robustness and trustworthiness of multiple-choice question answering, with a focus on source-attributed cues (claims about what another source answered) and multilingual evaluation.

Technical level: Intermediate. The core idea is easy to grasp, but the paper's central metric, controlled condition design, and multilingual evaluation setup assume familiarity with LLM benchmarking practice.

Scope: A controlled robustness audit of how four instruction-following models change their multiple-choice answers when a wrong option is attributed to a named or implied source, evaluated across two benchmarks and five languages.

What This Paper Is About

Language models are frequently given a question alongside a claim about what some other source — an expert, or simply "most people" — answered. This paper asks whether such claims can push a model off an answer it had already selected on its own, and whether the wording of the claim matters. The goal is to measure that instability rigorously by holding the wrong answer constant and varying only the framing that accompanies it.

Key Contributions

1. A new measurement, NC-MCAR (neutral-conditioned misleading cue adoption rate). It counts only trials where the model first chose the correct answer under a neutral prompt, then switched to a specific wrong option when a cue was attached to it. This isolates genuine instability rather than ordinary error.
2. A controlled condition design. The same wrong option and the same final instruction are reused across misleading conditions; only the cue template attached to that option changes, making attribution-wording the sole manipulated variable.
3. A multilingual, multi-model audit. Four instruction-following models are evaluated on MMLU-Pro and IndicMMLU-Pro in English, Hindi, Bengali, Tamil, and Telugu, totaling roughly 220,000 model outputs.
4. Documentation of a specific failure mode. The paper establishes that a bare, unverified source claim can outweigh an answer the model had previously given that was consistent with the task evidence, under forced-choice prompting.

Main Findings

• Expert attribution is far more destabilizing than majority attribution. The expert template produced 41.1% aggregate NC-MCAR, compared with 12.5% for the majority template — despite both conditions using the same wrong option and the same final instruction.
• The effect is about framing, not answer content. Because the wrong option and closing instruction were held fixed, the divergence between the 41.1% and 12.5% figures is attributable to the cue template itself.
• Models do not simply collapse on every prompt. Filler accuracy stayed well above expert-wrong accuracy, indicating the effect is selective rather than a general breakdown in task performance.
• Models can still follow correct cues. Correct-cue prompts yielded high valid-response accuracy, so the instability is not a blanket refusal or inability to use source information.
• The phenomenon spans languages and benchmarks. The audit covers MMLU-Pro and IndicMMLU-Pro across English, Hindi, Bengali, Tamil, and Telugu, so this is not an English-only artifact.
• Interpretive caution is explicit. The authors frame NC-MCAR as a measure of answer instability, not as proof the model "knew" the answer or that all deference to a source is irrational.

Methodology in Plain English

The researchers built a controlled comparison. For each question, they picked one wrong answer and kept it fixed. Then they attached different cue templates to that same wrong answer — for example, framing it as coming from an expert versus from a majority of people — while keeping the final instruction identical across conditions.

To avoid counting ordinary mistakes, they first ran each model on a neutral version of the question with no cue. Only when the model answered correctly there did they look at what happened when a cue was introduced. A "switch" to the pre-selected wrong option on those trials is what feeds the NC-MCAR metric. This filters out cases where the model simply did not know the answer to begin with.

They ran this procedure across four instruction-following models, two multiple-choice benchmarks, and five languages, accumulating about 220,000 outputs. They also tracked filler accuracy and performance on correct-cue prompts as controls, to check whether the models were broadly failing or specifically deferring.

Why This Matters

Impact on research: The work reframes a familiar concern — models caving to social pressure — as a measurable, template-sensitive property with a metric that other researchers can reuse. Holding the wrong answer and instruction constant is a methodological contribution in its own right, since it makes cue wording an isolated variable. It also extends the sycophancy and grounding literature into multilingual territory, which has been underrepresented.

Real-world applications:

• Retrieval-augmented and tool-using systems, where a model is handed content labeled as coming from a source and must decide how much weight to give that label.
• AI-assisted decision support in medicine, law, and finance, where a misattributed "expert" claim could override a correct model judgment at exactly the wrong moment.
• Multilingual deployment, where the same deference behavior appears in Hindi, Bengali, Tamil, and Telugu, affecting users beyond English-speaking markets.
• Benchmark and evaluation design, since reported accuracy can be inflated or deflated depending on what provenance cues appear in the prompt.

Industry relevance: Any product that pipes external content into an LLM prompt — search assistants, enterprise copilots, customer support bots, document Q&A — is exposed to this failure mode. The finding that a one-line, unverified attribution can flip a correct answer is directly actionable for prompt design, output verification, and evaluation suites.

Future Directions

1. Testing open-ended and free-form generation. The audit is limited to forced-choice prompts; whether the same deference appears when models must produce answers rather than select them remains open.
2. Understanding the mechanism. Why does an expert label carry roughly three times the pull of a majority label? Explanations might involve training data priors, instruction-tuning dynamics, or learned deference to authority.
3. Measuring calibrated deference. The authors note that not all deference is irrational. Distinguishing appropriate trust in a reliable source from reflexive capitulation is a natural next step.
4. Mitigations and generalization. Candidate fixes — provenance-aware prompting, consistency checks, or training interventions — need evaluation, as does whether the effect holds for additional models, languages, and cue phrasings beyond those tested.

Target Audience

Researchers and practitioners working on LLM robustness, trustworthiness, and evaluation; engineers building retrieval-augmented or tool-augmented systems that inject external claims into prompts; and multilingual NLP teams who need evidence about how these behaviors travel across languages. Readers with basic familiarity with benchmark evaluation will get the most from it, though the central argument is accessible to a general technical audience.

Authors’ abstract

Language models often receive a question together with a claim about what another source answered. We audit whether such claims destabilize answers in multiple-choice question answering. For each item, we hold one wrong option fixed across misleading conditions and vary the cue template attached to it. We introduce \emph{neutral-conditioned misleading cue adoption rate} (NC-MCAR), which measures switches to that option only on valid cued trials where the same model first selected the gold answer under a neutral prompt. This is a measure of answer instability, not proof that the model knew the answer or that all deference is irrational. We evaluate four instruction-following models on MMLU-Pro and IndicMMLU-Pro in English, Hindi, Bengali, Tamil, and Telugu. Across 220{,}000 outputs, the expert template yields 41.1\% aggregate NC-MCAR, compared with 12.5\% for the majority template. These two conditions use the same wrong option and final instruction. Filler accuracy remains well above expert-wrong accuracy, while correct-cue prompts have high valid-response accuracy. The audit documents answer instability relevant to grounding under the tested forced-choice prompts: a bare, unverified source claim can outweigh an answer that was previously consistent with the task evidence.