Arguments that Alter Minds: LLM Rationales Sway Human (and LLM) Notions of Plausibility

AI summary

Overview

• Research area: Natural Language Processing / commonsense reasoning, LLM-generated explanations, human-AI interaction, and the persuasiveness of machine-generated rationales.
• Technical level: Intermediate. The paper is readable without deep technical background, but assumes familiarity with multiple-choice commonsense benchmarks (Social IQA, CommonsenseQA), Likert-scale annotation, and basic statistical testing.
• Scope (one sentence): The paper measures how plausibility judgments from humans and from 17 LLMs change when an LLM-written argument for (PRO) or against (CON) a multiple-choice commonsense answer is shown alongside it.

What This Paper Is About

Commonsense multiple-choice questions rarely have answers that are definitively true or false; answers instead sit on a continuum of plausibility, and almost any answer can be argued for or against by pointing to circumstances in which it would hold. This paper asks whether an LLM can write such arguments convincingly enough to move those plausibility judgments — for human raters and for other LLMs. The authors build on prior work that collected baseline plausibility ratings for question-answer pairs, then add LLM-generated PRO, CON, and combined PRO+CON rationales and measure how ratings shift.

Key Contributions

1. A new dataset of rationale-conditioned plausibility judgments. The authors collect 3,000 new human plausibility ratings (in addition to 1,000 previously collected baseline ratings) and 13,600 LLM ratings (in addition to ratings for the baseline setting), covering 100 questions (50 from SIQA and 50 from CQA), 200 question-answer pairs, and 800 question-answer-rationale tuples across four rationale settings (NO, PRO, CON, PRO+CON).
2. A demonstration that LLM-written rationales systematically shift plausibility judgments. Both human and LLM ratings change significantly with rationales relative to the no-rationale setting, across gold-label and distractor answers and across both datasets.
3. Identification of a bimodal effect of PRO rationales on humans. PRO rationales raise mean ratings for implausible distractor answers but lower mean ratings for highly plausible gold-label answers — a pattern the authors state has not been documented in prior work on LLM-generated explanations.
4. A causal regression analysis of what drives the shifts. Using OLS regression, the authors quantify a strong anchoring effect of the initial plausibility rating and show that CON rationales carry larger-magnitude coefficients than PRO rationales, while rationale length has only a weak relationship with rating change.

Main Findings

• Overall influence: Human and LLM plausibility judgments both shift significantly in the presence of PRO, CON, and PRO+CON rationales relative to the NO rationale setting. For LLMs, all p-values from chi-squared tests of homogeneity comparing rating distributions were below 0.0001. For humans, chi-squared p-values were 0.069, 2.17E-9, and 0.001 for the PRO, CON, and PRO+CON settings on SIQA, and 0.001, 9.24E-7, and 0.0006 for CQA.
• PRO raises, CON lowers — on average: PRO rationales raise mean plausibility ratings overall (for example, human SIQA overall mean rose to 3.33, a change of +0.1; CQA overall mean was 3.39, a change of 0.0), while CON rationales lower them (human SIQA overall 2.28, a change of −0.94; CQA overall 2.62, a change of −0.76).
• A bimodal human reaction to PRO rationales: For humans, PRO rationales raise mean ratings on distractor answers (SIQA distractor 2.81, +0.47; CQA distractor 2.86, +0.45) but lower them on gold-label answers (SIQA gold 3.84, −0.26; CQA gold 3.91, −0.44). The authors hypothesize a Gricean Maxim of Quantity explanation: arguing that an obvious fact is "plausible" may be read as arguing it is at most plausible, introducing doubt.
• LLMs do not show that same split for PRO: For OpenAI and Non-OpenAI models, PRO rationales raise mean plausibility for gold-label and distractor answers alike (for example, OpenAI CQA gold 4.55, +0.25, and distractor 3.27, +1.03).
• CON rationales hit gold answers hardest: With CON rationales, both answer types drop, with much larger drops for gold-label answers. The authors note the drop for gold answers exceeds one full Likert scale point on average (SIQA gold 2.72, −1.39; CQA gold 3.28, −1.08), the largest changes observed for any setting.
• PRO+CON produces mixed, averaged effects: Ratings drop for the gold label (for example, human SIQA gold 3.36, −0.75; CQA gold 3.7, −0.65) but rise or hold for the distractor (human SIQA distractor 2.39, +0.05; OpenAI SIQA distractor 2.42, +0.16). The authors suggest PRO+CON pushes ratings toward the center of the scale, or that the result sits between the PRO and CON shifts.
• Direction-of-change percentages differ by rater group: For SIQA with PRO rationales, human ratings increased in 28% of cases, decreased in 50%, and were unchanged in 22%; OpenAI models increased in 40%, decreased in 2%, unchanged in 58%; Non-OpenAI models increased in 61%, decreased in 1%, unchanged in 38%. With CON rationales on SIQA, the increase/decrease/no-change split was 2%/69%/29% for humans, 0%/84%/16% for OpenAI models, and 5%/66%/29% for Non-OpenAI models.
• OpenAI models are more sensitive: OpenAI models were more sensitive to PRO, CON, and PRO+CON rationales than Non-OpenAI models, which the authors hypothesize is because the rationales were generated by GPT-4o.
• Strong anchoring effect: The NO Rationale plausibility rating is a statistically significant predictor (p < 0.05) of the rating change for humans, OpenAI models, and Non-OpenAI models. All coefficients are negative (for example, human SIQA gold label −0.4581, distractor −0.5495; human CQA gold −0.5092, distractor −0.4983), meaning higher initial plausibility yields smaller subsequent change. The anchoring effect is more pronounced for distractors than for gold-label answers in all rater groups.
• Rationale type effects in the regression: CON rationales carry large negative coefficients that sharply reduce plausibility for correct answers (for example, OpenAI SIQA gold −1.2592, distractor −1.1177; human SIQA gold −0.8823, distractor −0.8525). PRO rationales increase plausibility but with smaller magnitude (human SIQA gold 0.2417, distractor 0.1082). PRO+CON typically carries small negative coefficients.
• Length does not matter much: Regression coefficients for rationale length (number of words) are close to zero across settings, indicating only a weak linear relationship with rating scores.

Methodology in Plain English

The authors started from an existing collection of Likert-scale plausibility ratings for individual question-answer pairs from two commonsense benchmarks, Social IQA (SIQA) and CommonsenseQA (CQA), gathered with no arguments attached (the NO rationale setting). They randomly sampled 100 questions — 50 from each benchmark — and for each question selected the dataset's gold label plus a randomly chosen distractor answer. Using GPT-4o, they generated two kinds of rationales per question-answer pair: a PRO rationale arguing the answer is plausible and a CON rationale arguing it is implausible.

Before running the main study, they ran a small preference study on 40 randomly sampled question-answer pairs from the SIQA validation split, generating rationales with GPT-4o, GPT-4o-mini, Llama 3.1-8B-Instruct, and Llama 3.1-70B-Instruct, and asking 4 human annotators which rationale best explained the plausibility or implausibility of the answer and was most believable. GPT-4o received the most votes and was used for the full study.

For the human study, they used the 1–5 scale (1-Impossible, 2-Technically Possible, 3-Plausible, 4-Likely, 5-Very Likely), collecting 5 judgments for each of the PRO, CON, and PRO+CON settings — 15 new judgments per selected question-answer pair. Annotators were recruited through Prolific, paid $15/hour, and screened for US location, English as primary language, no literacy difficulties, at least an undergraduate degree, a 99–100% approval rate, and a 50–50 male/female split. Each annotator could judge at most one question-answer-rationale tuple per question per rationale type. Total human annotation cost was $1,000.

For the LLM study, they replicated the same stimuli across 17 models, split into 8 OpenAI models (GPT 3.5, 4, 4-turbo, 4o, 4o-mini, 4.5-preview, o1, o3-mini) and 9 Non-OpenAI models (LLaMA-3.1-Instruct 8B and 70B, LLaMa-3.2-Instruct 3B, LLaMa-3.3-Instruct 70B, Mistral Instruct 8x22B, Yi-1.5-Chat 6B/9B/34B, and DeepSeek-R1 Qwen-32B). This split was designed to mitigate self-preference bias, since GPT-4o wrote the rationales. Non-OpenAI models ran on eight NVIDIA RTXA5000 GPUs with 24 GPU hours allocated; OpenAI models ran on CPU with 8 hours allocated. Decoding used a maximum generation length of 50 tokens, temperature 1 for GPT-4o rationale generation, and temperature 0 (OpenAI) and 0.1 (Non-OpenAI) for plausibility ratings. No hyperparameter search was performed and all results come from a single run.

Analysis consisted of comparing mean ratings across conditions, chi-squared tests of homogeneity to test distribution shifts, and OLS regressions with rationale type and the initial NO-rationale rating as features and the change in rating as the dependent variable (with the initial rating centered on a [−2, 2] scale).

Why This Matters

Impact on research. The work introduces a novel use of LLMs as tools for probing human cognition, and it challenges the assumption that commonsense reasoning is a domain where human judgment is immune to machine influence. It also shows that human and LLM raters do not respond identically to the same explanations, which has implications for how commonsense reasoning benchmarks are built and how annotation quality is assumed to hold up under framing.

Real-world applications:

• Human-AI collaboration and decision support: shows that LLM rationales can be used deliberately to challenge a user's reasoning, stress-test arguments, and surface alternative perspectives.
• Safety and manipulation risk: demonstrates that LLMs can steer people toward more implausible inferences through persuasive argumentation, and that users may not realize when this has happened.
• Benchmark and annotation design: indicates that the presence or absence of an accompanying argument can change the ratings that benchmarks are built on.
• LLM evaluation: shows that models' own plausibility judgments are easily shifted by rationales, which matters when LLMs are used as judges or annotators.

Industry relevance. Any product that pairs a model's answer with an explanation — assistants, search, recommendation, education, or decision-support tools — inherits the persuasion effect documented here. The authors argue this is more concerning, not less, in specialized or technical domains than in everyday commonsense, and they call for safeguards including transparency, bias mitigation, and mechanisms to detect and counteract manipulative uses.

Future Directions

1. Test other models and architectures. The paper notes that persuasive power and style can vary significantly across models; the study uses a single model (GPT-4o) to generate rationales, and the authors hope future work examines how much different models influence human and LLM plausibility judgments.
2. Move beyond English commonsense. The scope is limited to two English-language commonsense datasets; the authors suggest exploring technical, scientific, legal, and multilingual settings where arguments are more constrained by factual evidence.
3. Study annotator demographics and culture. Human annotators were US-based English speakers on Prolific; the authors plan work on how cognitive abilities, age, grade level, and other demographic characteristics influence responses to LLM-generated explanations, and note that common sense itself can be culturally situated.
4. Address the detection and mitigation gap. The paper raises the open problem of building safeguards that let users recognize when their beliefs are being steered, and mechanisms to counteract harmful manipulative uses of LLM rationales.

Target Audience

Researchers in NLP and commonsense reasoning, explainability and human-AI interaction, and AI safety researchers studying persuasion and influence. It is also relevant to practitioners who design systems that present model-generated justifications to users, and to anyone responsible for benchmark construction, crowdsourced annotation, or LLM-as-judge evaluation, since it shows that the presence of an argument can measurably change the judgments collected.

Authors’ abstract

We investigate the degree to which human (and LLM) plausibility judgments of multiple-choice commonsense benchmark answers are subject to influence by (im)plausibility arguments for or against an answer, in particular, using rationales generated by LLMs. We collect 3,000 plausibility judgments from humans and another 13,600 judgments from LLMs. Overall, we observe increases and decreases in mean human plausibility ratings in the presence of LLM-generated PRO and CON rationales, respectively, suggesting that, on the whole, human judges find these rationales convincing. Experiments with LLMs reveal similar patterns of influence. Our findings demonstrate a novel use of LLMs for studying aspects of human cognition, while also raising practical concerns that, even in domains where humans are ``experts'' (i.e., common sense), LLMs have the potential to exert considerable influence on people's beliefs.