The Pulse
Political-Bias Warnings Cut ChatGPT’s Persuasive Effect by 28%
A U.S. experiment found that the strongest warning about ChatGPT’s partisan bias reduced belief change by 28%, though most of the measured persuasion effect remained. That warning combined a claim about skewed training data with political c
AI.info Team · Written with AI ·
A warning that ChatGPT was politically biased reduced the chatbot’s persuasive effect by 28% in a U.S. experiment compared with a control group. Published October 9 in PNAS Nexus, the study suggests that a user’s view of an AI system’s political credibility can shape how much its arguments shift beliefs.
Researchers Matthew DiGiuseppe and Joshua Robison tested ChatGPT (GPT-4.1) in a preregistered survey experiment with 2,144 U.S. participants. The finding concerns a warning about perceived bias—not evidence that the model actually favored one political party in the experiment.
A warning before the conversation
Participants discussed one of six economic-policy topics where their views differed from the consensus among orthodox academic economists. Each person took part in a three-round conversation with a nonreasoning version of ChatGPT, prompted to argue for the consensus view while remaining truthful. Participants who did not hold an eligible economic misconception were assigned a conversation about the value of an iPhone relative to Android devices.
The researchers assigned participants to one of four groups. A control group saw a blank screen before the chat; another received a general message about potential AI bias. The other two groups received warnings that ChatGPT had been trained on data skewed toward sources aligned with the opposing political party. The strongest warning also added partisan cues about OpenAI CEO Sam Altman, including information about his political donations and public statements, and showed participants an image of him with a political figure.
The warning weakened persuasion, but did not erase it
The strongest out-party-bias warning reduced the measured persuasion effect by 28% compared with the control group; the study reports a 95% confidence interval of 16% to 40%. Participants who received the warning still updated their beliefs, at roughly three-quarters the level of the control group.
“Even under this adversarial framing, the majority of the persuasion effect survived.”
Matthew DiGiuseppe and Joshua Robison, study authors
The warning also changed the tone of the exchange. Analysis of conversation transcripts found that warned participants pushed back more and responded less receptively, rather than simply leaving the conversation. The study’s abstract also reports lower trust in and willingness to use AI among participants who received the political-bias message.
What the experiment can—and cannot—show
The experiment tests the effect of a short credibility warning in a particular setting: a brief ChatGPT conversation about economic-policy misconceptions among U.S. survey participants. It does not establish how a similar warning would affect longer conversations, other subjects, different AI systems or people outside the United States.
Nor does the 28% figure measure ChatGPT’s underlying political bias. The researchers manipulated what participants were told about the model before they spoke with it, then measured how their beliefs changed. The result concerns the influence of perceived partisan alignment on persuasion, not the presence or size of any actual partisan slant in the chatbot.
Credibility becomes part of the interaction
The study’s authors describe perceived partisan alignment as a factor that conditions, but does not eliminate, the persuasive power of conversational AI. Their results suggest that a chatbot’s response alone does not determine whether a user accepts its argument: the user’s perception of the source also matters.
For this experiment, the concrete result is that a warning about bias made participants more skeptical and reduced belief change, while leaving most of the measured persuasive effect intact.