Probing Multimodal Large Language Models on Cognitive Biases in Chinese Short-Video Misinformation

AI summary

Overview

Research area: Evaluation of Multimodal Large Language Models (MLLMs) on misinformation detection and cognitive bias, focused on Chinese short-video platforms.

Technical level: Intermediate — readers should know what MLLMs, Chain-of-Thought prompting, OCR and ASR are, but the experimental design is explained clearly.

Scope: The paper builds a 200-video annotated benchmark of Chinese short-video health misinformation, evaluates eight frontier MLLMs across five input settings, and measures whether popularity signals and channel verification status induce herd effects and authority bias in model judgments.

What This Paper Is About

Short-video platforms like Douyin and Kuaishou spread misinformation using persuasive audiovisual cues, professional-looking aesthetics, and plausible but unverified "experiments." Existing benchmarks mostly target text news or synthesized data, so they cannot test whether a model reasons about internal logical failures rather than just retrieving external facts.

The paper's goal is to build a small but rigorously verified dataset where the reason each video is wrong is annotated, then use it to ask which modality matters most, whether model reasoning matches human evidence-based logic, and how far models mirror human cognitive biases such as herd effects and authority bias.

Key Contributions

1. A manually verified benchmark (Fine-VDK) of 200 Chinese short videos from Douyin and Kuaishou, evenly split into 100 misinformation and 100 truthful videos, drawn from professional fact-checking channels on Bilibili and Douyin and from the Piyao rumor-refuting platform.
2. A fine-grained error taxonomy grounded in collected evidence (academic papers, national standards, legal documents, common knowledge/Wikipedia): experimental errors (n=16), logical fallacies (n=55), and fabricated claims (n=29).
3. A five-setting evaluation framework — Claim, Textual, Aural, Visual, and Multimodal — with a 7-point Likert trustworthiness judgment converted into a normalized Belief Score, applied to eight frontier MLLMs.
4. Two cognitive-bias experiments: a herd/popularity effect study that scales view, like, share and comment counts across nine levels, and an authority-bias study that manipulates channel verification status (Unverified, Yellow-Individual, Blue-Enterprise, Red-Organization), including a random ID-permutation control.

Main Findings

• Best and worst models: Gemini-2.5-Pro achieves the highest multimodal Belief Score at 71.5/100, while o3 performs worst at 35.2. Gemini-2.5-Pro is described as balanced across the true and false subsets with no significant weaknesses.
• o3 fails on the true subset: o3's low score comes from a conservative reasoning style — its false-subset performance is mediocre (54.0) but its true-subset multimodal score is only 16.3, and CoT analysis shows it avoids affirmative judgments about video correctness.
• Qwen models show a "true" label bias: Qwen models systematically predict "true," scoring high on the true subset but much lower on the false subset, and this bias is amplified by visual content in the Visual and Multimodal settings.
• Modality ordering: Average Belief Scores are comparable across Textual (55.0), Visual (55.3) and Multimodal (55.8), while Aural is roughly 10 points lower at 45.9. The Claim setting peaks at 65.3 but is treated as a reference, not a measure of model capability.
• Multimodal does not consistently help belief judgment: Multimodal inputs are often surpassed by visual context alone for belief scores; however, the reasoning-quality ratings (scored 1–4 by Gemini-2.5-Pro for coverage of the annotated error reason) are higher in the Multimodal setting than in the Claim setting.
• Logical fallacies are hardest: On the false subset in the Multimodal setting, logical fallacies score 45.9 versus 51.8 and 53.0 for the other error types.
• Domain difficulty varies: Health, Medical & Lifestyle is the least challenging domain (average 66.5 multimodal), while Agri-Livestock & Fresh Produce is the most difficult (average 41.5). Chemicals, Appliances & Materials shows the largest false/true imbalance (33.7 on false versus 59.9 on true), attributed to claims that were historically true but rendered obsolete by newer materials and standards.
• Herd effect present but counterintuitive: Raising popularity metrics across nine levels (from 10B/550M/373M/50M down to 100/5/3/0) does not lower belief scores on false videos — scores instead increase, suggesting engagement metrics boost confidence in initial judgments rather than inducing trust.
• Authority bias confirmed: On the false subset, the highest verification level (Organization) yields an average Belief Score of 36.8 versus 73.0 for Unverified accounts, meaning verified channels carry more deceptive content — and models still trust them more.
• IDs influence judgments independently of content: When channel IDs are randomly permuted, "True"-set IDs consistently lower predicted scores relative to "False"-set IDs regardless of the video's actual label. GPT-4o and Qwen-2.5 rate only 2.25% and 1.12% of false-set IDs as reliable, versus 28.8% and 19.2% of true-set IDs.
• Statistical robustness: On the Agri-Livestock & Fresh Produce subset (n=41), a t-test between Gemini-2.5-Pro (Mean=88.6, SD=27.5) and GPT-4o (Mean=69.1, SD=32.0) gives t=2.96, df=40, p=0.0025.
• Negation effects: The false subset contains 16 negative and 84 affirmative claims; the true subset contains 62 negative and 38 affirmative claims. Inverting claims raises scores for affirmations-to-negations (51.4 to 58.0 on the false set) and lowers scores for negations-to-affirmations (50.8 to 35.3 on the false set).
• Efficiency gap: Text-based settings consume under 1.2k tokens on average per video, while visual processing varies drastically — Gemini-2.5-Pro uses about 6.5k tokens for the Multimodal setting, whereas Qwen-VL-Max and Seed-1.6 exceed 40k.
• Adding OCR text to Multimodal helps some models: Explicitly adding OCR screen text improves overall scores for models such as GPT-4o (59.3 to 62.2), o3 and Gemini-2.5-Flash.

Methodology in Plain English

The researchers started from the debunking side: they identified 100 fact-checking cases from professional channels that explicitly refute specific claims and point to evidence. From those debunking points they traced back the original misinformation videos on Douyin and Kuaishou, then built a matching control set of 100 truthful videos by taking verified claims from Piyao and finding corresponding promotional or educational videos with similar topics and visual style.

Every misinformation video was analyzed by hand: the team reconstructed its logical chain, recorded the error reason, and classified the supporting evidence into academic papers, national standards, legal documents, or common knowledge/Wikipedia. This led to the three error types. Videos were also tagged into four topic domains and annotated with metadata including cover image, title, channel ID, likes and shares, release date, and URL, plus Douyin's official channel verification status.

Videos were trimmed of ads and intros, averaging 53.5 ± 39.3 seconds (median 44.5s, range 2.9–278.3s), then split into three streams: visual frames sampled at 0.5 FPS and capped at 32 frames (average 21.8 ± 9.1 frames), on-screen text via commercial OCR (average 229.0 ± 175.4 tiktoken tokens) and ASR transcripts (average 306.0 ± 239.0 tiktoken tokens), both manually audited.

Eight MLLMs were prompted with Chain-of-Thought reasoning and asked to rate each video on a 7-point Likert scale (1 = confident it is factual, 7 = confident it is misinformation, 4 = neutral). The authors define a Belief Score that only rewards skepticism toward misinformation and belief toward factual content, normalizing the rating by the distance from the neutral point. A separate Gemini-2.5-Pro pass rated whether each model's reasoning actually covered the human-annotated error reason on a four-level scale. For the bias experiments, popularity statistics were injected as text prompts and channel IDs were revealed or randomly permuted.

Why This Matters

Impact on research. Prior multimodal misinformation benchmarks rely on general news verification or on synthesized content, which cannot probe whether a model detects internal logical failure. This dataset supplies evidence-grounded error labels and shows that multimodal input does not automatically improve truth judgments, while reasoning quality does improve — a distinction that matters for how future benchmarks are designed. It also supplies a controlled demonstration that social metadata, not just content, drives model verdicts.

Real-world applications

• Content moderation systems on short-video platforms that must distinguish genuine medical claims from viral but flawed "experiments."
• Fact-checking assistance tools that triage which videos a human reviewer should examine first.
• Platform-policy evaluation, since results show verified-organization channels carried more deceptive content on this dataset yet received more model trust.
• Consumer-facing misinformation warnings that account for how popularity and verification badges skew perception.

Industry relevance. The token-consumption results (roughly 6.5k tokens for Gemini-2.5-Pro versus over 40k for Qwen-VL-Max and Seed-1.6 on the same multimodal input) quantify the cost of frame-heavy pipelines. The finding that verified IDs bias model verdicts is directly relevant to platforms that display verification badges alongside AI-generated content labels. The code and dataset are released on GitHub and HuggingFace.

Future Directions

• Line of work toward full-multimodal and biased-robust models. The paper calls for MLLMs that are both capable of understanding multimodal content and resilient to sophisticated deceptive tactics on social media.
• Dynamic frame sampling. The fixed 0.5 FPS, 32-frame cap may miss subtle visual manipulations or micro-experiments that occur in very short intervals; the authors suggest sampling rates adapted to visual complexity.
• Data contamination auditing. The underlying debunking facts (national standards, widely circulated myth corrections) may already exist in training corpora, which could inflate measured reasoning ability if models are retrieving memorized facts rather than performing zero-shot analysis.
• Cross-linguistic and cross-cultural generalization. The study is limited to the Simplified Chinese Douyin and Kuaishou ecosystem; whether the observed herd and authority effects transfer to other languages and platforms is untested.
• Scaling beyond 200 videos. The authors defend the sample size on annotation-quality grounds and note they avoid drawing conclusions from atomic intersections such as fabricated claims within the agri-livestock true subset, analyzing marginal distributions instead.

Target Audience

Researchers working on multimodal model evaluation, misinformation detection, and AI safety; platform trust-and-safety teams designing moderation or labeling systems; and social scientists studying how credibility cues such as verification badges and engagement counts influence both human and machine judgment. Readers who want a concrete, evidence-grounded benchmark design with a taxonomy of reasoning failures will benefit most, as will those interested in the use of experimental psychology paradigms to probe model behavior.

Authors’ abstract

Short-video platforms have become major channels for misinformation, where deceptive claims frequently leverage visual experiments and social cues. While Multimodal Large Language Models (MLLMs) have demonstrated impressive reasoning capabilities, their robustness against misinformation entangled with cognitive biases remains under-explored. In this paper, we introduce a comprehensive evaluation framework using a high-quality, manually annotated dataset of 200 short videos spanning four health domains. This dataset provides fine-grained annotations for three deceptive patterns-experimental errors, logical fallacies, and fabricated claims-each verified by evidence such as national standards and academic literature. We evaluate eight frontier MLLMs across five modality settings. Experimental results demonstrate that Gemini-2.5-Pro achieves the highest performance in the multimodal setting with a belief score of 71.5/100, while o3 performs the worst at 35.2. Furthermore, we investigate social cues that induce false beliefs in videos and find that models are susceptible to biases like authoritative channel IDs.