# Probing Multimodal Large Language Models on Cognitive Biases in Chinese Short-Video Misinformation

arXiv: https://arxiv.org/abs/2601.06600 · HTML: https://mlsanity.com/paper/2601.06600/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 27 of its 69 references that are in the map.
- First posted on arXiv 10 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)

## Builds on (most cited first, 27 of 27)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [LVBench: An Extreme Long Video Understanding Benchmark](https://mlsanity.com/paper/2406.08035/) (2024)
- [Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge](https://mlsanity.com/paper/2410.02736/) (2024)
- [Humans or LLMs as the Judge? A Study on Judgement Biases](https://mlsanity.com/paper/2402.10669/) (2024)
- [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018)
- [MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding](https://mlsanity.com/paper/2406.14515/) (2024)
- [Simulating Opinion Dynamics with Networks of LLM-based Agents](https://mlsanity.com/paper/2311.09618/) (2023)
- [OASIS: Open Agent Social Interaction Simulations with One Million Agents](https://mlsanity.com/paper/2411.11581/) (2024)
- [Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse](https://mlsanity.com/paper/2410.21333/) (2024)
- [Cognitive Bias in Decision-Making with LLMs](https://mlsanity.com/paper/2403.00811/) (2024)
- [An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLM](https://mlsanity.com/paper/2403.18406/) (2024)
- [Exploring Prosocial Irrationality for LLM Agents: A Social Cognition View](https://mlsanity.com/paper/2405.14744/) (2024)
- [FakingRecipe: Detecting Fake News on Short Video Platforms from the Perspective of Creative Process](https://mlsanity.com/paper/2407.16670/) (2024)
- [Towards Automatic Detection of Misinformation in Online Medical Videos](https://mlsanity.com/paper/1909.01543/) (2019)
- [InfiniBench: A Benchmark for Large Multi-Modal Models in Long-Form Movies and TV Shows](https://mlsanity.com/paper/2406.19875/) (2024)
- [Don't Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluation](https://mlsanity.com/paper/2505.16222/) (2025)
- [Fact-R1: Towards Explainable Video Misinformation Detection with Deep Reasoning](https://mlsanity.com/paper/2505.16836/) (2025)
- [The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societies](https://mlsanity.com/paper/2509.18052/) (2025)
- [Misleading Metadata Detection on YouTube](https://mlsanity.com/paper/1901.08759/) (2019)
- [VideoScan: Enabling Efficient Streaming Video Understanding via Frame-level Semantic Carriers](https://mlsanity.com/paper/2503.09387/) (2025)
- [SOTOPIA-S4: a user-friendly system for flexible, customizable, and large-scale social simulation](https://mlsanity.com/paper/2504.16122/) (2025)
- [Are Large Language Models Sensitive to the Motives Behind Communication?](https://mlsanity.com/paper/2510.19687/) (2025)
- [Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biases](https://mlsanity.com/paper/2502.05849/) (2025)

## Built on by (most cited first, 2 of 2)

- [HumanStudy-Bench: Towards AI Agent Design for Participant Simulation](https://mlsanity.com/paper/2602.00685/) (2026)
- [When Seeing Is Not Believing -- A Benchmark for Search-Grounded Video Misinformation Detection](https://mlsanity.com/paper/2606.04098/) (2026)

## Abstract (arXiv)

Short-video platforms have become major channels for misinformation, where deceptive claims frequently leverage visual experiments and social cues. While Multimodal Large Language Models (MLLMs) have demonstrated impressive reasoning capabilities, their robustness against misinformation entangled with cognitive biases remains under-explored. In this paper, we introduce a comprehensive evaluation framework using a high-quality, manually annotated dataset of 200 short videos spanning four health domains. This dataset provides fine-grained annotations for three deceptive patterns-experimental errors, logical fallacies, and fabricated claims-each verified by evidence such as national standards and academic literature. We evaluate eight frontier MLLMs across five modality settings. Experimental results demonstrate that Gemini-2.5-Pro achieves the highest performance in the multimodal setting with a belief score of 71.5/100, while o3 performs the worst at 35.2. Furthermore, we investigate social cues that induce false beliefs in videos and find that models are susceptible to biases like authoritative channel IDs.