When Slower Isn't Truer: Inverse Scaling Law of Truthfulness in Multimodal Reasoning
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Reasoning models have attracted increasing attention for their ability to tackle complex tasks, embodying the System II (slow thinking) paradigm in contrast to System I (fast, intuitive responses). Yet a key question remains: Does slower reasoning necessarily lead to more truthful answers? Our findings suggest otherwise. We conduct the first systematic study of the inverse scaling law in slow-thinking paradigms for multimodal reasoning. We find that when confronted with incomplete or misleading visual inputs, slow-thinking models are more prone to fabricating plausible yet false details to justify untruthful reasoning. To analyze this behavior, we construct a 5,000-sample hierarchical prompt dataset annotated by 50 human participants. The prompts progressively increase in complexity, revealing a consistent pattern: slower reasoning models tend to follow depth-first search (DFS) thinking, persistently exploring flawed premises, while faster chat models favor breadth-first search (BFS) inference, showing greater caution under uncertainty. These findings reveal a critical vulnerability of reasoning models: while effective in structured domains such as math, their DFS-style reasoning becomes fragile when confronted with ambiguous, multimodal inputs.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 19 of its 35 references that are in the map. First posted on arXiv 26 May 2025.
Builds on
19 of 35 references.
References · 19
- 2024GPT-4o System Cardcited by 4,588
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
- 2016Concrete Problems in AI Safetycited by 1,642
- 2023PaLM 2 Technical Reportcited by 1,162
- 2023Large Language Models are not Fair Evaluatorscited by 1,085
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
- 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
- 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
- 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
- 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
- 2018Supervising strong learners by amplifying weak expertscited by 145
7 more
- 2024RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthinesscited by 131
- 2024Critique-out-Loud Reward Modelscited by 106
- 2024OffsetBias: Leveraging Debiased Data for Tuning Evaluatorscited by 94
- 2024Visual Hallucinations of Multi-modal Large Language Modelscited by 68
- 2024MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Modelscited by 53
- 2025Multimodal RewardBench: Holistic Evaluation of Reward Models for Vision Language Modelscited by 44
- 2025Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedbackcited by 30
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024AtomThink: Multimodal Slow Thinking with Atomic Step Reasoning
- 2025Rethinking External Slow-Thinking: From Snowball Errors to Probability of Correct Reasoning
- 2025Investigating Inference-time Scaling for Chain of Multi-modal Thought: A Preliminary Study
- 2025Compose and Fuse: Revisiting the Foundational Bottlenecks in Multimodal Reasoning
- 2025Downscaling Intelligence: Exploring Perception and Reasoning Bottlenecks in Small Multimodal Models
- 2025Journey Before Destination: On the importance of Visual Faithfulness in Slow Thinking
- 2026Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do
- 2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Law