When Thinking Hurts: Epistemic Signals in the Reasoning Chains of Visual Language Models
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Uncertainty quantification for visual language models (VLMs) conventionally targets the answer token distribution. We provide the first three-family empirical characterisation of answer entropy behaviour in thinking-mode VLMs. Running four models on identical POPE adversarial samples, we find three qualitatively distinct patterns: Qwen3-VL-8B-Thinking shows complete collapse (ans H AUROC = 0.492); GLM-4.1V-9B-Thinking shows no collapse (0.716); and InternVL3-8B shows selective thinking (chains on only 50% of samples, ans H = 0.675 full / 0.602 thinking-only). Across all three thinking-mode models, thinking chain entropy outperforms answer entropy on the subset where chains are generated (0.647, 0.759, 0.608 vs. 0.492, 0.716, 0.602 respectively), suggesting chain signals are the more reliable predictor whenever chains are present. This holds strongly for Qwen and GLM, but with only marginal and statistically unreliable advantage for InternVL3 (n_FP = 17). A 300-sample VQAv2 pilot confirms chain entropy (0.680) outperforms answer entropy (0.595) on VQAv2 questions, with the gap largest for free-form answers (0.733 vs. 0.467). On harder reasoning tasks (HallusionBench) both Qwen models show moderate signal (approx. 0.64), consistent with incomplete pre-commitment on difficult questions. We additionally document structured abstention affecting 12-22% of queries with asymmetry toward absent-object queries, and a practical abstention gate raising accuracy from 71.0% to 93.8% at 62.7% coverage with no additional inference cost.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 8 of its 15 references that are in the map. First posted on arXiv 9 Jul 2026.
Builds on
8 of 15 references.
Closest ancestors · 2
Same direction, newest first
- 2026The Cost of Reasoning: Chain-of-Thought Induces Overconfidence in Vision-Language Models17 Mar
- 2025Knowing the Answer Isn't Enough: Fixing Reasoning Path Failures in LVLMs6 Dec
Other references · 6
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 84
- 2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimationcited by 71
- 2026Uncertainty Quantification for Multimodal Large Language Models with Incoherence-adjusted Semantic Volumecited by 7
- 2025Think Right: Learning to Mitigate Under-Over Thinking via Adaptive, Attentive Compressioncited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026The Chain Holds, the Answer Folds: Trace-Answer Dissociation in Reasoning Models Under Adversarial Pressure
- 2025When to Think and When to Look: Uncertainty-Guided Lookback
- 2025Journey Before Destination: On the importance of Visual Faithfulness in Slow Thinking
- 2026Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models
- 2025Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs
- 2026Counterfactual Tests for Measuring Chain-of-Thought Faithfulness in Visual Language Models
- 2024Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation
- 2026Decoupling the Effect of Chain-of-Thought Reasoning: A Human Label Variation Perspective