From Perception to Integration: Revisiting the Internal Dynamics of Reasoning in Vision-Language Models
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Vision-language models (VLMs) can answer simple visual questions, but often struggle when one question requires several visual judgments. We study this gap with controlled tasks for feature binding, numerosity, spatial relations, and amodal completion, together with a Composite task that combines them. Matched counterfactual image pairs isolate changes in the visual evidence needed to answer. Across four models, direct answers, hidden-state readouts, and state interventions show that the individual judgments can be made without explicit reasoning and that intervening on the corresponding states can affect the answer. During reasoning, the Composite answer becomes decodable from hidden states and usable from shortened traces, often before the model stops on its own. We train a small detector to predict this readiness and stop reasoning at that point. On MMStar and RealWorldQA, this reduces mean reasoning tokens by 79.1% and 74.5%, while average accuracy rises by 3.13 and 3.30 percentage points, respectively. These findings connect the internal development of answer readiness to a practical rule for allocating reasoning computation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 38 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
22 of 38 references.
Closest ancestors · 5
Same direction, newest first
- 2026Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do21 Jun
- 2025How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decoding27 Aug
- 2025MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency13 Feb
- 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning25 Mar
- 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb
Other references · 17
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
- 2025Dynamic Early Exit in Reasoning Modelscited by 219
- 2024How to use and interpret activation patchingcited by 214
- 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 159
- 2024Towards Interpreting Visual Information Processing in Vision-Language Modelscited by 122
- 2024Cross-modal Information Flow in Multimodal Large Language Modelscited by 67
- 2024Understanding Information Storage and Transfer in Multi-modal Large Language Modelscited by 49
- 2024VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Modelscited by 43
- 2021Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQAcited by 30
5 more
- 2025Knowing Before Saying: LLM Representations Encode Information About Chain-of-Thought Success Before Completioncited by 23
- 2023Probing Conceptual Understanding of Large Visual-Language Modelscited by 17
- 2025Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMscited by 13
- 2026When Is Thinking Enough? Early Exit via Sufficiency Assessment for Efficient Reasoningcited by 6
- 2026Pathways of Visual Information Flow in Vision-Language Modelscited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs
- 2026Seeing is not Enough: Vision-Language Models Perceive Evidence but Fail to Act
- 2026Seeing and Solving Are Not Enough for Vision-Language Models
- 2026Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models
- 2024Zero-Shot Visual Reasoning by Vision-Language Models: Benchmarking and Analysis
- 2025Fast or Slow? Integrating Fast Intuition and Deliberate Thinking for Enhancing Visual Question Answering
- 2026The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models
- 2026Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning