From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Multimodal Large Language Models (MLLMs) strive to achieve a profound, human-like understanding of and interaction with the physical world, but often exhibit a shallow and incoherent integration when acquiring information (Perception) and conducting reasoning (Cognition). This disconnect leads to a spectrum of reasoning failures, with hallucination being the most prominent. Collectively, these issues expose a fundamental challenge: the ability to process pixels does not yet confer the ability to construct a coherent, credible internal world model. To systematically dissect and address this challenge, this survey introduces a novel and unified analytical framework: ``From Perception to Cognition." We deconstruct the complex process of vision-language interactive understanding into two interdependent layers: Perception, the foundational ability to accurately extract visual information and achieve fine-grained alignment with textual instructions; and Cognition, the higher-order capability for proactive, multi-step, goal-oriented reasoning built upon this perceptual foundation, the core of which is the formation of a dynamic observe-think-verify reasoning loop. Guided by this framework, this paper systematically analyzes the key bottlenecks of current MLLMs at both layers. It surveys the landscape of cutting-edge methods designed to address these challenges, spanning from techniques that enhance low-level visual representations to those that improve high-level reasoning paradigms. Furthermore, we review critical benchmarks and delineate future research directions. This survey aims to provide the research community with a clear, structured perspective for understanding the intrinsic limitations of current MLLMs and to illuminate the path toward building next-generation models capable of deep reasoning and a genuine understanding of the world.
Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 175 of its 350 references that are in the map. First posted on arXiv 29 Sep 2025.
Builds on
175 of 350 references.
Closest ancestors · 71
Same direction, newest first
- 2025Explain Before You Answer: A Survey on Compositional Visual Reasoning24 Aug
- 2025Multimodal Chain of Continuous Thought for Latent-Space Reasoning in Vision-Language Models18 Aug
- 2025Ovis2.5 Technical Report15 Aug
- 2025SIFThinker: Spatially-Aware Image Focus for Visual Reasoning8 Aug
- 2025MathReal: We Keep It Real! A Real Scene Benchmark for Evaluating Math Reasoning in Multimodal Large Language Models8 Aug
- 2025CausalStep: A Benchmark for Explicit Stepwise Causal Reasoning in Videos22 Jul
- 2025Bootstrapping Grounded Chain-of-Thought in Multimodal LLMs for Data-Efficient Model Adaptation3 Jul
- 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoning2 Jul
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers30 Jun
- 2025VGR: Visual Grounded Reasoning13 Jun
- 2025CausalVQA: A Physically Grounded Causal Reasoning Benchmark for Video Models11 Jun
- 2025Socratic-MCTS: Test-Time Visual Reasoning by Asking the Right Questions10 Jun
38 more
- 2025VReST: Enhancing Reasoning in Large Vision-Language Models through Tree Search and Self-Reward Mechanism10 Jun
- 2025Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification8 Jun
- 2025MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical Chain-of-Thought Reasoning5 Jun
- 2025GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinking1 Jun
- 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoning29 May
- 2025ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering29 May
- 2025Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning26 May
- 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use25 May
- 2025v1: Learning to Point Visual Tokens for Multimodal Grounded Reasoning24 May
- 2025VLM-R3: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought22 May
- 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning21 May
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
- 2025Visual Agentic Reinforcement Fine-Tuning20 May
- 2025MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable Step-Level Supervision19 May
- 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May
- 2025VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large Video Language Models13 May
- 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models8 May
- 2025R-Bench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation4 May
- 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models30 Apr
- 2025Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimization25 Apr
- 2025VisuoThink: Empowering LVLM Reasoning with Multimodal Tree Search12 Apr
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
- 2025MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models8 Apr
- 2025Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)4 Apr
- 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models26 Mar
- 2025DWIM: Towards Tool-aware Visual Reasoning via Discrepancy-aware Workflow Generation & Instruct-Masking Tuning25 Mar
- 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoning23 Mar
- 2025MMR: A Large-scale Benchmark Dataset for Multi-target and Multi-granularity Reasoning Segmentation18 Mar
- 2025Grounded Chain-of-Thought for Multimodal Large Language Models17 Mar
- 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey16 Mar
- 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization13 Mar
- 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement9 Mar
- 2025CMMCoT: Enhancing Complex Multi-Image Comprehension via Multi-Modal Chain-of-Thought and Memory Augmentation7 Mar
- 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
- 2025R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts27 Feb
- 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thought13 Jan
- 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs10 Jan
- 2025ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding9 Jan
50 newest of 71 All 71 (JSON)
Other references · 104
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
38 more
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2023A Survey of Large Language Modelscited by 2,576
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
- 2025DINOv3cited by 1,414
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
- 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
- 2024OpenAI o1 System Cardcited by 1,244
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
- 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
- 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
- 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
- 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
- 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
- 2022A Survey on In-context Learningcited by 721
- 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
- 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
- 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
- 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 386
- 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 365
- 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 350
- 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
- 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
- 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 286
- 2023Demystifying CLIP Datacited by 270
- 2025Why Language Models Hallucinatecited by 220
- 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 216
- 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
- 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 175
- 2024Ovis: Structural Embedding Alignment for Multimodal Large Language Modelcited by 169
- 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 154
- 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 137
- 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 136
50 most cited of 104 All 104 (JSON)
Built on by
Citing papers per year: 2025: 8; 2026: 18.
26 papers.
Sort:
By area:
- 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokenscited by 71
- 2025V-Thinker: Interactive Thinking with Imagescited by 26
- 2025Mull-Tokens: Modality-Agnostic Latent Thinkingcited by 19
- 2025Agentic Learner with Grow-and-Refine Multimodal Semantic Memorycited by 13
- 2026OneReason Technical Reportcited by 8
- 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AIcited by 3
- 2025VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMscited by 3
- 2025Large Model Enabled Embodied Intelligence for 6G Integrated Perception, Communication, and Computation Networkcited by 2
- 2026Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzlescited by 2
- 2026Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editingcited by 2
- 2026Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Modelscited by 2
- 2026SVSR: A Self-Verification and Self-Rectification Paradigm for Multimodal Reasoningcited by 2
14 more
- 2025MMRPT: MultiModal Reinforcement Pre-Training via Masked Vision-Dependent Reasoningcited by 1
- 2026M3-ACE: Rectifying Visual Perception in Multimodal Math Reasoning via Multi-Agentic Context Engineeringcited by 1
- 2026MLLM-as-a-Judge Exhibits Model Preference Biascited by 1
- 2026VG-CoT: Towards Trustworthy Visual Reasoning via Grounded Chain-of-Thoughtcited by 1
- 2026Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoningcited by 1
- 2026VisionPulse: Dynamic Visual Sparsity for Efficient Multimodal Reasoningcited by 1
- 2026CometVLA: Co-Training on an Embodied Data Pyramid towards Physical Understandingcited by 1
- 2026MolLangData: A Large-Scale Dataset for Molecular Structure-Language Description via a Rule-Regularized Methodcited by 0
- 2026CoTZero: Annotation-Free Human-Like Vision Reasoning via Hierarchical Synthetic CoTcited by 0
- 2026SaFeR-ToolKit: Structured Reasoning via Virtual Tool Calling for Multimodal Safetycited by 0
- 2026Mags-RL: Wearing Multimodal LLMs a Magnifying Glass via Agentic Reinforcement Learning For Complex Scene Reasoningcited by 0
- 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Modelscited by 0
- 2026Mixture of Cognitive Experts in Large Vision-Language Modelscited by 0
- 2026WhisperRec: Latent Reasoning for Efficient Foundation Recommendation Modelscited by 0
- 2026CometVLA: Co-Training on an Embodied Data Pyramid towards Physical Understanding31 Aug 2026
- 2026WhisperRec: Latent Reasoning for Efficient Foundation Recommendation Models29 Jul 2026
- 2026Mixture of Cognitive Experts in Large Vision-Language Models12 Jul 2026
- 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models24 Jun 2026
- 2026OneReason Technical Report4 Jun 2026
- 2026VisionPulse: Dynamic Visual Sparsity for Efficient Multimodal Reasoning29 May 2026
- 2026Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning27 May 2026
- 2026Mags-RL: Wearing Multimodal LLMs a Magnifying Glass via Agentic Reinforcement Learning For Complex Scene Reasoning27 May 2026
- 2026VG-CoT: Towards Trustworthy Visual Reasoning via Grounded Chain-of-Thought23 Apr 2026
- 2026MLLM-as-a-Judge Exhibits Model Preference Bias13 Apr 2026
- 2026SVSR: A Self-Verification and Self-Rectification Paradigm for Multimodal Reasoning11 Apr 2026
- 2026Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models11 Apr 2026
14 more
- 2026Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editing10 Apr 2026
- 2026M3-ACE: Rectifying Visual Perception in Multimodal Math Reasoning via Multi-Agentic Context Engineering9 Mar 2026
- 2026SaFeR-ToolKit: Structured Reasoning via Virtual Tool Calling for Multimodal Safety3 Mar 2026
- 2026CoTZero: Annotation-Free Human-Like Vision Reasoning via Hierarchical Synthetic CoT9 Feb 2026
- 2026MolLangData: A Large-Scale Dataset for Molecular Structure-Language Description via a Rule-Regularized Method2 Feb 2026
- 2026Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzles20 Jan 2026
- 2025VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs24 Dec 2025
- 2025Large Model Enabled Embodied Intelligence for 6G Integrated Perception, Communication, and Computation Network17 Dec 2025
- 2025Mull-Tokens: Modality-Agnostic Latent Thinking11 Dec 2025
- 2025MMRPT: MultiModal Reinforcement Pre-Training via Masked Vision-Dependent Reasoning8 Dec 2025
- 2025Agentic Learner with Grow-and-Refine Multimodal Semantic Memory26 Nov 2025
- 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens24 Nov 2025
- 2025V-Thinker: Interactive Thinking with Images6 Nov 2025
- 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI6 Oct 2025
- 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokenscited by 71
- 2025V-Thinker: Interactive Thinking with Imagescited by 26
- 2025Agentic Learner with Grow-and-Refine Multimodal Semantic Memorycited by 13
- 2026OneReason Technical Reportcited by 8
- 2025VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMscited by 3
- 2026Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzlescited by 2
- 2026Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Modelscited by 2
- 2026SVSR: A Self-Verification and Self-Rectification Paradigm for Multimodal Reasoningcited by 2
- 2025MMRPT: MultiModal Reinforcement Pre-Training via Masked Vision-Dependent Reasoningcited by 1
- 2026M3-ACE: Rectifying Visual Perception in Multimodal Math Reasoning via Multi-Agentic Context Engineeringcited by 1
- 2026VG-CoT: Towards Trustworthy Visual Reasoning via Grounded Chain-of-Thoughtcited by 1
- 2026Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoningcited by 1
12 most cited of 18 All 18 (JSON)
- 2026WhisperRec: Latent Reasoning for Efficient Foundation Recommendation Models29 Jul 2026
- 2026Mixture of Cognitive Experts in Large Vision-Language Models12 Jul 2026
- 2026OneReason Technical Report4 Jun 2026
- 2026VisionPulse: Dynamic Visual Sparsity for Efficient Multimodal Reasoning29 May 2026
- 2026Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning27 May 2026
- 2026Mags-RL: Wearing Multimodal LLMs a Magnifying Glass via Agentic Reinforcement Learning For Complex Scene Reasoning27 May 2026
- 2026VG-CoT: Towards Trustworthy Visual Reasoning via Grounded Chain-of-Thought23 Apr 2026
- 2026SVSR: A Self-Verification and Self-Rectification Paradigm for Multimodal Reasoning11 Apr 2026
- 2026Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models11 Apr 2026
- 2026M3-ACE: Rectifying Visual Perception in Multimodal Math Reasoning via Multi-Agentic Context Engineering9 Mar 2026
- 2026SaFeR-ToolKit: Structured Reasoning via Virtual Tool Calling for Multimodal Safety3 Mar 2026
- 2026CoTZero: Annotation-Free Human-Like Vision Reasoning via Hierarchical Synthetic CoT9 Feb 2026
12 newest of 18 All 18 (JSON)
- 2025Mull-Tokens: Modality-Agnostic Latent Thinkingcited by 19
- 2025Large Model Enabled Embodied Intelligence for 6G Integrated Perception, Communication, and Computation Networkcited by 2
- 2026Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editingcited by 2
- 2026MolLangData: A Large-Scale Dataset for Molecular Structure-Language Description via a Rule-Regularized Methodcited by 0
- 2026Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editing10 Apr 2026
- 2026MolLangData: A Large-Scale Dataset for Molecular Structure-Language Description via a Rule-Regularized Method2 Feb 2026
- 2025Large Model Enabled Embodied Intelligence for 6G Integrated Perception, Communication, and Computation Network17 Dec 2025
- 2025Mull-Tokens: Modality-Agnostic Latent Thinking11 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2025Multimodal LLM Augmented Reasoning for Interpretable Visual Perception Analysis
- 2025VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity
- 2025See, Think, Learn: A Self-Taught Multimodal Reasoner
- 2024Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination
- 2026MM-OPD: Towards One More Bottleneck Between Perception and Reasoning
- 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forward
- 2026Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs
- 2026See2Think: Do Multimodal Models Really Use Intermediate Visual States?