mlsanityData to 14 Oct 2026

From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models

arXiv 2509.25373 ↗First posted 29 Sep 2025cs.AIcited by 26

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) strive to achieve a profound, human-like understanding of and interaction with the physical world, but often exhibit a shallow and incoherent integration when acquiring information (Perception) and conducting reasoning (Cognition). This disconnect leads to a spectrum of reasoning failures, with hallucination being the most prominent. Collectively, these issues expose a fundamental challenge: the ability to process pixels does not yet confer the ability to construct a coherent, credible internal world model. To systematically dissect and address this challenge, this survey introduces a novel and unified analytical framework: ``From Perception to Cognition." We deconstruct the complex process of vision-language interactive understanding into two interdependent layers: Perception, the foundational ability to accurately extract visual information and achieve fine-grained alignment with textual instructions; and Cognition, the higher-order capability for proactive, multi-step, goal-oriented reasoning built upon this perceptual foundation, the core of which is the formation of a dynamic observe-think-verify reasoning loop. Guided by this framework, this paper systematically analyzes the key bottlenecks of current MLLMs at both layers. It surveys the landscape of cutting-edge methods designed to address these challenges, spanning from techniques that enhance low-level visual representations to those that improve high-level reasoning paradigms. Furthermore, we review critical benchmarks and delineate future research directions. This survey aims to provide the research community with a clear, structured perspective for understanding the intrinsic limitations of current MLLMs and to illuminate the path toward building next-generation models capable of deep reasoning and a genuine understanding of the world.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 175 of its 350 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

175 of 350 references.

Closest ancestors · 71

Same direction, newest first

  1. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoning24 Aug
  2. 2025Multimodal Chain of Continuous Thought for Latent-Space Reasoning in Vision-Language Models18 Aug
  3. 2025Ovis2.5 Technical Report15 Aug
  4. 2025SIFThinker: Spatially-Aware Image Focus for Visual Reasoning8 Aug
  5. 2025MathReal: We Keep It Real! A Real Scene Benchmark for Evaluating Math Reasoning in Multimodal Large Language Models8 Aug
  6. 2025CausalStep: A Benchmark for Explicit Stepwise Causal Reasoning in Videos22 Jul
  7. 2025Bootstrapping Grounded Chain-of-Thought in Multimodal LLMs for Data-Efficient Model Adaptation3 Jul
  8. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoning2 Jul
  9. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers30 Jun
  10. 2025VGR: Visual Grounded Reasoning13 Jun
  11. 2025CausalVQA: A Physically Grounded Causal Reasoning Benchmark for Video Models11 Jun
  12. 2025Socratic-MCTS: Test-Time Visual Reasoning by Asking the Right Questions10 Jun
38 more
  1. 2025VReST: Enhancing Reasoning in Large Vision-Language Models through Tree Search and Self-Reward Mechanism10 Jun
  2. 2025Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification8 Jun
  3. 2025MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical Chain-of-Thought Reasoning5 Jun
  4. 2025GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinking1 Jun
  5. 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoning29 May
  6. 2025ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering29 May
  7. 2025Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning26 May
  8. 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use25 May
  9. 2025v1: Learning to Point Visual Tokens for Multimodal Grounded Reasoning24 May
  10. 2025VLM-R3: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought22 May
  11. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning21 May
  12. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
  13. 2025Visual Agentic Reinforcement Fine-Tuning20 May
  14. 2025MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable Step-Level Supervision19 May
  15. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May
  16. 2025VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large Video Language Models13 May
  17. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models8 May
  18. 2025R-Bench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation4 May
  19. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models30 Apr
  20. 2025Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimization25 Apr
  21. 2025VisuoThink: Empowering LVLM Reasoning with Multimodal Tree Search12 Apr
  22. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
  23. 2025MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models8 Apr
  24. 2025Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)4 Apr
  25. 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models26 Mar
  26. 2025DWIM: Towards Tool-aware Visual Reasoning via Discrepancy-aware Workflow Generation & Instruct-Masking Tuning25 Mar
  27. 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoning23 Mar
  28. 2025MMR: A Large-scale Benchmark Dataset for Multi-target and Multi-granularity Reasoning Segmentation18 Mar
  29. 2025Grounded Chain-of-Thought for Multimodal Large Language Models17 Mar
  30. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey16 Mar
  31. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization13 Mar
  32. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement9 Mar
  33. 2025CMMCoT: Enhancing Complex Multi-Image Comprehension via Multi-Modal Chain-of-Thought and Memory Augmentation7 Mar
  34. 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
  35. 2025R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts27 Feb
  36. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thought13 Jan
  37. 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs10 Jan
  38. 2025ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding9 Jan

50 newest of 71 All 71 (JSON)

Other references · 104

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2024GPT-4o System Cardcited by 4,588
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  10. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  11. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  12. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
38 more
  1. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  2. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  4. 2023A Survey of Large Language Modelscited by 2,576
  5. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  6. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  7. 2025DINOv3cited by 1,414
  8. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  9. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  10. 2024OpenAI o1 System Cardcited by 1,244
  11. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  12. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  13. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  14. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  15. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  16. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  17. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
  18. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
  19. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  20. 2022A Survey on In-context Learningcited by 721
  21. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  22. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  23. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  24. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 386
  25. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 365
  26. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 350
  27. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  28. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  29. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 286
  30. 2023Demystifying CLIP Datacited by 270
  31. 2025Why Language Models Hallucinatecited by 220
  32. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 216
  33. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
  34. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 175
  35. 2024Ovis: Structural Embedding Alignment for Multimodal Large Language Modelcited by 169
  36. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 154
  37. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 137
  38. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 136

50 most cited of 104 All 104 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 18.

26 papers.

Sort:

By area:

  1. 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokenscited by 71
  2. 2025V-Thinker: Interactive Thinking with Imagescited by 26
  3. 2025Mull-Tokens: Modality-Agnostic Latent Thinkingcited by 19
  4. 2025Agentic Learner with Grow-and-Refine Multimodal Semantic Memorycited by 13
  5. 2026OneReason Technical Reportcited by 8
  6. 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AIcited by 3
  7. 2025VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMscited by 3
  8. 2025Large Model Enabled Embodied Intelligence for 6G Integrated Perception, Communication, and Computation Networkcited by 2
  9. 2026Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzlescited by 2
  10. 2026Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editingcited by 2
  11. 2026Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Modelscited by 2
  12. 2026SVSR: A Self-Verification and Self-Rectification Paradigm for Multimodal Reasoningcited by 2
14 more
  1. 2025MMRPT: MultiModal Reinforcement Pre-Training via Masked Vision-Dependent Reasoningcited by 1
  2. 2026M3-ACE: Rectifying Visual Perception in Multimodal Math Reasoning via Multi-Agentic Context Engineeringcited by 1
  3. 2026MLLM-as-a-Judge Exhibits Model Preference Biascited by 1
  4. 2026VG-CoT: Towards Trustworthy Visual Reasoning via Grounded Chain-of-Thoughtcited by 1
  5. 2026Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoningcited by 1
  6. 2026VisionPulse: Dynamic Visual Sparsity for Efficient Multimodal Reasoningcited by 1
  7. 2026CometVLA: Co-Training on an Embodied Data Pyramid towards Physical Understandingcited by 1
  8. 2026MolLangData: A Large-Scale Dataset for Molecular Structure-Language Description via a Rule-Regularized Methodcited by 0
  9. 2026CoTZero: Annotation-Free Human-Like Vision Reasoning via Hierarchical Synthetic CoTcited by 0
  10. 2026SaFeR-ToolKit: Structured Reasoning via Virtual Tool Calling for Multimodal Safetycited by 0
  11. 2026Mags-RL: Wearing Multimodal LLMs a Magnifying Glass via Agentic Reinforcement Learning For Complex Scene Reasoningcited by 0
  12. 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Modelscited by 0
  13. 2026Mixture of Cognitive Experts in Large Vision-Language Modelscited by 0
  14. 2026WhisperRec: Latent Reasoning for Efficient Foundation Recommendation Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Multimodal LLM Augmented Reasoning for Interpretable Visual Perception Analysis
  2. 2025VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity
  3. 2025See, Think, Learn: A Self-Taught Multimodal Reasoner
  4. 2024Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination
  5. 2026MM-OPD: Towards One More Bottleneck Between Perception and Reasoning
  6. 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forward
  7. 2026Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs
  8. 2026See2Think: Do Multimodal Models Really Use Intermediate Visual States?