mlsanityData to 14 Oct 2026

Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey

arXiv 2503.12605 ↗First posted 16 Mar 2025cs.CVcited by 182

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

By extending the advantage of chain-of-thought (CoT) reasoning in human-like step-by-step processes to multimodal contexts, multimodal CoT (MCoT) reasoning has recently garnered significant research attention, especially in the integration with multimodal large language models (MLLMs). Existing MCoT studies design various methodologies and innovative reasoning paradigms to address the unique challenges of image, video, speech, audio, 3D, and structured data across different modalities, achieving extensive success in applications such as robotics, healthcare, autonomous driving, and multimodal generation. However, MCoT still presents distinct challenges and opportunities that require further focus to ensure consistent thriving in this field, where, unfortunately, an up-to-date review of this domain is lacking. To bridge this gap, we present the first systematic survey of MCoT reasoning, elucidating the relevant foundational concepts and definitions. We offer a comprehensive taxonomy and an in-depth analysis of current methodologies from diverse perspectives across various application scenarios. Furthermore, we provide insights into existing challenges and future research directions, aiming to foster innovation toward multimodal AGI.

Key facts. Cited by 182 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 222 of its 260 references that are in the map. First posted on arXiv 16 Mar 2025.

Builds on

222 of 260 references.

Closest ancestors · 66

Same direction, newest first

  1. 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing13 Mar
  2. 2025VisualPRM: An Effective Process Reward Model for Multimodal Reasoning13 Mar
  3. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
  4. 2025VisRL: Intention-Driven Visual Perception via Reinforced Reasoning10 Mar
  5. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning10 Mar
  6. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning10 Mar
  7. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement9 Mar
  8. 2025Integrating Chain-of-Thought for Multimodal Alignment: A Study on 3D Vision-Language Learning8 Mar
  9. 2025MM-Verify: Enhancing Multimodal Reasoning with Chain-of-Thought Verification19 Feb
  10. 2025MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency13 Feb
  11. 2025AStar: Boosting Multimodal Reasoning with Automated Structured Thinking4 Feb
  12. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step23 Jan
38 more
  1. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thought13 Jan
  2. 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs10 Jan
  3. 2025ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding9 Jan
  4. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark9 Jan
  5. 2025Unlocking Multimodal Mathematical Reasoning via Process Reward Model8 Jan
  6. 2025Socratic Questioning: Learn to Self-guide Multimodal Reasoning in the Wild6 Jan
  7. 2025Virgo: A Preliminary Exploration on Reproducing o1-like MLLM3 Jan
  8. 2024Diving into Self-Evolving Training for Multimodal Reasoning23 Dec
  9. 2024Progressive Multimodal Reasoning via Active Retrieval19 Dec
  10. 2024CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models17 Dec
  11. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  12. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale6 Dec
  13. 2024Perception Tokens Enhance Visual Reasoning in Multimodal Language Models4 Dec
  14. 2024VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selection22 Nov
  15. 2024Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models21 Nov
  16. 2024Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination15 Nov
  17. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
  18. 2024What Factors Affect Multi-Modal In-Context Learning? An In-Depth Exploration27 Oct
  19. 2024Improve Vision Language Model Chain-of-thought Reasoning21 Oct
  20. 2024MiCEval: Unveiling Multimodal Chain of Thought's Quality via Image Description and Reasoning Steps18 Oct
  21. 2024Visual-O1: Understanding Ambiguous Instructions via Multi-modal Multi-turn Chain-of-thoughts Reasoning4 Oct
  22. 2024CaRDiff: Video Salient Object Ranking Chain of Thought Reasoning for Saliency Prediction with Diffusion21 Aug
  23. 2024MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Engine11 Jul
  24. 2024VideoCoT: A Video Chain-of-Thought Dataset with Active Annotation Tool7 Jul
  25. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models13 Jun
  26. 2024M3CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought26 May
  27. 2024Chain-of-Thought Prompting for Demographic Inference with Large Multimodal Models24 May
  28. 2024Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models22 May
  29. 2024Enhancing Semantics in Multimodal Chain of Thought via Soft Negative Sampling16 May
  30. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition7 May
  31. 2024WorldQA: Multimodal World Knowledge in Videos through Long-Chain Reasoning6 May
  32. 2024Cantor: Inspiring Multimodal Chain-of-Thought of MLLM24 Apr
  33. 2024TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding15 Apr
  34. 2024Soft-Prompting with Graph-of-Thought for Multi-modal Representation Learning6 Apr
  35. 2024A Picture Is Worth a Graph: A Blueprint Debate Paradigm for Multimodal Reasoning22 Mar
  36. 2024Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models19 Mar
  37. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  38. 2024KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning23 Jan

50 newest of 66 All 66 (JSON)

Other references · 156

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  8. 2023Visual Instruction Tuningcited by 7,869
  9. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  11. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  12. 2025Qwen2.5-VL Technical Reportcited by 5,587
38 more
  1. 2024GPT-4o System Cardcited by 4,588
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  8. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  10. 2023A Survey of Large Language Modelscited by 2,576
  11. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  12. 2022Emergent Abilities of Large Language Modelscited by 2,512
  13. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  14. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  15. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  16. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  17. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  18. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  19. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  20. 2023PaLM 2 Technical Reportcited by 1,162
  21. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  22. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  23. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  24. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  25. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  26. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  27. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  28. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  29. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  30. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  31. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 615
  32. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  33. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  34. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  35. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  36. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  37. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  38. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 565

50 most cited of 156 All 156 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 95; 2026: 87.

182 papers.

Sort:

By area:

  1. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  2. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 197
  3. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 132
  4. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challengescited by 74
  5. 2025Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?cited by 58
  6. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 55
  7. 2025Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoningcited by 51
  8. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learningcited by 49
  9. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 44
  10. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 42
  11. 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
  12. 2025MedFrameQA: A Multi-Image Medical VQA Benchmark for Clinical Reasoningcited by 34
38 more
  1. 2025ThinkSound: Chain-of-Thought Reasoning in Multimodal Large Language Models for Audio Generation and Editingcited by 34
  2. 2025MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Modelscited by 34
  3. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 33
  4. 2025Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuningcited by 32
  5. 2025Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedbackcited by 32
  6. 2025JarvisEvo: Towards a Self-Evolving Photo Editing Agent with Synergistic Editor-Evaluator Optimizationcited by 31
  7. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 30
  8. 2025R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Searchcited by 30
  9. 2025SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Modelscited by 29
  10. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 28
  11. 2025VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoningcited by 26
  12. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  13. 2025Chart-R1: Chain-of-Thought Supervision and Reinforcement for Advanced Chart Reasonercited by 25
  14. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  15. 2025A Survey on Agentic Multimodal Large Language Modelscited by 23
  16. 2025MMR-V: What's Left Unsaid? A Benchmark for Multimodal Deep Reasoning in Videoscited by 22
  17. 2025RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learningcited by 21
  18. 2026Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoringcited by 21
  19. 2025Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generationcited by 19
  20. 2025ViTCoT: Video-Text Interleaved Chain-of-Thought for Boosting Video Understanding in Large Language Modelscited by 18
  21. 2025RBench-V: A Primary Assessment for Visual Reasoning Models with Multi-modal Outputscited by 17
  22. 2025SATORI-R1: Incentivizing Multimodal Reasoning through Explicit Visual Anchoringcited by 17
  23. 2025Reasoning in Computer Vision: Taxonomy, Models, Tasks, and Methodologiescited by 17
  24. 2025On the Faithfulness of Visual Thinking: Measurement and Enhancementcited by 17
  25. 2025Chiron-o1: Igniting Multimodal Large Language Models towards Generalizable Medical Reasoning via Mentor-Intern Collaborative Searchcited by 16
  26. 2025Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Schemecited by 14
  27. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 14
  28. 2026When and How Much to Imagine: Adaptive Test-Time Scaling with World Models for Visual Spatial Reasoningcited by 14
  29. 2025VisCRA: A Visual Chain Reasoning Attack for Jailbreaking Multimodal Large Language Modelscited by 13
  30. 2026Not All Queries Need Deep Thought: CoFiCot for Adaptive Coarse-to-fine Stateful Refinementcited by 13
  31. 2025ViC-Bench: Benchmarking Visual-Interleaved Chain-of-Thought Capability in MLLMs with Free-Style Intermediate State Representationscited by 12
  32. 2025Reinforced Reasoning for Embodied Planningcited by 12
  33. 2025BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Datasetcited by 12
  34. 2025PixelCraft: A Multi-Agent System for High-Fidelity Visual Reasoning on Structured Imagescited by 12
  35. 2025Understanding-in-Generation: Reinforcing Generative Capability of Unified Model via Infusing Understanding into Generationcited by 11
  36. 2025ControlAudio: Tackling Text-Guided, Timing-Indicated and Intelligible Audio Generation via Progressive Diffusion Modelingcited by 11
  37. 2026DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Drivingcited by 11
  38. 2025Visual-Aware CoT: Achieving High-Fidelity Visual Consistency in Unified Modelscited by 10

50 most cited of 182 All 182 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do
  2. 2025Cognitive Chain-of-Thought (CoCoT): Structured Multimodal Reasoning about Social Situations
  3. 2023Enhancing Human-like Multi-Modal Reasoning: A New Challenging Dataset and Comprehensive Framework
  4. 2025Uni-cot: Towards Unified Chain-of-Thought Reasoning Across Text and Vision
  5. 2025Multimodal Chain of Continuous Thought for Latent-Space Reasoning in Vision-Language Models
  6. 2025Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought
  7. 2026TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding
  8. 2025Cross Domain Evaluation of Multimodal Chain-of-Thought Reasoning of different datasets into the Amazon CoT Framework