mlsanityData to 13 Oct 2026

Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers

arXiv 2506.23918 ↗First posted 30 Jun 2025cs.CVcited by 197

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Recent progress in multimodal reasoning has been significantly advanced by textual Chain-of-Thought (CoT), a paradigm where models conduct reasoning within language. This text-centric approach, however, treats vision as a static, initial context, creating a fundamental "semantic gap" between rich perceptual data and discrete symbolic thought. Human cognition often transcends language, utilizing vision as a dynamic mental sketchpad. A similar evolution is now unfolding in AI, marking a fundamental paradigm shift from models that merely think about images to those that can truly think with images. This emerging paradigm is characterized by models leveraging visual information as intermediate steps in their thought process, transforming vision from a passive input into a dynamic, manipulable cognitive workspace. In this survey, we chart this evolution of intelligence along a trajectory of increasing cognitive autonomy, which unfolds across three key stages: from external tool exploration, through programmatic manipulation, to intrinsic imagination. To structure this rapidly evolving field, our survey makes four key contributions. (1) We establish the foundational principles of the think with image paradigm and its three-stage framework. (2) We provide a comprehensive review of the core methods that characterize each stage of this roadmap. (3) We analyze the critical landscape of evaluation benchmarks and transformative applications. (4) We identify significant challenges and outline promising future directions. By providing this structured overview, we aim to offer a clear roadmap for future research towards more powerful and human-aligned multimodal AI.

Key facts. Cited by 197 papers in the map (data through 13 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 197 of its 298 references that are in the map. First posted on arXiv 30 Jun 2025.

Builds on

197 of 298 references.

Closest ancestors · 66

Same direction, newest first

  1. 2025VGR: Visual Grounded Reasoning13 Jun
  2. 2025Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoning12 Jun
  3. 2025Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification8 Jun
  4. 2025VisioMath: Benchmarking Figure-based Mathematical Reasoning in LMMs7 Jun
  5. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learning4 Jun
  6. 2025ControlThinker: Unveiling Latent Semantics for Controllable Image Generation through Visual Reasoning4 Jun
  7. 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoning29 May
  8. 2025MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibration29 May
  9. 2025Thinking with Generated Images28 May
  10. 2025Thinking with Visual Abstract: Enhancing Multimodal Reasoning via Visual Abstraction26 May
  11. 2025Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning26 May
  12. 2025SATORI-R1: Incentivizing Multimodal Reasoning through Explicit Visual Anchoring25 May
38 more
  1. 2025Caption This, Reason That: VLMs Caught in the Middle24 May
  2. 2025v1: Learning to Point Visual Tokens for Multimodal Grounded Reasoning24 May
  3. 2025OCR-Reasoning Benchmark: Unveiling the True Capabilities of MLLMs in Complex Text-Rich Image Reasoning22 May
  4. 2025GoT-R1: Unleashing Reasoning Capability of MLLM for Visual Generation with Reinforcement Learning22 May
  5. 2025Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models22 May
  6. 2025VLM-R3: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought22 May
  7. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning21 May
  8. 2025GRIT: Teaching MLLMs to Think with Images21 May
  9. 2025Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought21 May
  10. 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs21 May
  11. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
  12. 2025Visual Agentic Reinforcement Fine-Tuning20 May
  13. 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning20 May
  14. 2025Game-RL: Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning20 May
  15. 2025ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models19 May
  16. 2025VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning17 May
  17. 2025MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning15 May
  18. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May
  19. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models8 May
  20. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT1 May
  21. 2025VisuoThink: Empowering LVLM Reasoning with Multimodal Tree Search12 Apr
  22. 2025MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models8 Apr
  23. 2025Explain with Visual Keypoints Like a Real Mentor! A Benchmark for Multimodal Solution Explanation4 Apr
  24. 2025VGRP-Bench: Visual Grid Reasoning Puzzle Benchmark for Large Vision-Language Models29 Mar
  25. 2025Math Blind: Failures in Diagram Understanding Undermine Reasoning in MLLMs26 Mar
  26. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey16 Mar
  27. 2025VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity14 Mar
  28. 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing13 Mar
  29. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement9 Mar
  30. 2025CMMCoT: Enhancing Complex Multi-Image Comprehension via Multi-Modal Chain-of-Thought and Memory Augmentation7 Mar
  31. 2025MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contexts28 Feb
  32. 2025MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problems27 Feb
  33. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning5 Feb
  34. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step23 Jan
  35. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thought13 Jan
  36. 2025ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding9 Jan
  37. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark9 Jan
  38. 2024LATTE: Learning to Think with Vision Specialists7 Dec

50 newest of 66 All 66 (JSON)

Other references · 131

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  2. 2025Qwen2.5-VL Technical Reportcited by 5,499
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,191
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,022
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,011
  6. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,851
  7. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,775
  8. 2023Mastering Diverse Domains through World Modelscited by 1,224
  9. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,066
  10. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,052
  11. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 913
  12. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 899
38 more
  1. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 872
  2. 2023Reasoning with Language Model is Planning with World Modelcited by 851
  3. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 848
  4. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 799
  5. 2024Emu3: Next-Token Prediction is All You Needcited by 768
  6. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 740
  7. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 591
  8. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 502
  9. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 483
  10. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 475
  11. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 470
  12. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 461
  13. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 451
  14. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 446
  15. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 381
  16. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 375
  17. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 368
  18. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 367
  19. 2025Seed1.5-VL Technical Reportcited by 367
  20. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 352
  21. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 348
  22. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 329
  23. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 315
  24. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 299
  25. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 292
  26. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 288
  27. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
  28. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 245
  29. 2025Transfer between Modalities with MetaQueriescited by 232
  30. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 228
  31. 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
  32. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
  33. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 205
  34. 2024A Survey on Efficient Inference for Large Language Modelscited by 202
  35. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
  36. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 197
  37. 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 191
  38. 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMscited by 173

50 most cited of 131 All 131 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 74; 2026: 123.

197 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 200
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 177
  3. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 117
  4. 2025Latent Visual Reasoningcited by 101
  5. 2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Modelscited by 83
  6. 2025PyVision: Agentic Vision with Dynamic Toolingcited by 73
  7. 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 71
  8. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 70
  9. 2025Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMscited by 42
  10. 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 40
  11. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 38
  12. 2025Planning with Reasoning using Vision Language World Modelcited by 36
38 more
  1. 2025TIR-Bench: A Comprehensive Benchmark for Agentic Thinking-with-Images Reasoningcited by 34
  2. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 34
  3. 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
  4. 2025RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmarkcited by 33
  5. 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learningcited by 32
  6. 2025CostBench: Evaluating Multi-Turn Cost-Optimal Planning and Adaptation in Dynamic Environments for LLM Tool-Use Agentscited by 32
  7. 2025Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedbackcited by 31
  8. 2025SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learningcited by 31
  9. 2025JarvisEvo: Towards a Self-Evolving Photo Editing Agent with Synergistic Editor-Evaluator Optimizationcited by 30
  10. 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 29
  11. 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoningcited by 29
  12. 2025Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmarkcited by 28
  13. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 27
  14. 2025Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearchcited by 27
  15. 2025Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Spacecited by 27
  16. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  17. 2025V-Thinker: Interactive Thinking with Imagescited by 26
  18. 2026Code2World: A GUI World Model via Renderable Code Generationcited by 26
  19. 2025Skyra: AI-Generated Video Detection via Grounded Artifact Reasoningcited by 25
  20. 2026Embed-RL: Reinforcement Learning for Reasoning-Driven Multimodal Embeddingscited by 24
  21. 2025A Survey on Agentic Multimodal Large Language Modelscited by 22
  22. 2026Think3D: Thinking with Space for Spatial Reasoningcited by 22
  23. 2026AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarioscited by 22
  24. 2025UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAGcited by 21
  25. 2025VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoocited by 20
  26. 2025AdaTooler-V: Adaptive Tool-Use for Images and Videoscited by 20
  27. 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruningcited by 19
  28. 2025TimeSearch-R: Adaptive Temporal Search for Long-Form Video Understanding via Self-Verification Reinforcement Learningcited by 19
  29. 2025Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generationcited by 18
  30. 2025DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Modelscited by 18
  31. 2026Mind-Brush: Integrating Agentic Cognitive Search and Reasoning into Image Generationcited by 18
  32. 2025On the Faithfulness of Visual Thinking: Measurement and Enhancementcited by 17
  33. 2025Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMscited by 17
  34. 2025MARS-SQL: A multi-agent reinforcement learning framework for Text-to-SQLcited by 14
  35. 2026LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoningcited by 14
  36. 2025Environment Scaling for Interactive Agentic Experience Collection: A Surveycited by 13
  37. 2026Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Modelscited by 13
  38. 2025Think Twice to See More: Iterative Visual Reasoning in Medical VLMscited by 12

50 most cited of 197 All 197 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning
  2. 2026Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do
  3. 2026Visual Generation Unlocks Human-Like Reasoning through Multimodal World Models
  4. 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoning
  5. 2026Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMs
  6. 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens
  7. 2025Thinking with Programming Vision: Towards a Unified View for Thinking with Images
  8. 2025DeepSketcher: Internalizing Visual Manipulation for Multimodal Reasoning