mlsanityData to 14 Oct 2026

DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning

arXiv 2505.14362 ↗First posted 20 May 2025cs.CVcited by 410

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Large Vision-Language Models excel at multimodal understanding but struggle to deeply integrate visual information into their predominantly text-based reasoning processes, a key challenge in mirroring human cognition. To address this, we introduce DeepEyes, a model that learns to "think with images", trained end-to-end with reinforcement learning without requiring pre-collected reasoning data for cold-start supervised fine-tuning (SFT). Notably, this ability emerges natively, leveraging the model's own grounding capability as an intrinsic function rather than relying on external specialized models or APIs. We enable this capability through active perception, where the model learns to strategically ground its reasoning in visual information, guided by a tailored data selection and reward strategy. DeepEyes achieves significant performance gains on general perception and reasoning benchmarks and also demonstrates improvement in grounding, hallucination, and mathematical reasoning tasks. Interestingly, we observe the distinct evolution of active perception from initial exploration to efficient and accurate exploitation, and diverse thinking patterns that closely mirror human visual reasoning processes. Code is available at https://github.com/Visual-Agent/DeepEyes.

Key facts. Cited by 410 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 70 of its 80 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

70 of 80 references.

Closest ancestors · 23

Same direction, newest first

  1. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement10 Apr
  2. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
  3. 2025Kimi-VL Technical Report10 Apr
  4. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
  5. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning10 Mar
  6. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement9 Mar
  7. 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model7 Mar
  8. 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
  9. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thought13 Jan
  10. 2025ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding9 Jan
  11. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  12. 2024Perception Tokens Enhance Visual Reasoning in Multimodal Language Models4 Dec
11 more
  1. 2024DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Models29 Oct
  2. 2024Visual Agents as Fast and Slow Thinkers16 Aug
  3. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts6 Jul
  4. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?1 Jul
  5. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning25 Mar
  6. 2024Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models19 Mar
  7. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  8. 2024KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning23 Jan
  9. 2023Multi-modal Latent Space Learning for Chain-of-Thought Reasoning in Language Models14 Dec
  10. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  11. 2023LISA: Reasoning Segmentation via Large Language Model1 Aug

Other references · 47

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2024Qwen2.5 Technical Reportcited by 4,942
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  11. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  12. 2023Qwen Technical Reportcited by 3,344
35 more
  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  3. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  4. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  5. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  6. 2025s1: Simple test-time scalingcited by 1,341
  7. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  8. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  9. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  10. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  11. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  12. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  13. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  14. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
  15. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  16. 2025Seed1.5-VL Technical Reportcited by 373
  17. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 316
  18. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
  19. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 291
  20. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 195
  21. 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 193
  22. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 166
  23. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 161
  24. 2024Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Modelscited by 139
  25. 2024Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Expertscited by 108
  26. 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Explorationcited by 103
  27. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  28. 2025DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understandingcited by 49
  29. 2024MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Modelscited by 46
  30. 2024LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillationcited by 45
  31. 2025Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learningcited by 37
  32. 2025UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spacescited by 36
  33. 2025InternVL-X: Advancing and Accelerating InternVL Series with Efficient Visual Token Compressioncited by 21
  34. 2024PKRD-CoT: A Unified Chain-of-thought Prompting for Multi-Modal Large Language Models in Autonomous Drivingcited by 17
  35. 2025Perception in Reflectioncited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 131; 2026: 279.

410 papers.

Sort:

By area:

  1. 2025Qwen3-VL Technical Reportcited by 2,793
  2. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  3. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  4. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  5. 2025Thyme: Think Beyond Imagescited by 167
  6. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 125
  7. 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 120
  8. 2025Perception-Aware Policy Optimization for Multimodal Reasoningcited by 99
  9. 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMscited by 96
  10. 2025Zebra-CoT: A Dataset for Interleaved Vision Language Reasoningcited by 84
  11. 2025Monet: Reasoning in Latent Visual Space Beyond Images and Languagecited by 82
  12. 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillationcited by 77
38 more
  1. 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 75
  2. 2025Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodologycited by 69
  3. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Viewscited by 65
  4. 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoningcited by 56
  5. 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 56
  6. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 55
  7. 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 50
  8. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learningcited by 49
  9. 2025Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidencecited by 49
  10. 2025Kwai Keye-VL Technical Reportcited by 47
  11. 2025ACTIVE-o3: Empowering MLLMs with Active Perception via Pure Reinforcement Learningcited by 43
  12. 2025Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMscited by 43
  13. 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 42
  14. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 39
  15. 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learningcited by 34
  16. 2025Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for Efficient and Enhanced Video Reasoningcited by 33
  17. 2025SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learningcited by 33
  18. 2025Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedbackcited by 32
  19. 2025We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoningcited by 32
  20. 2025UI-S1: Advancing GUI Automation via Semi-online Reinforcement Learningcited by 32
  21. 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoningcited by 31
  22. 2025JarvisEvo: Towards a Self-Evolving Photo Editing Agent with Synergistic Editor-Evaluator Optimizationcited by 31
  23. 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 29
  24. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 28
  25. 2025Video-MTR: Reinforced Multi-Turn Reasoning for Long Video Understandingcited by 28
  26. 2025Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Spacecited by 28
  27. 2025VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoningcited by 27
  28. 2025ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesiscited by 26
  29. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  30. 2025Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Toolscited by 26
  31. 2025SIFThinker: Spatially-Aware Image Focus for Visual Reasoningcited by 25
  32. 2025Skyra: AI-Generated Video Detection via Grounded Artifact Reasoningcited by 25
  33. 2025Sketch-in-Latents: Eliciting Unified Reasoning in MLLMscited by 25
  34. 2026Think3D: Thinking with Space for Spatial Reasoningcited by 24
  35. 2026GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoningcited by 24
  36. 2026Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoningcited by 24
  37. 2026Embed-RL: Reinforcement Learning for Reasoning-Driven Multimodal Embeddingscited by 24
  38. 2025A Survey on Agentic Multimodal Large Language Modelscited by 23

50 most cited of 410 All 410 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026ViThinker: Active Vision-Language Reasoning via Dynamic Perceptual Querying
  2. 2025VisRL: Intention-Driven Visual Perception via Reinforced Reasoning
  3. 2025More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models
  4. 2025VisPlay: Self-Evolving Vision-Language Models from Images
  5. 2025V-Thinker: Interactive Thinking with Images
  6. 2025Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Models
  7. 2025Qwen Look Again: Guiding Vision-Language Reasoning Models to Re-attention Visual Information
  8. 2026H-GRPO: Permutation-Invariant Reinforcement Learning for Grounded Visual Reasoning