mlsanityData to 14 Oct 2026

OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning

arXiv 2505.08617 ↗First posted 13 May 2025cs.CVcited by 163

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

While humans can flexibly leverage interactive visual cognition for complex problem-solving, enabling Large Vision-Language Models (LVLMs) to learn similarly adaptive behaviors with visual tools remains challenging. A significant hurdle is the current lack of standardized infrastructure, which hinders integrating diverse tools, generating rich interaction data, and training robust agents effectively. To address these gaps, we introduce OpenThinkIMG, the first open-source, comprehensive end-to-end framework for tool-augmented LVLMs. It features standardized vision tool interfaces, scalable trajectory generation for policy initialization, and a flexible training environment. Furthermore, considering supervised fine-tuning (SFT) on static demonstrations offers limited policy generalization for dynamic tool invocation, we propose a novel reinforcement learning (RL) framework V-ToolRL to train LVLMs to learn adaptive policies for invoking external vision tools. V-ToolRL enables LVLMs to autonomously discover optimal tool-usage strategies by directly optimizing for task success using feedback from tool interactions. We empirically validate V-ToolRL on challenging chart reasoning tasks. Our RL-trained agent, built upon a Qwen2-VL-2B, significantly outperforms its SFT-initialized counterpart (+28.83 points) and surpasses established supervised tool-learning baselines like Taco and CogCom by an average of +12.7 points. Notably, it also surpasses prominent closed-source models like GPT-4.1 by +8.68 accuracy points. We hope OpenThinkIMG can serve as a foundational framework for advancing dynamic, tool-augmented visual reasoning, helping the community develop AI agents that can genuinely "think with images".

Key facts. Cited by 163 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 44 references that are in the map. First posted on arXiv 13 May 2025.

Builds on

30 of 44 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
  2. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning10 Mar
  3. 2025ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding9 Jan
  4. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models13 Jun
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  6. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models8 Mar

Other references · 24

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2023Segment Anythingcited by 8,234
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  10. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  11. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  12. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
12 more
  1. 2015VQA: Visual Question Answeringcited by 2,506
  2. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  3. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  4. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 341
  5. 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 193
  6. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 132
  7. 2024ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wildcited by 79
  8. 2024ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLMcited by 73
  9. 2024MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learningcited by 69
  10. 2025Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understandingcited by 47
  11. 2024Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning?cited by 25
  12. 2024Timo: Towards Better Temporal Reasoning for Language Modelscited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 77; 2026: 86.

163 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  2. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  4. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  5. 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokenscited by 124
  6. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 119
  7. 2025Latent Visual Reasoningcited by 104
  8. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 91
  9. 2025Monet: Reasoning in Latent Visual Space Beyond Images and Languagecited by 82
  10. 2025PyVision: Agentic Vision with Dynamic Toolingcited by 79
  11. 2025Kwai Keye-VL 1.5 Technical Reportcited by 73
  12. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Viewscited by 65
38 more
  1. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 55
  2. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learningcited by 53
  3. 2025AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgetingcited by 50
  4. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learningcited by 49
  5. 2025Kwai Keye-VL Technical Reportcited by 47
  6. 2025Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMscited by 43
  7. 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 42
  8. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 39
  9. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 37
  10. 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
  11. 2025Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thoughtcited by 35
  12. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 35
  13. 2025CostBench: Evaluating Multi-Turn Cost-Optimal Planning and Adaptation in Dynamic Environments for LLM Tool-Use Agentscited by 33
  14. 2025SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learningcited by 33
  15. 2025Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedbackcited by 32
  16. 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoningcited by 31
  17. 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 29
  18. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 28
  19. 2025Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Spacecited by 28
  20. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  21. 2025Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Toolscited by 26
  22. 2025CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimizationcited by 26
  23. 2025SIFThinker: Spatially-Aware Image Focus for Visual Reasoningcited by 25
  24. 2025Skyra: AI-Generated Video Detection via Grounded Artifact Reasoningcited by 25
  25. 2026Think3D: Thinking with Space for Spatial Reasoningcited by 24
  26. 2026Embed-RL: Reinforcement Learning for Reasoning-Driven Multimodal Embeddingscited by 24
  27. 2025A Survey on Agentic Multimodal Large Language Modelscited by 23
  28. 2026AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarioscited by 23
  29. 2025FrameMind: Frame-Interleaved Video Reasoning via Reinforcement Learningcited by 21
  30. 2025AdaTooler-V: Adaptive Tool-Use for Images and Videoscited by 21
  31. 2025Benchmarking Multimodal Knowledge Conflict for Large Multimodal Modelscited by 19
  32. 2025DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Modelscited by 19
  33. 2025On the Faithfulness of Visual Thinking: Measurement and Enhancementcited by 17
  34. 2025Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMscited by 17
  35. 2026LensWalk: Agentic Video Understanding by Planning How You See in Videoscited by 17
  36. 2025CodeDance: A Dynamic Tool-integrated MLLM for Executable Visual Reasoningcited by 16
  37. 2025Thinking with Programming Vision: Towards a Unified View for Thinking with Imagescited by 15
  38. 2026ToolGate: Contract-Grounded and Verified Tool Execution for LLMscited by 15

50 most cited of 163 All 163 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use
  2. 2025VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selection
  3. 2026VC-Tooler: Learning Compositional and Adaptive Visual Tool Use
  4. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles
  5. 2025V-Thinker: Interactive Thinking with Images
  6. 2025Visual Agentic Reinforcement Fine-Tuning
  7. 2025Reinforced Visual Perception with Tools
  8. 2025SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL