mlsanityData to 7 Oct 2026

What Does Vision Tool-Use Reinforcement Learning Really Learn? Disentangling Tool-Induced and Intrinsic Effects for Crop-and-Zoom

arXiv 2602.01334 ↗First posted 1 Feb 2026cs.CVcited by 9

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Vision tool-use reinforcement learning (RL) can equip vision language models with visual operators such as crop-and-zoom and achieves strong performance gains, yet it remains unclear whether these gains are driven by improvements in tool use or evolving intrinsic capabilities. We introduce MED (Measure--Explain--Diagnose), a coarse-to-fine framework that disentangles intrinsic capability changes from tool-induced effects, decomposes the tool-induced performance difference into gain and harm terms, and probes the mechanisms driving their evolution. Across checkpoint-level analyses in the crop-and-zoom setting on two VLMs with different tool priors and six benchmarks, we find that improvements are dominated by intrinsic learning, while tool-use RL mainly reduces tool-induced harm (e.g., fewer call-induced errors and weaker tool schema interference) and yields limited progress in tool-based correction of intrinsic failures. Overall, in the crop-and-zoom setting studied here, current vision tool-use RL learns to coexist safely with tools rather than master them.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 46 references that are in the map. First posted on arXiv 1 Feb 2026.

Builds on

36 of 46 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025AdaTooler-V: Adaptive Tool-Use for Images and Videos18 Dec
  2. 2025Knowing the Answer Isn't Enough: Fixing Reasoning Path Failures in LVLMs6 Dec
  3. 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs21 May
  4. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
  5. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May

Other references · 31

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  6. 2025Qwen3-VL Technical Reportcited by 2,534
  7. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  8. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  9. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  10. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
  11. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
  12. 2019Scene Text Visual Question Answeringcited by 327
19 more
  1. 2019PlotQA: Reasoning over Scientific Plotscited by 317
  2. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 267
  3. 2015Evaluation of Deep Convolutional Nets for Document Image Classification and Retrievalcited by 212
  4. 2025Thyme: Think Beyond Imagescited by 160
  5. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 151
  6. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 119
  7. 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 116
  8. 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuningcited by 86
  9. 2020On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answeringcited by 78
  10. 2024SPIQA: A Dataset for Multimodal Question Answering on Scientific Paperscited by 75
  11. 2025ACTIVE-o3: Empowering MLLMs with Active Perception via Pure Reinforcement Learningcited by 41
  12. 2025Reinforced Visual Perception with Toolscited by 38
  13. 2025MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMscited by 31
  14. 2025Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearchcited by 25
  15. 2025HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decouplingcited by 16
  16. 2025From Illusion to Intention: Visual Rationale Learning for Vision-Language Reasoningcited by 9
  17. 2025Revisiting the Necessity of Lengthy Chain-of-Thought in Vision-centric Reasoning Generalizationcited by 8
  18. 2025VFaith: Do Large Multimodal Models Really Reason on Seen Images Rather than Previous Memories?cited by 7
  19. 2022Hamming Distributions of Popular Perceptual Hashing Techniquescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selection
  2. 2026ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning
  3. 2026VC-Tooler: Learning Compositional and Adaptive Visual Tool Use
  4. 2026ReGRPO: Reflection-Augmented Policy Optimization for Tool-Using Agents
  5. 2026GUI-Eyes: Tool-Augmented Perception for Visual Grounding in GUI Agents
  6. 2026Visual Reasoning through Tool-supervised Reinforcement Learning
  7. 2025Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning
  8. 2025Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme