# Diversity Over Frequency: Rethinking Tool Use in Visual Chain-of-Thought Agents

arXiv: https://arxiv.org/abs/2606.00096 · HTML: https://mlsanity.com/paper/2606.00096/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 25 May 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 10 of 10)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning](https://mlsanity.com/paper/2505.08617/) (2025)
- [Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs](https://mlsanity.com/paper/2505.15436/) (2025)
- [VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use](https://mlsanity.com/paper/2505.19255/) (2025)
- [ACTIVE-o3: Empowering MLLMs with Active Perception via Pure Reinforcement Learning](https://mlsanity.com/paper/2505.21457/) (2025)
- [Simple o3: Towards Interleaved Vision-Language Reasoning](https://mlsanity.com/paper/2508.12109/) (2025)
- [Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learning](https://mlsanity.com/paper/2510.10959/) (2025)
- [Revisiting the Necessity of Lengthy Chain-of-Thought in Vision-centric Reasoning Generalization](https://mlsanity.com/paper/2511.22586/) (2025)
- [Visual CoT Makes VLMs Smarter but More Fragile](https://mlsanity.com/paper/2509.23789/) (2025)

## Abstract (arXiv)

Visual agents employ external visual tools within visual chains of thought to incorporate fine-grained evidence. While prior work has mainly studied these tools in visual search tasks, their role in more complex visual reasoning remains underexplored. In this paper, we move beyond simple visual search tasks to investigate more challenging tasks, including 3D spatial reasoning and medical visual question answering, where agents must integrate tool-acquired local evidence with the global context. We identify a {tool-use collapse phenomenon: models progressively stop using tools while still achieving higher task accuracy. Moreover, we observe a clear asymmetry: (i) completely eliminating tool use degrades performance, whereas (ii) incentivizing tool use yields only marginal gains despite substantially increasing usage. We find that vanilla training and tool-use encouragement both reduce rollout diversity, explaining why higher tool use does not yield stronger reasoning performance. Motivated by these findings, we add an entropy regularization term to encourage diverse rollout exploration, achieving the best performance despite gradually declining tool usage. Overall, our findings suggest a training-time view of tools as scaffolding, where broader exploration over language generation and visual tool invocation improves reasoning despite tool-use collapse. Project page: https://scaffolded-exploration.github.io