Beyond the Eye: Efficient Multimodal Reasoning via Self-Regulated Implicit Visual Tools
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Recent multimodal large language models (MLLMs) have made remarkable progress on fine-grained perception tasks under the "Thinking with Images" (TwI) paradigm by iteratively performing various visual tool operations. However, this paradigm relies heavily on frequent external tool calls and repeated image re-encoding, which leads to substantial computational overhead and inference latency. To address these issues, we propose Beyond the Eye (BEE), a novel implicit visual tool paradigm centered on self-regulated capability. BEE directly incorporates visual tool invocation behaviors into the training objective and encourages the model to develop a self-regulated invocation mechanism. This design enables the model to adaptively balance internal knowledge and implicit tools, avoiding redundant tool usage while substantially reducing inference latency. Specifically, BEE involves a two-stage training process: (1) Formalized Chain-of-Thought (CoT) Supervised Fine-tuning (SFT). We construct CoT trajectories with structured tool slots and mixed invocation states. This stage activates the model's implicit tool representations and adaptive switching capability. (2) Self-regulated Reward-Driven Alignment. To address redundant tool usage caused by ambiguous cognitive boundaries, we first introduce the Net Tool Gain (NTG) metric to quantify this phenomenon. Based on this observation, we further propose a self-regulated reward mechanism. This mechanism penalizes ineffective tool dependency and encourages the model to perform knowledge routing, ensuring that implicit tools are invoked only when the model's internal knowledge is insufficient. BEE achieves state-of-the-art performance in fine-grained visual perception while remaining competitive in general reasoning tasks and achieving substantial gains in inference efficiency.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 102 references that are in the map. First posted on arXiv 13 Jul 2026.
Builds on
22 of 102 references.
Closest ancestors · 11
Same direction, newest first
- 2026Latent Visual States for Efficient Multimodal Reasoning23 Jun
- 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens24 Nov
- 2025Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMs28 Oct
- 2025Reinforced Visual Perception with Tools1 Sep
- 2025Multimodal Chain of Continuous Thought for Latent-Space Reasoning in Vision-Language Models18 Aug
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning1 Jul
- 2025MiMo-VL Technical Report4 Jun
- 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey16 Mar
- 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
- 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model7 Mar
- 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts6 Jul
Other references · 11
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2024OpenAI o1 System Cardcited by 1,244
- 2025Kimi K2: Open Agentic Intelligencecited by 567
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
- 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoningcited by 78
- 2025Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Spacecited by 43
- 2026The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlookcited by 43
- 2025Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearchcited by 27
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoning
- 2025Thinking with Programming Vision: Towards a Unified View for Thinking with Images
- 2026Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs
- 2025Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space
- 2026Adaptive Look, Flexible Thought: Learning How and How Long to Think in Multimodal Reasoning
- 2025DeepSketcher: Internalizing Visual Manipulation for Multimodal Reasoning
- 2026Reflect to Inform: Boosting Multimodal Reasoning via Information-Gain-Driven Verification
- 2024Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination