mlsanityData to 7 Oct 2026

Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs

arXiv 2505.15436 ↗First posted 21 May 2025cs.CVcited by 94

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Vision language models (VLMs) have achieved impressive performance across a variety of computer vision tasks. However, the multimodal reasoning capability has not been fully explored in existing models. In this paper, we propose a Chain-of-Focus (CoF) method that allows VLMs to perform adaptive focusing and zooming in on key image regions based on obtained visual cues and the given questions, achieving efficient multimodal reasoning. To enable this CoF capability, we present a two-stage training pipeline, including supervised fine-tuning (SFT) and reinforcement learning (RL). In the SFT stage, we construct the MM-CoF dataset, comprising 3K samples derived from a visual agent designed to adaptively identify key regions to solve visual tasks with different image resolutions and questions. We use MM-CoF to fine-tune the Qwen2.5-VL model for cold start. In the RL stage, we leverage the outcome accuracies and formats as rewards to update the Qwen2.5-VL model, enabling further refining the search and reasoning strategy of models without human priors. Our model achieves significant improvements on multiple benchmarks. On the V* benchmark that requires strong visual reasoning capability, our model outperforms existing VLMs by 5% among 8 image resolutions ranging from 224 to 4K, demonstrating the effectiveness of the proposed CoF method and facilitating the more efficient deployment of VLMs in practical applications.

Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 50 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

34 of 50 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
  2. 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning20 May
  3. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement10 Apr
  4. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization13 Mar
  5. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models13 Jun
  6. 2023Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models5 Dec

Other references · 28

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2023Segment Anythingcited by 8,158
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  10. 2025Qwen2.5-VL Technical Reportcited by 5,418
  11. 2024Qwen2.5 Technical Reportcited by 4,713
  12. 2024GPT-4o System Cardcited by 4,467
16 more
  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  3. 2023A Survey on Multimodal Large Language Modelscited by 898
  4. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  5. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  6. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
  7. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
  8. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 284
  9. 2020Detection and Tracking Meet Drones Challengecited by 261
  10. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 193
  11. 2018TallyQA: Answering Complex Counting Questionscited by 152
  12. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 151
  13. 2025Reward-Guided Speculative Decoding for Efficient LLM Reasoningcited by 106
  14. 2024Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usagecited by 49
  15. 2025DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understandingcited by 47
  16. 2024Instruction-Guided Visual Maskingcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 44; 2026: 50.

94 papers.

Sort:

By area:

  1. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  3. 2025Thyme: Think Beyond Imagescited by 160
  4. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 119
  5. 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 116
  6. 2025Latent Visual Reasoningcited by 99
  7. 2025Monet: Reasoning in Latent Visual Space Beyond Images and Languagecited by 75
  8. 2025PyVision: Agentic Vision with Dynamic Toolingcited by 71
  9. 2025VGR: Visual Grounded Reasoningcited by 70
  10. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 54
  11. 2025TIR-Bench: A Comprehensive Benchmark for Agentic Thinking-with-Images Reasoningcited by 34
  12. 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 28
38 more
  1. 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoningcited by 27
  2. 2025SIFThinker: Spatially-Aware Image Focus for Visual Reasoningcited by 25
  3. 2025Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Spacecited by 25
  4. 2025Simple o3: Towards Interleaved Vision-Language Reasoningcited by 24
  5. 2025Skyra: AI-Generated Video Detection via Grounded Artifact Reasoningcited by 24
  6. 2025Reasoning in the Dark: Interleaved Vision-Text Reasoning in Latent Spacecited by 20
  7. 2025SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RLcited by 20
  8. 2025A Survey on Agentic Multimodal Large Language Modelscited by 19
  9. 2025Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMscited by 16
  10. 2025CodeDance: A Dynamic Tool-integrated MLLM for Executable Visual Reasoningcited by 16
  11. 2026VTC-Bench: Evaluating Agentic Multimodal Models via Compositional Visual Tool Chainingcited by 12
  12. 2025ViPER: Empowering the Self-Evolution of Visual Perception Abilities in Vision-Language Modelcited by 11
  13. 2025DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoningcited by 10
  14. 2025Look Less, Reason More: Rollout-Guided Adaptive Pixel-Space Reasoningcited by 9
  15. 2025From Illusion to Intention: Visual Rationale Learning for Vision-Language Reasoningcited by 9
  16. 2026What Does Vision Tool-Use Reinforcement Learning Really Learn? Disentangling Tool-Induced and Intrinsic Effects for Crop-and-Zoomcited by 9
  17. 2025SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Modelscited by 8
  18. 2025Deep But Reliable: Advancing Multi-turn Reasoning for Thinking with Imagescited by 8
  19. 2025Mixture-of-Visual-Thoughts: Exploring Context-Adaptive Reasoning Mode Selection for General Visual Reasoningcited by 7
  20. 2026TerraScope: Pixel-Grounded Visual Reasoning for Earth Observationcited by 7
  21. 2025Small Drafts, Big Verdict: Information-Intensive Visual Reasoning via Speculationcited by 6
  22. 2025Video Spatial Reasoning with Object-Centric 3D Rolloutcited by 6
  23. 2026Zoom-IQA: Image Quality Assessment with Reliable Region-Aware Reasoningcited by 6
  24. 2026Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoningcited by 6
  25. 2025TAR: Temporal Anchor-Constrained Reasoning for Video Temporal Groundingcited by 5
  26. 2025VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoningcited by 5
  27. 2026MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learningcited by 5
  28. 2025Select Less, Reason More: Prioritizing Evidence Purity for Video Reasoningcited by 4
  29. 2026MEDVISTAGYM: A Scalable Training Environment for Thinking with Medical Images via Tool-Integrated Reinforcement Learningcited by 4
  30. 2026Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thoughtcited by 4
  31. 2026Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimizationcited by 4
  32. 2026Zoom Consistency: A Free Confidence Signal in Multi-Step Visual Grounding Pipelinescited by 4
  33. 2026Visual Reasoning through Tool-supervised Reinforcement Learningcited by 4
  34. 2026Do Multimodal Agents Really Benefit from Tool Use? A Systematic Study of Capability Gainscited by 4
  35. 2025edgeVLM: Cloud-edge Collaborative Real-time VLM based on Context Transfercited by 3
  36. 2025Learning Active Perception via Self-Evolving Preference Optimization for GUI Groundingcited by 3
  37. 2025GeoViS: Geospatially Rewarded Visual Search for Remote Sensing Visual Groundingcited by 3
  38. 2026Beyond Accuracy: Evaluating Grounded Visual Evidence in Thinking with Imagescited by 3

50 most cited of 94 All 94 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Weaving Context Across Images: Improving Vision-Language Models through Focus-Centric Visual Chains
  2. 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration
  3. 2026Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs
  4. 2026Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Models
  5. 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use
  6. 2026See Further, Think Deeper: Advancing VLM's Reasoning Ability with Low-level Visual Cues and Reflection
  7. 2025Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme
  8. 2025Activating Visual Context and Commonsense Reasoning through Masked Prediction in VLMs