mlsanityData to 7 Oct 2026

DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understanding

arXiv 2504.14920 ↗First posted 21 Apr 2025cs.CVcited by 47

Not on the map. Why?

Cited by 36 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Humans can effortlessly locate desired objects in cluttered environments, relying on a cognitive mechanism known as visual search to efficiently filter out irrelevant information and focus on task-related regions. Inspired by this process, we propose Dyfo (Dynamic Focus), a training-free dynamic focusing visual search method that enhances fine-grained visual understanding in large multimodal models (LMMs). Unlike existing approaches which require additional modules or data collection, Dyfo leverages a bidirectional interaction between LMMs and visual experts, using a Monte Carlo Tree Search (MCTS) algorithm to simulate human-like focus adjustments. This enables LMMs to focus on key visual regions while filtering out irrelevant content, without introducing additional training caused by vocabulary expansion or the integration of specialized localization modules. Experimental results demonstrate that Dyfo significantly improves fine-grained visual understanding and reduces hallucination issues in LMMs, achieving superior performance across both fixed and dynamic resolution models. The code is available at https://github.com/PKU-ICST-MIPL/DyFo_CVPR2025

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 43 of its 64 references that are in the map. First posted on arXiv 21 Apr 2025.

Builds on

43 of 64 references.

References · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  11. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  12. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
31 more
  1. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  2. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  3. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  4. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  5. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  6. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  7. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  8. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
  9. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  10. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  11. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  12. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
  13. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  14. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  15. 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
  16. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 498
  17. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
  18. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 447
  19. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 264
  20. 2023Compositional Chain-of-Thought Prompting for Large Multimodal Modelscited by 160
  21. 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 158
  22. 2023HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Datacited by 124
  23. 2024PSALM: Pixelwise SegmentAtion with Large Multi-Modal Modelcited by 100
  24. 2023Evaluation and Analysis of Hallucination in Large Vision-Language Modelscited by 99
  25. 2024MoVA: Adapting Mixture of Vision Experts to Multimodal Contextcited by 82
  26. 2023LISA++: An Improved Baseline for Reasoning Segmentation with Large Language Modelcited by 70
  27. 2020Predicting Goal-directed Human Attention Using Inverse Reinforcement Learningcited by 53
  28. 2018Finding any Waldo: zero-shot invariant and efficient visual searchcited by 28
  29. 2023RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentationcited by 24
  30. 2020Modeling human visual search: A combined Bayesian searcher and saliency map approach for eye movement guidance in natural scenescited by 6
  31. 2023Fine-grained Cross-modal Fusion based Refinement for Text-to-Image Synthesiscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 21; 2026: 26.

47 papers.

Sort:

By area:

  1. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
  2. 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 116
  3. 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMscited by 94
  4. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 88
  5. 2025Zoom-Refine: Boosting High-Resolution Multimodal Understanding via Localized Zoom and Self-Refinementcited by 18
  6. 2025HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decouplingcited by 16
  7. 2025Thinking in 360deg: Humanoid Visual Search in the Wildcited by 14
  8. 2025Look Where It Matters: Training-Free Ultra-HR Remote Sensing VQA via Adaptive Zoom Searchcited by 11
  9. 2025From Illusion to Intention: Visual Rationale Learning for Vision-Language Reasoningcited by 9
  10. 2026GeoEyes: On-Demand Visual Focusing for Evidence-Grounded Understanding of Ultra-High-Resolution Remote Sensing Imagerycited by 9
  11. 2026What, Whether and How? Unveiling Process Reward Models for Thinking with Images Reasoningcited by 7
  12. 2025Reinforcing VLMs to Use Tools for Detailed Visual Reasoning Under Resource Constraintscited by 6
35 more
  1. 2025CoFFT: Chain of Foresight-Focus Thought for Visual Language Modelscited by 6
  2. 2026DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Modelscited by 6
  3. 2026Improving Visual Reasoning with Iterative Evidence Refinementcited by 5
  4. 2025CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decodingcited by 4
  5. 2025Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMscited by 4
  6. 2025Thinking with Images via Self-Calling Agentcited by 4
  7. 2025Training Multi-Image Vision Agents via End2End Reinforcement Learningcited by 4
  8. 2026Glance-or-Gaze: Incentivizing LMMs to Adaptively Focus Search via Reinforcement Learningcited by 4
  9. 2025GeoViS: Geospatially Rewarded Visual Search for Remote Sensing Visual Groundingcited by 3
  10. 2026E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenescited by 3
  11. 2026DRS-GUI: Dynamic Region Search for Training-Free GUI Groundingcited by 3
  12. 2025Blink: Dynamic Visual Token Resolution for Enhanced Multimodal Understandingcited by 2
  13. 2026Seeing as Experts Do: A Knowledge-Augmented Agent for Open-Set Fine-Grained Visual Understandingcited by 2
  14. 2026Are Tools Always Beneficial? Learning to Invoke Tools Adaptively for Dual-Mode Multimodal LLM Reasoningcited by 2
  15. 2026Evaluating and Enhancing Negation Comprehension in Remote Sensing MLLMscited by 2
  16. 2025Iterative Zoom-In: Temporal Interval Exploration for Long Video Understandingcited by 1
  17. 2025Training-Free In-Context Forensic Chain for Image Manipulation Detection and Localizationcited by 1
  18. 2025MRD: Multi-resolution Retrieval-Detection Fusion for High-Resolution Image Understandingcited by 1
  19. 2025Figure It Out: Improve the Frontier of Reasoning with Executable Visual Statescited by 1
  20. 2026Learning to Focus and Precise Cropping: A Reinforcement Learning Framework with Information Gaps and Grounding Loss for MLLMscited by 1
  21. 2026Reinforce to Learn, Elect to Reason: A Dual Paradigm for Video Reasoningcited by 1
  22. 2026AutoFocus: Uncertainty-Aware Active Visual Search for GUI Groundingcited by 1
  23. 2026Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Modelscited by 1
  24. 2026ActiveScope: Actively Seeking and Correcting Perception for MLLMscited by 1
  25. 2026iGVLM: Dynamic Instruction-Guided Vision Encoding for Question-Aware Multimodal Understandingcited by 0
  26. 2026MAG-3D: Multi-Agent Grounded Reasoning for 3D Understandingcited by 0
  27. 2026Combating Visual Neglect and Semantic Drift in Large Multimodal Models for Enhanced Cross-Modal Retrievalcited by 0
  28. 2026Perceptual Flow Network for Visually Grounded Reasoningcited by 0
  29. 2026Beyond Thinking: Imagining in 360^circ for Humanoid Visual Searchcited by 0
  30. 2026Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Groundingcited by 0
  31. 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Modelscited by 0
  32. 2026Towards High-Resolution Visual Perception via Hierarchical Entity Explorationcited by 0
  33. 2026BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perceptioncited by 0
  34. 2026InSight-doc: Agentic Visual Perception for Long-Document Understandingcited by 0
  35. 2026LOCI: A Locator-Critic with Refinement Loopcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CVSearch: Empowering Multimodal LLMs with Cognitive Visual Search for High-Resolution Image Perception
  2. 2026LOCUS: Local Visual Cue Search for Enhancing Fine-Grained Perception in Multimodal Large Language Models
  3. 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration
  4. 2026VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs
  5. 2026Learning to See What You Need: Gaze Attention for Multimodal Large Language Models
  6. 2026LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models
  7. 2025Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoning
  8. 2025I Spy With My Model's Eye: Visual Search as a Behavioural Test for MLLMs