mlsanityData to 14 Oct 2026

Zoom-Refine: Boosting High-Resolution Multimodal Understanding via Localized Zoom and Self-Refinement

arXiv 2506.01663 ↗First posted 2 Jun 2025cs.CVcited by 18

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal Large Language Models (MLLM) often struggle to interpret high-resolution images accurately, where fine-grained details are crucial for complex visual understanding. We introduce Zoom-Refine, a novel training-free method that enhances MLLM capabilities to address this issue. Zoom-Refine operates through a synergistic process of textitLocalized Zoom and textitSelf-Refinement. In the textitLocalized Zoom step, Zoom-Refine leverages the MLLM to provide a preliminary response to an input query and identifies the most task-relevant image region by predicting its bounding box coordinates. During the textitSelf-Refinement step, Zoom-Refine then integrates fine-grained details from the high-resolution crop (identified by textitLocalized Zoom) with its initial reasoning to re-evaluate and refine its preliminary response. Our method harnesses the MLLM's inherent capabilities for spatial localization, contextual reasoning and comparative analysis without requiring additional training or external experts. Comprehensive experiments demonstrate the efficacy of Zoom-Refine on two challenging high-resolution multimodal benchmarks. Code is available at hrefhttps://github.com/xavier-yu114/Zoom-Refinecolormagentagithub.com/xavier-yu114/Zoom-Refine

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 59 of its 73 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

59 of 73 references.

Closest ancestors · 23

Same direction, newest first

  1. 2025Zoomer: Adaptive Image Focus Optimization for Black-box MLLM30 Apr
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  3. 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration25 Nov
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
  5. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models28 Aug
  6. 2024MaVEn: An Effective Multi-granularity Hybrid Visual Encoding Framework for Multimodal Large Language Model22 Aug
  7. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs24 Jun
  8. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks12 Jun
  9. 2024Wings: Learning Multimodal LLMs without Text-only Forgetting5 Jun
  10. 2024Visual Anchors Are Strong Information Aggregators For Multimodal Large Language Model28 May
  11. 2024CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts9 May
  12. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites25 Apr
11 more
  1. 2024TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding15 Apr
  2. 2024InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD9 Apr
  3. 2024Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models5 Mar
  4. 2024DualFocus: Integrating Macro and Micro Perspectives in Multi-modal Large Language Models22 Feb
  5. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model29 Jan
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks21 Dec
  7. 2023VILA: On Pre-training for Visual Language Models12 Dec
  8. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captions21 Nov
  9. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models11 Nov
  10. 2023Improved Baselines with Visual Instruction Tuning5 Oct
  11. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuning5 May

Other references · 36

  1. 2023Visual Instruction Tuningcited by 7,869
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2023Qwen Technical Reportcited by 3,344
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  8. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  9. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  10. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  11. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  12. 2024Yi: Open Foundation Models by 01.AIcited by 719
24 more
  1. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  2. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  3. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  4. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  5. 2023Language Models can Solve Computer Taskscited by 372
  6. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 365
  7. 2024What matters when building vision-language models?cited by 326
  8. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 324
  9. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  10. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 280
  11. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  12. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
  13. 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 217
  14. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 174
  15. 2023mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Videocited by 154
  16. 2024Beyond LLaVA-HD: Diving into High-Resolution Large Multimodal Modelscited by 76
  17. 2025DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understandingcited by 49
  18. 2024Large Language Models Can Self-Correct with Key Condition Verificationcited by 39
  19. 2024Small Language Model Can Self-correctcited by 23
  20. 2023HRVQA: A Visual Question Answering Benchmark for High-Resolution Aerial Imagescited by 21
  21. 2024Leveraging Visual Tokens for Extended Text Contexts in Multi-Modal Learningcited by 17
  22. 2024When Hindsight is Not 20/20: Testing Limits on Reflective Thinking in Large Language Modelscited by 12
  23. 2024Graph-based Unsupervised Disentangled Representation Learning via Multimodal Large Language Modelscited by 10
  24. 2024Aggregate-and-Adapt Natural Language Prompts for Downstream Generalization of CLIPcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 17.

18 papers.

Sort:

By area:

  1. 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillationcited by 77
  2. 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 50
  3. 2026DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Modelscited by 6
  4. 2026Improving Visual Reasoning with Iterative Evidence Refinementcited by 5
  5. 2026TikArt: Stabilizing Aperture-Guided Fine-Grained Visual Reasoning with Reinforcement Learningcited by 4
  6. 2026Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoningcited by 3
  7. 2026SPD-Faith Bench: Diagnosing and Improving Faithfulness in Chain-of-Thought for Multimodal Large Language Modelscited by 2
  8. 2025MRD: Multi-resolution Retrieval-Detection Fusion for High-Resolution Image Understandingcited by 1
  9. 2026Learning to Look Again: Loss-Gap Supervision for Free-form Crop Routing in Vision-Language Modelscited by 1
  10. 2026Perceptual Flow Network for Visually Grounded Reasoningcited by 0
  11. 2026ROVER: Routing Object-Centric Visual Evidence for Grounded Multi-Image Reasoningcited by 0
  12. 2026Mags-RL: Wearing Multimodal LLMs a Magnifying Glass via Agentic Reinforcement Learning For Complex Scene Reasoningcited by 0
6 more
  1. 2026Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Groundingcited by 0
  2. 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Modelscited by 0
  3. 2026VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Contextcited by 0
  4. 2026Zero-MELO: Test-Time Evidence Calibration with Multimodal LLMs for Zero-Shot Micro-Gesture Recognitioncited by 0
  5. 2026Paint What You See: Benchmarking Dexterous Visual Tool Use in Multimodal Agentscited by 0
  6. 2026JRDB-AVR: An Active Visual Reasoning Benchmark for Embodied Agents in Real-World Environmentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models
  2. 2025HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decoupling
  3. 2025Visual Funnel: Resolving Contextual Blindness in Multimodal Large Language Models
  4. 2026Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models
  5. 2025Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping
  6. 2025PixelRefer: A Unified Framework for Spatio-Temporal Object Referring with Arbitrary Granularity
  7. 2024AdaptVision: Dynamic Input Scaling in MLLMs for Versatile Scene Understanding
  8. 2025A Training-Free, Task-Agnostic Framework for Enhancing MLLM Performance on High-Resolution Images