# Zoom-Refine: Boosting High-Resolution Multimodal Understanding via Localized Zoom and Self-Refinement

arXiv: https://arxiv.org/abs/2506.01663 · HTML: https://mlsanity.com/paper/2506.01663/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 59 of its 73 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 30 of 59)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://mlsanity.com/paper/2311.12793/) (2023)
- [Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs](https://mlsanity.com/paper/2406.16860/) (2024)
- [DeepSeek-VL: Towards Real-World Vision-Language Understanding](https://mlsanity.com/paper/2403.05525/) (2024)
- [MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies](https://mlsanity.com/paper/2404.06395/) (2024)
- [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024)
- [CogVLM: Visual Expert for Pretrained Language Models](https://mlsanity.com/paper/2311.03079/) (2023)
- [VILA: On Pre-training for Visual Language Models](https://mlsanity.com/paper/2312.07533/) (2023)
- [MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning](https://mlsanity.com/paper/2310.09478/) (2023)
- [Otter: A Multi-Modal Model with In-Context Instruction Tuning](https://mlsanity.com/paper/2305.03726/) (2023)
- [V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs](https://mlsanity.com/paper/2312.14135/) (2023)
- [Training Language Models to Self-Correct via Reinforcement Learning](https://mlsanity.com/paper/2409.12917/) (2024)
- [Language Models can Solve Computer Tasks](https://mlsanity.com/paper/2303.17491/) (2023)
- [Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models](https://mlsanity.com/paper/2403.18814/) (2024)
- [Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models](https://mlsanity.com/paper/2311.06607/) (2023)
- [InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model](https://mlsanity.com/paper/2401.16420/) (2024)
- [What matters when building vision-language models?](https://mlsanity.com/paper/2405.02246/) (2024)

## Built on by (most cited first, 18 of 18)

- [Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation](https://mlsanity.com/paper/2605.18740/) (2026)
- [Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception](https://mlsanity.com/paper/2602.11858/) (2026)
- [DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models](https://mlsanity.com/paper/2603.03857/) (2026)
- [Improving Visual Reasoning with Iterative Evidence Refinement](https://mlsanity.com/paper/2603.14117/) (2026)
- [TikArt: Stabilizing Aperture-Guided Fine-Grained Visual Reasoning with Reinforcement Learning](https://mlsanity.com/paper/2602.14482/) (2026)
- [Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoning](https://mlsanity.com/paper/2607.01191/) (2026)
- [SPD-Faith Bench: Diagnosing and Improving Faithfulness in Chain-of-Thought for Multimodal Large Language Models](https://mlsanity.com/paper/2602.07833/) (2026)
- [MRD: Multi-resolution Retrieval-Detection Fusion for High-Resolution Image Understanding](https://mlsanity.com/paper/2512.02906/) (2025)
- [Learning to Look Again: Loss-Gap Supervision for Free-form Crop Routing in Vision-Language Models](https://mlsanity.com/paper/2608.21762/) (2026)
- [Perceptual Flow Network for Visually Grounded Reasoning](https://mlsanity.com/paper/2605.02730/) (2026)
- [ROVER: Routing Object-Centric Visual Evidence for Grounded Multi-Image Reasoning](https://mlsanity.com/paper/2605.27959/) (2026)
- [Mags-RL: Wearing Multimodal LLMs a Magnifying Glass via Agentic Reinforcement Learning For Complex Scene Reasoning](https://mlsanity.com/paper/2605.27960/) (2026)
- [Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Grounding](https://mlsanity.com/paper/2606.16158/) (2026)
- [From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models](https://mlsanity.com/paper/2606.26196/) (2026)
- [VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Context](https://mlsanity.com/paper/2606.30288/) (2026)
- [Zero-MELO: Test-Time Evidence Calibration with Multimodal LLMs for Zero-Shot Micro-Gesture Recognition](https://mlsanity.com/paper/2608.14854/) (2026)
- [Paint What You See: Benchmarking Dexterous Visual Tool Use in Multimodal Agents](https://mlsanity.com/paper/2608.25417/) (2026)
- [JRDB-AVR: An Active Visual Reasoning Benchmark for Embodied Agents in Real-World Environments](https://mlsanity.com/paper/2609.35032/) (2026)

## Abstract (arXiv)

Multimodal Large Language Models (MLLM) often struggle to interpret high-resolution images accurately, where fine-grained details are crucial for complex visual understanding. We introduce Zoom-Refine, a novel training-free method that enhances MLLM capabilities to address this issue. Zoom-Refine operates through a synergistic process of textitLocalized Zoom and textitSelf-Refinement. In the textitLocalized Zoom step, Zoom-Refine leverages the MLLM to provide a preliminary response to an input query and identifies the most task-relevant image region by predicting its bounding box coordinates. During the textitSelf-Refinement step, Zoom-Refine then integrates fine-grained details from the high-resolution crop (identified by textitLocalized Zoom) with its initial reasoning to re-evaluate and refine its preliminary response. Our method harnesses the MLLM's inherent capabilities for spatial localization, contextual reasoning and comparative analysis without requiring additional training or external experts. Comprehensive experiments demonstrate the efficacy of Zoom-Refine on two challenging high-resolution multimodal benchmarks. Code is available at hrefhttps://github.com/xavier-yu114/Zoom-Refinecolormagentagithub.com/xavier-yu114/Zoom-Refine