Towards High-Resolution Visual Perception via Hierarchical Entity Exploration
On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning
Abstract (arXiv)
High-resolution (HR) image perception remains a key challenge in multimodal large language models (MLLMs), as fine-grained details are often lost when the image is processed as a whole. Existing methods either require training to teach models where to look or heuristically divide the image into fixed regions, both of which struggle to generalize in complex HR scenes. In this work, we propose Hierarchical Entity Exploration (HEE), a training-free and model-agnostic framework that transforms static image understanding into dynamic, query-guided entity exploration. HEE first evaluates each region using a dual scoring mechanism to determine whether it already contains sufficient evidence to answer the question. If not, it applies object detection within the most promising region to extract fine-grained entities, clusters them into coherent subregions, and organizes them into a multi-level semantic hierarchy for deeper exploration. When deeper regions still fail to yield confident answers, a confidence-guided backtracking mechanism revisits alternative paths to ensure adaptive perception. Extensive results show that HEE outperforms training-free methods like ZoomEye and RAP in both accuracy and efficiency on two complex HR benchmarks (Visual Probe and HR-Bench), across different MLLMs such as Qwen2.5-VL and LLaVA-OneVision. Moreover, HEE demonstrates generalization on the MME-RealWorld benchmark.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 58 of its 64 references that are in the map. First posted on arXiv 1 Jul 2026.
Builds on
58 of 64 references.
Closest ancestors · 13
Same direction, newest first
- 2025Where and What Matters: Sensitivity-Aware Task Vectors for Many-Shot Multimodal In-Context Learning11 Nov
- 2025HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decoupling28 Sep
- 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs24 Feb
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
- 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration25 Nov
- 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models28 Aug
- 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?23 Aug
- 2024LLaVA-OneVision: Easy Visual Task Transfer6 Aug
- 2024Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models5 Mar
- 2024InfiMM-HD: A Leap Forward in High-Resolution Multimodal Understanding3 Mar
- 2023Improved Baselines with Visual Instruction Tuning5 Oct
- 2023Investigating the Catastrophic Forgetting in Multimodal Large Language Models19 Sep
1 more
Other references · 45
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Segment Anythingcited by 8,158
- 2023Visual Instruction Tuningcited by 7,792
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
33 more
- 2022A ConvNet for the 2020scited by 3,545
- 2023Qwen Technical Reportcited by 3,273
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
- 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
- 2021Transformers in Vision: A Surveycited by 1,009
- 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
- 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
- 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
- 2024Yi: Open Foundation Models by 01.AIcited by 706
- 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
- 2024Qwen2-Audio Technical Reportcited by 543
- 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 498
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
- 2023Segment Anything in High Qualitycited by 373
- 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 361
- 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
- 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 116
- 2024DINO-X: A Unified Vision Model for Open-World Object Detection and Understandingcited by 85
- 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolvercited by 76
- 2025Tree Search for LLM Agent Reinforcement Learningcited by 73
- 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 63
- 2025DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understandingcited by 47
- 2024ConvLLaVA: Hierarchical Backbones as Visual Encoder for Large Multimodal Modelscited by 26
- 2021Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networkscited by 24
- 2023Deep Graph Reprogrammingcited by 22
- 2025Retrieval-Augmented Perception: High-Resolution Image Perception Meets Visual RAGcited by 22
- 2025Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimizationcited by 22
- 2022Where Does the Performance Improvement Come From? -- A Reproducibility Concern about Image-Text Retrievalcited by 19
- 2024WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledgecited by 16
- 2024GeReA: Question-Aware Prompt Captions for Knowledge-based Visual Question Answeringcited by 9
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026CVSearch: Empowering Multimodal LLMs with Cognitive Visual Search for High-Resolution Image Perception
- 2026BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception
- 2026When Images Look Right and Retrieve Wrong: Coverage-Guided Cross-Scale Re-Indexing for Knowledge-Faithful Generative Perception
- 2026VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenes
- 2026Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models
- 2025A Training-Free, Task-Agnostic Framework for Enhancing MLLM Performance on High-Resolution Images
- 2024Emergent Visual-Semantic Hierarchies in Image-Text Representations
- 2024MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Models