mlsanityData to 7 Oct 2026

Towards High-Resolution Visual Perception via Hierarchical Entity Exploration

arXiv 2607.00816 ↗First posted 1 Jul 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

High-resolution (HR) image perception remains a key challenge in multimodal large language models (MLLMs), as fine-grained details are often lost when the image is processed as a whole. Existing methods either require training to teach models where to look or heuristically divide the image into fixed regions, both of which struggle to generalize in complex HR scenes. In this work, we propose Hierarchical Entity Exploration (HEE), a training-free and model-agnostic framework that transforms static image understanding into dynamic, query-guided entity exploration. HEE first evaluates each region using a dual scoring mechanism to determine whether it already contains sufficient evidence to answer the question. If not, it applies object detection within the most promising region to extract fine-grained entities, clusters them into coherent subregions, and organizes them into a multi-level semantic hierarchy for deeper exploration. When deeper regions still fail to yield confident answers, a confidence-guided backtracking mechanism revisits alternative paths to ensure adaptive perception. Extensive results show that HEE outperforms training-free methods like ZoomEye and RAP in both accuracy and efficiency on two complex HR benchmarks (Visual Probe and HR-Bench), across different MLLMs such as Qwen2.5-VL and LLaVA-OneVision. Moreover, HEE demonstrates generalization on the MME-RealWorld benchmark.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 58 of its 64 references that are in the map. First posted on arXiv 1 Jul 2026.

Builds on

58 of 64 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025Where and What Matters: Sensitivity-Aware Task Vectors for Many-Shot Multimodal In-Context Learning11 Nov
  2. 2025HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decoupling28 Sep
  3. 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs24 Feb
  4. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  5. 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration25 Nov
  6. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models28 Aug
  7. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?23 Aug
  8. 2024LLaVA-OneVision: Easy Visual Task Transfer6 Aug
  9. 2024Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models5 Mar
  10. 2024InfiMM-HD: A Leap Forward in High-Resolution Multimodal Understanding3 Mar
  11. 2023Improved Baselines with Visual Instruction Tuning5 Oct
  12. 2023Investigating the Catastrophic Forgetting in Multimodal Large Language Models19 Sep
1 more
  1. 2023Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining?24 Aug

Other references · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2023Segment Anythingcited by 8,158
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  11. 2025Qwen2.5-VL Technical Reportcited by 5,418
  12. 2024GPT-4o System Cardcited by 4,467
33 more
  1. 2022A ConvNet for the 2020scited by 3,545
  2. 2023Qwen Technical Reportcited by 3,273
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  4. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  5. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  6. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  7. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  8. 2021Transformers in Vision: A Surveycited by 1,009
  9. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  10. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  11. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  12. 2024Yi: Open Foundation Models by 01.AIcited by 706
  13. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  14. 2024Qwen2-Audio Technical Reportcited by 543
  15. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 498
  16. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
  17. 2023Segment Anything in High Qualitycited by 373
  18. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 361
  19. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
  20. 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 116
  21. 2024DINO-X: A Unified Vision Model for Open-World Object Detection and Understandingcited by 85
  22. 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolvercited by 76
  23. 2025Tree Search for LLM Agent Reinforcement Learningcited by 73
  24. 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 63
  25. 2025DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understandingcited by 47
  26. 2024ConvLLaVA: Hierarchical Backbones as Visual Encoder for Large Multimodal Modelscited by 26
  27. 2021Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networkscited by 24
  28. 2023Deep Graph Reprogrammingcited by 22
  29. 2025Retrieval-Augmented Perception: High-Resolution Image Perception Meets Visual RAGcited by 22
  30. 2025Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimizationcited by 22
  31. 2022Where Does the Performance Improvement Come From? -- A Reproducibility Concern about Image-Text Retrievalcited by 19
  32. 2024WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledgecited by 16
  33. 2024GeReA: Question-Aware Prompt Captions for Knowledge-based Visual Question Answeringcited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CVSearch: Empowering Multimodal LLMs with Cognitive Visual Search for High-Resolution Image Perception
  2. 2026BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception
  3. 2026When Images Look Right and Retrieve Wrong: Coverage-Guided Cross-Scale Re-Indexing for Knowledge-Faithful Generative Perception
  4. 2026VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenes
  5. 2026Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models
  6. 2025A Training-Free, Task-Agnostic Framework for Enhancing MLLM Performance on High-Resolution Images
  7. 2024Emergent Visual-Semantic Hierarchies in Image-Text Representations
  8. 2024MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Models