mlsanityData to 14 Oct 2026

Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models

arXiv 2408.15556 ↗First posted 28 Aug 2024cs.CVcited by 161

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal large language models (MLLMs) have experienced significant advancements recently, but still struggle to recognize and interpret intricate details in high-resolution (HR) images effectively. While state-of-the-art (SOTA) MLLMs claim to process images at 4K resolution, existing MLLM benchmarks only support up to 2K, leaving the capabilities of SOTA models on true HR images largely untested. Furthermore, existing methods for enhancing HR image perception in MLLMs rely on computationally expensive visual instruction tuning. To address these limitations, we introduce HR-Bench, the first deliberately designed benchmark to rigorously evaluate MLLM performance on 4K&8K images. Through extensive experiments, we demonstrate that while downsampling HR images leads to vision information loss, leveraging complementary modalities, e.g., text, can effectively compensate for this loss. Building upon this insight, we propose Divide, Conquer and Combine (DC2), a novel training-free framework for enhancing MLLM perception of HR images. DC2 follows a three-staged approach: 1) Divide: recursively partitioning the HR image into patches and merging similar patches to minimize computational overhead, 2) Conquer: leveraging the MLLM to generate accurate textual descriptions for each image patch, and 3) Combine: utilizing the generated text descriptions to enhance the MLLM's understanding of the overall HR image. Extensive experiments show that: 1) the SOTA MLLM achieves 63% accuracy, which is markedly lower than the 87% accuracy achieved by humans on HR-Bench; 2) our DC2 brings consistent and significant improvements (a relative increase of +6% on HR-Bench and +8% on general multimodal benchmarks). The benchmark and code will be released to facilitate the multimodal R&D community.

Key facts. Cited by 161 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 44 references that are in the map. First posted on arXiv 28 Aug 2024.

Builds on

34 of 44 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024BLINK: Multimodal Large Language Models Can See but Not Perceive18 Apr
  2. 2024InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD9 Apr
  3. 2024InfiMM-HD: A Leap Forward in High-Resolution Multimodal Understanding3 Mar
  4. 2024The Revolution of Multimodal Large Language Models: A Survey19 Feb
  5. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model29 Jan
  6. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs11 Jan
  7. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks21 Dec
  8. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities4 Aug
  9. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul

Other references · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2023Segment Anythingcited by 8,234
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  8. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  9. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  10. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  11. 2019Towards VQA Models That Can Readcited by 1,718
  12. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
13 more
  1. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  2. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  3. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  4. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  5. 2022Token Merging: Your ViT But Fastercited by 1,021
  6. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  7. 2016A Diagram Is Worth A Dozen Imagescited by 804
  8. 2024Yi: Open Foundation Models by 01.AIcited by 719
  9. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  10. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  11. 2024Beyond LLaVA-HD: Diving into High-Resolution Large Multimodal Modelscited by 76
  12. 2024ConvLLaVA: Hierarchical Backbones as Visual Encoder for Large Multimodal Modelscited by 26
  13. 20243AM: An Ambiguity-Aware Multi-Modal Machine Translation Datasetcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 55; 2026: 102.

161 papers.

Sort:

By area:

  1. 2025Qwen3-VL Technical Reportcited by 2,793
  2. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 410
  3. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 254
  4. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 125
  5. 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 120
  6. 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Explorationcited by 103
  7. 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMscited by 96
  8. 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillationcited by 77
  9. 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokenscited by 71
  10. 2025Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodologycited by 69
  11. 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 50
  12. 2025TIR-Bench: A Comprehensive Benchmark for Agentic Thinking-with-Images Reasoningcited by 36
38 more
  1. 2025When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruningcited by 35
  2. 2025RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmarkcited by 33
  3. 2025One RL to See Them All: Visual Triple Unified Reinforcement Learningcited by 30
  4. 2025Scaling Vision Pre-Training to 4K Resolutioncited by 28
  5. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 28
  6. 2025Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearchcited by 27
  7. 2025CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimizationcited by 26
  8. 2025Retrieval-Augmented Perception: High-Resolution Image Perception Meets Visual RAGcited by 23
  9. 2025Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)cited by 23
  10. 2025ViSpec: Accelerating Vision-Language Models with Vision-Aware Speculative Decodingcited by 22
  11. 2025FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answeringcited by 20
  12. 2025Zoom-Refine: Boosting High-Resolution Multimodal Understanding via Localized Zoom and Self-Refinementcited by 18
  13. 2025HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decouplingcited by 17
  14. 2025On the Faithfulness of Visual Thinking: Measurement and Enhancementcited by 17
  15. 2025QSVD: Efficient Low-rank Approximation for Unified Query-Key-Value Weight Compression in Low-Precision Vision-Language Modelscited by 16
  16. 2025CodeDance: A Dynamic Tool-integrated MLLM for Executable Visual Reasoningcited by 16
  17. 2026PyVision-RL: Forging Open Agentic Vision Models via RLcited by 16
  18. 2026Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Modelscited by 14
  19. 2026VTC-Bench: Evaluating Agentic Multimodal Models via Compositional Visual Tool Chainingcited by 12
  20. 2025Native Visual Understanding: Resolving Resolution Dilemmas in Vision-Language Modelscited by 11
  21. 2025Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Modelscited by 11
  22. 2025Look Where It Matters: Training-Free Ultra-HR Remote Sensing VQA via Adaptive Zoom Searchcited by 11
  23. 2025A Benchmark for Ultra-High-Resolution Remote Sensing MLLMscited by 11
  24. 2026V-Zero: Answer-Label-Free On-Policy Distillation with Contrastive Evidence Gating for Fine-Grained Visual Reasoningcited by 11
  25. 2025InfiniteVL: Synergizing Linear and Sparse Attention for Highly-Efficient, Unlimited-Input Vision-Language Modelscited by 10
  26. 2026Visual Contrastive Self-Distillationcited by 10
  27. 2025Information Density Principle for MLLM Benchmarkscited by 9
  28. 2025Catching the Details: Self-Distilled RoI Predictors for Fine-Grained MLLM Perceptioncited by 9
  29. 2025From Illusion to Intention: Visual Rationale Learning for Vision-Language Reasoningcited by 9
  30. 2025GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluationcited by 9
  31. 2026What Does Vision Tool-Use Reinforcement Learning Really Learn? Disentangling Tool-Induced and Intrinsic Effects for Crop-and-Zoomcited by 9
  32. 2026Thinking Without Images: Internalizing Visual Manipulation with On-Policy Self-Distillationcited by 9
  33. 2024LLaVA-UHD v2: an MLLM Integrating High-Resolution Semantic Pyramid via Hierarchical Window Transformercited by 8
  34. 20254D-Bench: Benchmarking Multi-modal Large Language Models for 4D Object Understandingcited by 8
  35. 2025CropVLM: Learning to Zoom for Fine-Grained Vision-Language Perceptioncited by 8
  36. 2025Revisiting the Necessity of Lengthy Chain-of-Thought in Vision-centric Reasoning Generalizationcited by 8
  37. 2026UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMscited by 8
  38. 2026OPD-V: Visual On-Policy Self-Distillation with Modality Balancecited by 8

50 most cited of 161 All 161 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Demystifying the Visual Quality Paradox in Multimodal Large Language Models
  2. 2024DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception
  3. 2024MIBench: Evaluating Multimodal Large Language Models over Multiple Images
  4. 2024Incorporating Visual Experts to Resolve the Information Loss in Multimodal Large Language Models
  5. 2025Lifting the Veil on Visual Information Flow in MLLMs: Unlocking Pathways to Faster Inference
  6. 2025Perceiving Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models
  7. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
  8. 2024II-Bench: An Image Implication Understanding Benchmark for Multimodal Large Language Models