mlsanityData to 7 Oct 2026

Global Semantic-Guided Sub-image Feature Weight Allocation in High-Resolution Large Vision-Language Models

arXiv 2501.14276 ↗First posted 24 Jan 2025cs.CV cs.AIcited by 4

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

As the demand for high-resolution image processing in Large Vision-Language Models (LVLMs) grows, sub-image partitioning has become a popular approach for mitigating visual information loss associated with fixed-resolution processing. However, existing partitioning methods uniformly process sub-images, resulting in suboptimal image understanding. In this work, we reveal that the sub-images with higher semantic relevance to the entire image encapsulate richer visual information for preserving the model's visual understanding ability. Therefore, we propose the Global Semantic-guided Weight Allocator (GSWA) module, which dynamically allocates weights to sub-images based on their relative information density, emulating human visual attention mechanisms. This approach enables the model to focus on more informative regions, overcoming the limitations of uniform treatment. We integrate GSWA into the InternVL2-2B framework to create SleighVL, a lightweight yet high-performing model. Extensive experiments demonstrate that SleighVL outperforms models with comparable parameters and remains competitive with larger models. Our work provides a promising direction for more efficient and contextually aware high-resolution image processing in LVLMs, advancing multimodal system development.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 11 of its 48 references that are in the map. First posted on arXiv 24 Jan 2025.

Builds on

11 of 48 references.

References · 11

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  3. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  4. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  5. 2024PaliGemma: A versatile 3B VLM for transfercited by 679
  6. 2024InternLM2 Technical Reportcited by 391
  7. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  8. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 284
  9. 2024Ovis: Structural Embedding Alignment for Multimodal Large Language Modelcited by 167
  10. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 151
  11. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 113

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025HRVVS: A High-resolution Video Vasculature Segmentation Network via Hierarchical Autoregressive Residual Priorscited by 1
  2. 2025Pyramid Token Pruning for High-Resolution Large Vision-Language Models via Region, Token, and Instruction-Guided Importancecited by 1
  3. 2025MARVIS: Modality Adaptive Reasoning over VISualizationscited by 0
  4. 2025Trust the Model: Compact VLMs as In-Context Judges for Image-Text Data Qualitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond the Vision Encoder: Identifying and Mitigating Spatial Bias in Large Vision-Language Models
  2. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
  3. 2026VISion On Request: Enhanced VLLM efficiency with sparse, dynamically selected, vision-language interactions
  4. 2024ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsification
  5. 2025HERO: Rethinking Visual Token Early Dropping in High-Resolution Large Vision-Language Models
  6. 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
  7. 2026Large Vision-Language Models Get Lost in Attention
  8. 2025Weaving Context Across Images: Improving Vision-Language Models through Focus-Centric Visual Chains