From Inheritance to Saturation: Disentangling the Evolution of Visual Redundancy for Architecture-Aware MLLM Inference Acceleration
On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency
Abstract (arXiv)
High-resolution Multimodal Large Language Models (MLLMs) face prohibitive computational costs during inference due to the explosion of visual tokens. Existing acceleration strategies, such as token pruning or layer sparsity, suffer from severe "backbone dependency", performing well on Vicuna or Mistral architectures (e.g., LLaVA) but causing significant performance degradation when transferred to architectures like Qwen. To address this, we leverage truncated matrix entropy to uncover a universal three-stage inference lifecycle, decoupling visual redundancy into universal Intrinsic Visual Redundancy (IVR) and architecture-dependent Secondary Saturation Redundancy (SSR). Guided by this insight, we propose HalfV, a framework that first mitigates IVR via a unified pruning strategy and then adaptively handles SSR based on its specific manifestation. Experiments demonstrate that HalfV achieves superior efficiency-performance trade-offs across diverse backbones. Notably, on Qwen25-VL, it retains 96.8% performance at a 4.1× FLOPs speedup, significantly outperforming state-of-the-art baselines. Our code is available at https://github.com/civilizwa/HalfV.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 37 references that are in the map. First posted on arXiv 8 Apr 2026.
Builds on
21 of 37 references.
Closest ancestors · 8
Same direction, newest first
- 2025Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization28 May
- 2025FlowCut: Rethinking Redundancy via Information Flow for Efficient Vision-Language Models26 May
- 2025ShortV: Efficient Multimodal Large Language Models by Freezing Visual Tokens in Ineffective Layers1 Apr
- 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More17 Feb
- 2024Feather the Throttle: Revisiting Visual Token Pruning for Vision-Language Model Acceleration17 Dec
- 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction22 Oct
- 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct
- 2024Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference9 May
Other references · 13
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
- 2019Towards VQA Models That Can Readcited by 1,696
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2016A Diagram Is Worth A Dozen Imagescited by 793
- 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 103
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025RedundancyLens: Revealing and Exploiting Visual Token Processing Redundancy for Efficient Decoder-Only MLLMs
- 2026Structured Redundancy Modeling for Efficient Visual Token Pruning in High-Resolution MLLMs
- 2026AdaVSkip: Adaptive Visual Token Skipping Across Layers For Efficient MLLMs Inference
- 2025Lifting the Veil on Visual Information Flow in MLLMs: Unlocking Pathways to Faster Inference
- 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
- 2025RedVTP: Training-Free Acceleration of Diffusion Vision-Language Models Inference via Masked Token-Guided Visual Token Pruning
- 2026Reducing Peak Memory Usage for Modern Multimodal Large Language Model Pipelines
- 2025A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models