mlsanityData to 14 Oct 2026

Global Compression Commander: Plug-and-Play Inference Acceleration for High-Resolution Large Vision-Language Models

arXiv 2501.05179 ↗First posted 9 Jan 2025cs.CVcited by 32

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

Large vision-language models (LVLMs) excel at visual understanding, but face efficiency challenges due to quadratic complexity in processing long multi-modal contexts. While token compression can reduce computational costs, existing approaches are designed for single-view LVLMs and fail to consider the unique multi-view characteristics of high-resolution LVLMs with dynamic cropping. Existing methods treat all tokens uniformly, but our analysis reveals that global thumbnails can naturally guide the compression of local crops by providing holistic context for informativeness evaluation. In this paper, we first analyze dynamic cropping strategy, revealing both the complementary nature between thumbnails and crops, and the distinctive characteristics across different crops. Based on our observations, we propose ``Global Compression Commander'' (textiti.e., GlobalCom2), a novel plug-and-play token compression framework for HR-LVLMs. GlobalCom2 leverages thumbnail as the ``commander'' to guide the compression of local crops, adaptively preserving informative details while eliminating redundancy. Extensive experiments show that GlobalCom2 maintains over 90% performance while compressing 90% visual tokens, reducing FLOPs and peak memory to 9.1% and 60%.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 51 of its 65 references that are in the map. First posted on arXiv 9 Jan 2025.

Builds on

51 of 65 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024FrameFusion: Combining Similarity and Importance for Video Token Reduction on Large Vision Language Models30 Dec
  2. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Models5 Dec
  3. 2024Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs2 Dec
  4. 2024Filter, Correlate, Compress: Training-Free Token Reduction for MLLM Acceleration26 Nov
  5. 2024FoPru: Focal Pruning for Efficient Large Vision-Language Models21 Nov
  6. 2024Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model16 Nov
  7. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction22 Oct
  8. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct
  9. 2024Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models16 Sep
  10. 2024Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning for Efficient Adaptation7 Aug
  11. 2024VoCo-LLaMA: Towards Vision Compression with Large Language Models18 Jun
  12. 2024Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference9 May
2 more
  1. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models22 Mar
  2. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  10. 2023Qwen Technical Reportcited by 3,344
  11. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  12. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
25 more
  1. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  2. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  3. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  4. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  5. 2019Towards VQA Models That Can Readcited by 1,718
  6. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  7. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  8. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  9. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  10. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  11. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  12. 2022Token Merging: Your ViT But Fastercited by 1,021
  13. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  14. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  15. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  16. 2016A Diagram Is Worth A Dozen Imagescited by 804
  17. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  18. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 365
  19. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 363
  20. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 229
  21. 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 197
  22. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
  23. 2024Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Modelscited by 113
  24. 2024LLaVA-KD: A Framework of Distilling Multimodal Large Language Modelscited by 40
  25. 2024BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile Devicescited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 17; 2026: 15.

32 papers.

Sort:

By area:

  1. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 64
  2. 2025GUI-G2: Gaussian Reward Modeling for GUI Groundingcited by 63
  3. 2025HoliTom: Holistic Token Merging for Fast Video Large Language Modelscited by 58
  4. 2025StreamingTOM: Streaming Token Compression for Efficient Video Understandingcited by 43
  5. 2025LongCodeZip: Compress Long Context for Code Language Modelscited by 39
  6. 2025Accelerating Streaming Video Large Language Models via Hierarchical Token Compressioncited by 37
  7. 2025Shifting AI Efficiency From Model-Centric to Data-Centric Compressioncited by 29
  8. 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Groundingcited by 29
  9. 2025Variation-aware Vision Token Dropping for Faster Large Vision-Language Modelscited by 27
  10. 2026OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Modelscited by 25
  11. 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMscited by 24
  12. 2025Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Drivingcited by 16
20 more
  1. 2026N"uwa: Mending the Spatial Integrity Torn by VLM Token Pruningcited by 15
  2. 2025Mixing Importance with Diversity: Joint Optimization for KV Cache Compression in Large Vision-Language Modelscited by 14
  3. 2026Innovator-VL: A Multimodal Large Language Model for Scientific Discoverycited by 8
  4. 2025ToDRE: Effective Visual Token Pruning via Token Diversity and Task Relevancecited by 7
  5. 2025IPCV: Information-Preserving Compression for MLLM Visual Encoderscited by 5
  6. 2026EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labelingcited by 5
  7. 2026EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistantcited by 5
  8. 2026SVAgent: Storyline-Guided Long Video Understanding via Cross-Modal Multi-Agent Collaborationcited by 4
  9. 2026EarlyTom: Early Token Compression Completes Fast Video Understandingcited by 4
  10. 2025A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Modelscited by 3
  11. 2025OmniDocLayout: Towards Diverse Document Layout Generation via Coarse-to-Fine LLM Learningcited by 2
  12. 2026Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unificationcited by 2
  13. 2026Bridging Visual Representation and Reinforcement Learning from Verifiable Rewards in Large Vision-Language Modelscited by 2
  14. 2026V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Modelscited by 2
  15. 2026HeadRouter: Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Modelscited by 1
  16. 2026ERASE: Eliminating Redundant Visual Tokens via Adaptive Two-Stage Token Pruningcited by 1
  17. 2025Robust RGB-T Tracking via Learnable Visual Fourier Prompt Fine-tuning and Modality Fusion Prompt Generationcited by 0
  18. 2025ShaRP: SHAllow-LayeR Pruning for Efficient Video Large Language Modelscited by 0
  19. 2026ForensicZip: More Tokens are Better but Not Necessary in Forensic Vision-Language Modelscited by 0
  20. 2026Differentiable Efficient Operator Searchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMs
  2. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
  3. 2025LLMC+: Benchmarking Vision-Language Model Compression with a Plug-and-play Toolkit
  4. 2025A Glimpse to Compress: Dynamic Visual Token Pruning for Large Vision-Language Models
  5. 2025CROP: Contextual Region-Oriented Visual Token Pruning
  6. 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
  7. 2025LLaVA-UHD v3: Progressive Visual Compression for Efficient Native-Resolution Encoding in MLLMs
  8. 2026ApET: Approximation-Error Guided Token Compression for Efficient VLMs