# Video Compression Commander: Plug-and-Play Inference Acceleration for Video Large Language Models

arXiv: https://arxiv.org/abs/2505.14454 · HTML: https://mlsanity.com/paper/2505.14454/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving.
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 20 May 2025.

## Placed in

- Efficient LLM inference and serving / [Visual token pruning in multimodal models](https://mlsanity.com/inference/visual-token-pruning-in-multimodal-models/)

## Builds on (most cited first, 16 of 16)

- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding](https://mlsanity.com/paper/2308.09126/) (2023)
- [LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding](https://mlsanity.com/paper/2407.15754/) (2024)
- [SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference](https://mlsanity.com/paper/2410.04417/) (2024)
- [LongVILA: Scaling Long-Context Visual Language Models for Long Videos](https://mlsanity.com/paper/2408.10188/) (2024)
- [InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling](https://mlsanity.com/paper/2501.12386/) (2025)
- [Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More](https://mlsanity.com/paper/2502.11494/) (2025)
- [Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?](https://mlsanity.com/paper/2502.11501/) (2025)
- [MDP3: A Training-free Approach for List-wise Frame Selection in Video-LLMs](https://mlsanity.com/paper/2501.02885/) (2025)

## Built on by (most cited first, 30 of 38)

- [GUI-G2: Gaussian Reward Modeling for GUI Grounding](https://mlsanity.com/paper/2507.15846/) (2025)
- [StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding](https://mlsanity.com/paper/2508.15717/) (2025)
- [Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality](https://mlsanity.com/paper/2505.18227/) (2025)
- [StreamingTOM: Streaming Token Compression for Efficient Video Understanding](https://mlsanity.com/paper/2510.18269/) (2025)
- [LongCodeZip: Compress Long Context for Code Language Models](https://mlsanity.com/paper/2510.00446/) (2025)
- [Accelerating Streaming Video Large Language Models via Hierarchical Token Compression](https://mlsanity.com/paper/2512.00891/) (2025)
- [VideoITG: Multimodal Video Understanding with Instructed Temporal Grounding](https://mlsanity.com/paper/2507.13353/) (2025)
- [Variation-aware Vision Token Dropping for Faster Large Vision-Language Models](https://mlsanity.com/paper/2509.01552/) (2025)
- [A Survey of Token Compression for Efficient Multimodal Large Language Models](https://mlsanity.com/paper/2507.20198/) (2025)
- [OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Models](https://mlsanity.com/paper/2602.04804/) (2026)
- [OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models](https://mlsanity.com/paper/2511.14582/) (2025)
- [Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Driving](https://mlsanity.com/paper/2508.13305/) (2025)
- [UniComp: Rethinking Video Compression Through Informational Uniqueness](https://mlsanity.com/paper/2512.03575/) (2025)
- [MARC: Memory-Augmented RL Token Compression for Efficient Video Understanding](https://mlsanity.com/paper/2510.07915/) (2025)
- [DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression](https://mlsanity.com/paper/2603.15685/) (2026)
- [IPCV: Information-Preserving Compression for MLLM Visual Encoders](https://mlsanity.com/paper/2512.18747/) (2025)
- [EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistant](https://mlsanity.com/paper/2605.10343/) (2026)
- [video-SALMONN S: Memory-Enhanced Streaming Audio-Visual LLM](https://mlsanity.com/paper/2510.11129/) (2025)
- [SVAgent: Storyline-Guided Long Video Understanding via Cross-Modal Multi-Agent Collaboration](https://mlsanity.com/paper/2604.05079/) (2026)
- [EarlyTom: Early Token Compression Completes Fast Video Understanding](https://mlsanity.com/paper/2605.30010/) (2026)
- [Watch, Remember, Reason: Human-View Video Understanding with MLLMs](https://mlsanity.com/paper/2606.07433/) (2026)
- [VideoCompressa: Data-Efficient Video Understanding via Joint Temporal Compression and Spatial Reconstruction](https://mlsanity.com/paper/2511.18831/) (2025)
- [OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models](https://mlsanity.com/paper/2608.03812/) (2026)
- [Bridging Visual Representation and Reinforcement Learning from Verifiable Rewards in Large Vision-Language Models](https://mlsanity.com/paper/2603.27375/) (2026)
- [V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Models](https://mlsanity.com/paper/2603.27650/) (2026)
- [Dynamic Token Compression for Efficient Video Understanding through Reinforcement Learning](https://mlsanity.com/paper/2603.26365/) (2026)
- [KiToke: Kernel-based Interval-aware Token Compression for Video Large Language Models](https://mlsanity.com/paper/2604.03414/) (2026)
- [HeadRouter: Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Models](https://mlsanity.com/paper/2604.23717/) (2026)
- [EchoPrune: Interpreting Redundancy as Temporal Echoes for Efficient VideoLLMs](https://mlsanity.com/paper/2605.10050/) (2026)
- [Robust RGB-T Tracking via Learnable Visual Fourier Prompt Fine-tuning and Modality Fusion Prompt Generation](https://mlsanity.com/paper/2509.19733/) (2025)

## Abstract (arXiv)

Video large language models (VideoLLM) excel at video understanding, but face efficiency challenges due to the quadratic complexity of abundant visual tokens. Our systematic analysis of token compression methods for VideoLLMs reveals two critical issues: (i) overlooking distinctive visual signals across frames, leading to information loss; (ii) suffering from implementation constraints, causing incompatibility with modern architectures or efficient operators. To address these challenges, we distill three design principles for VideoLLM token compression and propose a plug-and-play inference acceleration framework "Video Compression Commander" (VidCom2). By quantifying each frame's uniqueness, VidCom2 adaptively adjusts compression intensity across frames, effectively preserving essential information while reducing redundancy in video sequences. Extensive experiments across various VideoLLMs and benchmarks demonstrate the superior performance and efficiency of our VidCom2. With only 25% visual tokens, VidCom2 achieves 99.6% of the original performance on LLaVA-OV while reducing 70.8% of the LLM generation latency. Notably, our Frame Compression Adjustment strategy is compatible with other token compression methods to further improve their performance. Our code is available at https://github.com/xuyang-liu16/VidCom2.