mlsanityData to 14 Oct 2026

Video Compression Commander: Plug-and-Play Inference Acceleration for Video Large Language Models

arXiv 2505.14454 ↗First posted 20 May 2025cs.CVcited by 38

On the map: Efficient LLM inference and serving → Visual token pruning in multimodal models

Abstract (arXiv)

Video large language models (VideoLLM) excel at video understanding, but face efficiency challenges due to the quadratic complexity of abundant visual tokens. Our systematic analysis of token compression methods for VideoLLMs reveals two critical issues: (i) overlooking distinctive visual signals across frames, leading to information loss; (ii) suffering from implementation constraints, causing incompatibility with modern architectures or efficient operators. To address these challenges, we distill three design principles for VideoLLM token compression and propose a plug-and-play inference acceleration framework "Video Compression Commander" (VidCom2). By quantifying each frame's uniqueness, VidCom2 adaptively adjusts compression intensity across frames, effectively preserving essential information while reducing redundancy in video sequences. Extensive experiments across various VideoLLMs and benchmarks demonstrate the superior performance and efficiency of our VidCom2. With only 25% visual tokens, VidCom2 achieves 99.6% of the original performance on LLaVA-OV while reducing 70.8% of the LLM generation latency. Notably, our Frame Compression Adjustment strategy is compatible with other token compression methods to further improve their performance. Our code is available at https://github.com/xuyang-liu16/VidCom2.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving. Builds on 16 of its 27 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

16 of 27 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?17 Feb
  2. 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More17 Feb

Other references · 14

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  6. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  7. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  8. 2022Token Merging: Your ViT But Fastercited by 1,021
  9. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  10. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  11. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 391
  12. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 309
2 more
  1. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
  2. 2025MDP3: A Training-free Approach for List-wise Frame Selection in Video-LLMscited by 43

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 19; 2026: 19.

38 papers.

Sort:

By area:

  1. 2025GUI-G2: Gaussian Reward Modeling for GUI Groundingcited by 63
  2. 2025StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understandingcited by 58
  3. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 50
  4. 2025StreamingTOM: Streaming Token Compression for Efficient Video Understandingcited by 43
  5. 2025LongCodeZip: Compress Long Context for Code Language Modelscited by 39
  6. 2025Accelerating Streaming Video Large Language Models via Hierarchical Token Compressioncited by 37
  7. 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Groundingcited by 29
  8. 2025Variation-aware Vision Token Dropping for Faster Large Vision-Language Modelscited by 27
  9. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 25
  10. 2026OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Modelscited by 25
  11. 2025OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Modelscited by 22
  12. 2025Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Drivingcited by 16
26 more
  1. 2025UniComp: Rethinking Video Compression Through Informational Uniquenesscited by 7
  2. 2025MARC: Memory-Augmented RL Token Compression for Efficient Video Understandingcited by 6
  3. 2026DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compressioncited by 6
  4. 2025IPCV: Information-Preserving Compression for MLLM Visual Encoderscited by 5
  5. 2026EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistantcited by 5
  6. 2025video-SALMONN S: Memory-Enhanced Streaming Audio-Visual LLMcited by 4
  7. 2026SVAgent: Storyline-Guided Long Video Understanding via Cross-Modal Multi-Agent Collaborationcited by 4
  8. 2026EarlyTom: Early Token Compression Completes Fast Video Understandingcited by 4
  9. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 4
  10. 2025VideoCompressa: Data-Efficient Video Understanding via Joint Temporal Compression and Spatial Reconstructioncited by 3
  11. 2026OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Modelscited by 3
  12. 2026Bridging Visual Representation and Reinforcement Learning from Verifiable Rewards in Large Vision-Language Modelscited by 2
  13. 2026V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Modelscited by 2
  14. 2026Dynamic Token Compression for Efficient Video Understanding through Reinforcement Learningcited by 1
  15. 2026KiToke: Kernel-based Interval-aware Token Compression for Video Large Language Modelscited by 1
  16. 2026HeadRouter: Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Modelscited by 1
  17. 2026EchoPrune: Interpreting Redundancy as Temporal Echoes for Efficient VideoLLMscited by 1
  18. 2025Robust RGB-T Tracking via Learnable Visual Fourier Prompt Fine-tuning and Modality Fusion Prompt Generationcited by 0
  19. 2025Sharp Eyes and Memory for VideoLLMs: Information-Aware Visual Token Pruning for Efficient and Reliable VideoLLM Reasoningcited by 0
  20. 2025ShaRP: SHAllow-LayeR Pruning for Efficient Video Large Language Modelscited by 0
  21. 2026Tango: Taming Visual Signals for Efficient Video Large Language Modelscited by 0
  22. 2026video-SALMONN-R3: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understandingcited by 0
  23. 2026MVPruner: Dynamic Token Pruning for Accelerating Multi-view Vision-Language Models in Autonomous Drivingcited by 0
  24. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0
  25. 2026Unified Agentic Video Editing Across Levels of Complexity and Creativitycited by 0
  26. 2026GleanVID: Complementary Token Selection for Efficient Video Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models
  2. 2026Rethinking Video Token Compression with a Global Codebook: Learning Once, Compressing Everywhere
  3. 2026TTF: Temporal Token Fusion for Efficient Video-Language Model
  4. 2026Video Codec-Guided Training-Free Token Compression for Video Large Language Models
  5. 2026One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding
  6. 2026OTT-Vid: Optimal Transport Temporal Token Compression for Video Large Language Models
  7. 2024VidCompress: Memory-Enhanced Temporal Compression for Video Understanding in Large Language Models
  8. 2026Rethinking Visual Token Compression for Video Large Language Models: A Simple Yet Strong Baseline