mlsanityData to 7 Oct 2026

Variation-aware Vision Token Dropping for Faster Large Vision-Language Models

arXiv 2509.01552 ↗First posted 1 Sep 2025cs.CVcited by 25

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

Large vision-language models (LVLMs) have demonstrated remarkable capabilities in multimodal understanding tasks. However, the increasing demand for high-resolution image and long-video understanding results in substantial token counts, consequently leading to reduced inference efficiency. Token compression offers a direct solution by reducing the number of tokens to be processed, thereby improving computational efficiency without architectural changes. Through extensive analysis, we identify two critical limitations in existing inner-LLM token compression methods: positional bias and incompatibility with efficient operators, which critically hinder their practical deployment for LVLM acceleration. This paper presents the first approach from a dynamic token variation perspective, revealing that visual token variations within LLMs exhibit task-agnostic properties. We propose Variation-aware Vision Token Dropping (textiti.e., V2Drop), which progressively removes visual tokens with minimal variation during LVLM inference, thereby enhancing computational efficiency. Extensive experiments across multiple models and benchmarks consistently demonstrate that V2Drop maintains 94.0% and 98.6% of the original performance for image and video understanding tasks respectively, while reducing LLM generation latency by 31.5% and 74.2%.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 46 of its 56 references that are in the map. First posted on arXiv 1 Sep 2025.

Builds on

46 of 56 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025A Survey of Token Compression for Efficient Multimodal Large Language Models27 Jul
  2. 2025Shifting AI Efficiency From Model-Centric to Data-Centric Compression25 May
  3. 2025Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?17 Feb
  4. 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More17 Feb
  5. 2025Global Compression Commander: Plug-and-Play Inference Acceleration for High-Resolution Large Vision-Language Models9 Jan
  6. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Models5 Dec
  7. 2024Filter, Correlate, Compress: Training-Free Token Reduction for MLLM Acceleration26 Nov
  8. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction22 Oct
  9. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct
  10. 2024HiRED: Attention-Guided Token Dropping for Efficient Inference of High-Resolution Vision-Language Models20 Aug
  11. 2024TokenPacker: Efficient Visual Projector for Multimodal LLM2 Jul
  12. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models22 Mar
1 more
  1. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 33

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  6. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  7. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  8. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  9. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  10. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  11. 2019Towards VQA Models That Can Readcited by 1,696
  12. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
21 more
  1. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  2. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  3. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  4. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  5. 2022Token Merging: Your ViT But Fastercited by 995
  6. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  7. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  8. 2016A Diagram Is Worth A Dozen Imagescited by 793
  9. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  10. 2025Seed1.5-VL Technical Reportcited by 367
  11. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 303
  12. 2024DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Modelscited by 93
  13. 2023Towards Unified Token Learning for Vision-Language Trackingcited by 62
  14. 2025A Survey on (M)LLM-Based GUI Agentscited by 61
  15. 2025HoliTom: Holistic Token Merging for Fast Video Large Language Modelscited by 58
  16. 2025Video-XL-Pro: Reconstructive Token Compression for Extremely Long Video Understandingcited by 49
  17. 2025Video Compression Commander: Plug-and-Play Inference Acceleration for Video Large Language Modelscited by 38
  18. 2025Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Drivingcited by 15
  19. 2025MMG-Vid: Maximizing Marginal Gains at Segment-level and Token-level for Efficient Video LLMscited by 13
  20. 2024Sparse-Tuning: Adapting Vision Transformers with Efficient Fine-tuning and Inferencecited by 12
  21. 2024M2IST: Multi-Modal Interactive Side-Tuning for Efficient Referring Expression Comprehensioncited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 18.

25 papers.

Sort:

By area:

  1. 2025StreamingTOM: Streaming Token Compression for Efficient Video Understandingcited by 41
  2. 2025Accelerating Streaming Video Large Language Models via Hierarchical Token Compressioncited by 34
  3. 2026OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Modelscited by 23
  4. 2025Efficient Multi-modal Large Language Models via Progressive Consistency Distillationcited by 22
  5. 2025AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMscited by 19
  6. 2025IPCV: Information-Preserving Compression for MLLM Visual Encoderscited by 5
  7. 2026EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistantcited by 5
  8. 2026A More Word-like Image Tokenization for MLLMscited by 3
  9. 2026EarlyTom: Early Token Compression Completes Fast Video Understandingcited by 3
  10. 2026V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Modelscited by 2
  11. 2026LRCP: Low-Rank Compressibility Guided Visual Token Pruning for Efficient LVLMscited by 2
  12. 2026HeadRouter: Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Modelscited by 1
13 more
  1. 2026Evading Visual Aphasia: Contrastive Adaptive Semantic Token Pruning for Vision-Language Modelscited by 1
  2. 2026Spectral Evolution-Guided Token Pruning in Multimodal Large Language Modelscited by 1
  3. 2026Reducing Temporal Redundancy for Efficient Vision-Language-Action Inferencecited by 1
  4. 2026E2S-Pruner: Progressive Two-Stage Evidence Fusion for Visual Token Pruning in Vision-Language Modelscited by 1
  5. 2025Improving MLLM Training Efficiency via Stage-Aware Sparsitycited by 0
  6. 2025ShaRP: SHAllow-LayeR Pruning for Efficient Video Large Language Modelscited by 0
  7. 2026Bridging Visual Representation and Reinforcement Learning from Verifiable Rewards in Large Vision-Language Modelscited by 0
  8. 2026IWP: Token Pruning as Implicit Weight Pruning in Large Vision Language Modelscited by 0
  9. 2026Attention-Free and Lightweight Token Reduction for Efficient Vision-Language Modelscited by 0
  10. 2026Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligencecited by 0
  11. 2026CoViST: Visual Token Compression via Composable Statescited by 0
  12. 2026Still There, No Longer Seen: Exposing Compression-Induced Risk in Large Vision-Language Modelscited by 0
  13. 2026VisionWeave: Weaving Elastic Visual Representations as a Native Capability of MLLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ApET: Approximation-Error Guided Token Compression for Efficient VLMs
  2. 2025AdaTok: Adaptive Token Compression with Object-Aware Representations for Efficient Multimodal LLMs
  3. 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
  4. 2025Hybrid Token Compression for Vision-Language Models
  5. 2024Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters
  6. 2026DUET-VLM: Dual stage Unified Efficient Token reduction for VLM Training and Inference
  7. 2026Vision Token Reduction via Attention-Driven Self-Compression for Efficient Multimodal Large Language Models
  8. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects