mlsanityData to 7 Oct 2026

Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?

arXiv 2502.11501 ↗First posted 17 Feb 2025cs.CL cs.CVcited by 76

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

Multimodal large language models (MLLMs) have shown remarkable performance for cross-modal understanding and generation, yet still suffer from severe inference costs. Recently, abundant works have been proposed to solve this problem with token pruning, which identifies the redundant tokens in MLLMs and then prunes them to reduce the computation and KV storage costs, leading to significant acceleration without training. While these methods claim efficiency gains, critical questions about their fundamental design and evaluation remain unanswered: Why do many existing approaches underperform even compared to naive random token selection? Are attention-based scoring sufficient for reliably identifying redundant tokens? Is language information really helpful during token pruning? What makes a good trade-off between token importance and duplication? Are current evaluation protocols comprehensive and unbiased? The ignorance of previous research on these problems hinders the long-term development of token pruning. In this paper, we answer these questions one by one, providing insights into the design of future token pruning methods.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 33 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

18 of 33 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct

Other references · 17

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  6. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  8. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  9. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  10. 2019Towards VQA Models That Can Readcited by 1,696
  11. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  12. 2016Modeling Context in Referring Expressionscited by 1,054
5 more
  1. 2022Token Merging: Your ViT But Fastercited by 995
  2. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  3. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  4. 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotationscited by 144
  5. 2024Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmarkcited by 53

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 33; 2026: 43.

76 papers.

Sort:

By area:

  1. 2025Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMscited by 92
  2. 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 66
  3. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 49
  4. 2025Don't Just Chase "Highlighted Tokens" in MLLMs: Revisiting Visual Holistic Context Retentioncited by 45
  5. 2025Video Compression Commander: Plug-and-Play Inference Acceleration for Video Large Language Modelscited by 38
  6. 2025Shifting AI Efficiency From Model-Centric to Data-Centric Compressioncited by 29
  7. 2025Variation-aware Vision Token Dropping for Faster Large Vision-Language Modelscited by 25
  8. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 22
  9. 2025Efficient Multi-modal Large Language Models via Progressive Consistency Distillationcited by 22
  10. 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruningcited by 19
  11. 2025AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMscited by 19
  12. 2025CATP: Contextually Adaptive Token Pruning for Efficient and Enhanced Multimodal In-Context Learningcited by 18
38 more
  1. 2025Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Drivingcited by 15
  2. 2025Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methodscited by 15
  3. 2025SparseVILA: Decoupling Visual Sparsity for Efficient VLM Inferencecited by 15
  4. 2025Towards General Continuous Memory for Vision-Language Modelscited by 14
  5. 2026N"uwa: Mending the Spatial Integrity Torn by VLM Token Pruningcited by 13
  6. 2025Can Visual Input Be Compressed? A Visual Token Compression Benchmark for Large Multimodal Modelscited by 11
  7. 2026IVC-Prune: Revealing the Implicit Visual Coordinates in LVLMs for Vision Token Pruningcited by 11
  8. 2026The Model Knows Which Tokens Matter: Automatic Token Selection via Noise Gatingcited by 11
  9. 2025VideoNSA: Native Sparse Attention Scales Video Understandingcited by 10
  10. 2026Towards Long-horizon Agentic Multimodal Searchcited by 9
  11. 2026ViCA: Efficient Multimodal LLMs with Vision-Only Cross-Attentioncited by 8
  12. 2026Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Frameworkcited by 8
  13. 2026Innovator-VL: A Multimodal Large Language Model for Scientific Discoverycited by 7
  14. 2025D2Pruner: Debiased Importance and Structural Diversity for MLLM Token Pruningcited by 6
  15. 2025IPCV: Information-Preserving Compression for MLLM Visual Encoderscited by 5
  16. 2026What Do Visual Tokens Really Encode? Uncovering Sparsity and Redundancy in Multimodal Large Language Modelscited by 5
  17. 2026Large Vision-Language Models Get Lost in Attentioncited by 5
  18. 2026EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistantcited by 5
  19. 2025EgoPrune: Efficient Token Pruning for Egomotion Video Reasoning in Embodied Agentcited by 4
  20. 2026Do We Need Distinct Representations for Every Speech Token? Unveiling and Exploiting Redundancy in Large Speech Language Modelscited by 4
  21. 2026EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labelingcited by 4
  22. 2025TransPrune: Token Transition Pruning for Efficient Large Vision-Language Modelcited by 3
  23. 2025ERGO: Efficient High-Resolution Visual Understanding for Vision-Language Modelscited by 3
  24. 2025HIVTP: A Training-Free Method to Improve VLMs Efficiency via Hierarchical Visual Token Pruning Using Middle-Layer-Based Importance Scorecited by 3
  25. 2025The Few Govern the Many:Unveiling Few-Layer Dominance for Time Series Modelscited by 3
  26. 2026Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Groundingcited by 3
  27. 2026G2TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Modelscited by 3
  28. 2026Moving Beyond Diversity: Visual Token Pruning as Subspace Reconstruction for Efficient VLMscited by 3
  29. 2025Does Acceleration Cause Hidden Instability in Vision Language Models? Uncovering Instance-Level Divergence Through a Large-Scale Empirical Studycited by 2
  30. 2025GreedyPrune: Retenting Critical Visual Token Set for Large Vision Language Modelscited by 2
  31. 2025Reading Images Like Texts: Sequential Image Understanding in Vision-Language Modelscited by 2
  32. 2025OmniDocLayout: Towards Diverse Document Layout Generation via Coarse-to-Fine LLM Learningcited by 2
  33. 2025Script: Graph-Structured and Query-Conditioned Semantic Token Pruning for Multimodal Large Language Modelscited by 2
  34. 2026MI-Pruner: Crossmodal Mutual Information-guided Token Pruner for Efficient MLLMscited by 2
  35. 2026LiteFrame: Efficient Vision Encoders Unlock Frame Scaling in Video LLMscited by 2
  36. 2026Locality Matters for Training-Free Audio Token Compression in Audio-Language Modelscited by 2
  37. 2026Accelerating Multimodal Large Language Models with Prior-Corrected Token Reductioncited by 2
  38. 2025SignalLLM: A General-Purpose LLM Agent Framework for Automated Signal Processingcited by 1

50 most cited of 76 All 76 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026TOPS: First-Principles Visual Token Pruning via Constructing Token Optimal Preservation Sets for Efficient MLLM Inference
  2. 2026IDPruner: Harmonizing Importance and Diversity in Visual Token Pruning for MLLMs
  3. 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More
  4. 2026Capturing Token Tendencies for Training-Free Token Pruning in Multimodal Large Language Models
  5. 2024Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models
  6. 2026Stepwise Token Selection for Efficient Multimodal Large Language Models
  7. 2025What Kind of Visual Tokens Do We Need? Training-free Visual Token Pruning for Multi-modal Large Language Models from the Perspective of Graph
  8. 2024Sparsity Meets Similarity: Leveraging Long-Tail Distribution for Dynamic Optimized Token Representation in Multimodal Large Language Models