mlsanityData to 14 Oct 2026

Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More

arXiv 2502.11494 ↗First posted 17 Feb 2025cs.CL cs.CVcited by 155

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

Vision tokens in multimodal large language models often dominate huge computational overhead due to their excessive length compared to linguistic modality. Abundant recent methods aim to solve this problem with token pruning, which first defines an importance criterion for tokens and then prunes the unimportant vision tokens during inference. However, in this paper, we show that the importance is not an ideal indicator to decide whether a token should be pruned. Surprisingly, it usually results in inferior performance than random token pruning and leading to incompatibility to efficient attention computation operators.Instead, we propose DART (Duplication-Aware Reduction of Tokens), which prunes tokens based on its duplication with other tokens, leading to significant and training-free acceleration. Concretely, DART selects a small subset of pivot tokens and then retains the tokens with low duplication to the pivots, ensuring minimal information loss during token pruning. Experiments demonstrate that DART can prune 88.9% vision tokens while maintaining comparable performance, leading to a 1.99× and 2.99× speed-up in total time and prefilling stage, respectively, with good compatibility to efficient attention operators. Our codes are available at https://github.com/ZichenWen1/DART.

Key facts. Cited by 155 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 63 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

32 of 63 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Feather the Throttle: Revisiting Visual Token Pruning for Vision-Language Model Acceleration17 Dec
  2. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction22 Oct
  3. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct
  4. 2024HiRED: Attention-Guided Token Dropping for Efficient Inference of High-Resolution Vision-Language Models20 Aug

Other references · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  7. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  8. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  9. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  10. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  11. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  12. 2019Towards VQA Models That Can Readcited by 1,718
16 more
  1. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  2. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  3. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  4. 2022Token Merging: Your ViT But Fastercited by 1,021
  5. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
  6. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  7. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  8. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 422
  9. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  10. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 363
  11. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 183
  12. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  13. 2024MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformercited by 54
  14. 2021Vision Transformers with Patch Diversificationcited by 45
  15. 2023Mitigating Over-smoothing in Transformers via Regularized Nonlocal Functionalscited by 32
  16. 2024Long-Form Speech Generation with Spoken Language Modelscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 50; 2026: 105.

155 papers.

Sort:

By area:

  1. 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 97
  2. 2025Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMscited by 96
  3. 2025EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Modelscited by 93
  4. 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 68
  5. 2025Don't Just Chase "Highlighted Tokens" in MLLMs: Revisiting Visual Holistic Context Retentioncited by 47
  6. 2025VScan: Rethinking Visual Token Reduction for Efficient Large Vision-Language Modelscited by 41
  7. 2025Visual Representation Alignment for Multimodal Large Language Modelscited by 41
  8. 2025Video Compression Commander: Plug-and-Play Inference Acceleration for Video Large Language Modelscited by 38
  9. 2025Accelerating Streaming Video Large Language Models via Hierarchical Token Compressioncited by 37
  10. 2025Shifting AI Efficiency From Model-Centric to Data-Centric Compressioncited by 29
  11. 2025Variation-aware Vision Token Dropping for Faster Large Vision-Language Modelscited by 27
  12. 2025Efficient Multi-modal Large Language Models via Progressive Consistency Distillationcited by 22
38 more
  1. 2025AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMscited by 22
  2. 2025FastDriveVLA: Efficient End-to-End Driving via Plug-and-Play Reconstruction-based Token Pruningcited by 20
  3. 2025FlexSelect: Flexible Token Selection for Efficient Long Video Understandingcited by 18
  4. 2025CATP: Contextually Adaptive Token Pruning for Efficient and Enhanced Multimodal In-Context Learningcited by 18
  5. 2025Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methodscited by 17
  6. 2025Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Drivingcited by 16
  7. 2025Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Priorcited by 15
  8. 2025Visual Multi-Agent System: Mitigating Hallucination Snowballing via Visual Flowcited by 13
  9. 2025Can Visual Input Be Compressed? A Visual Token Compression Benchmark for Large Multimodal Modelscited by 13
  10. 2025VideoScan: Enabling Efficient Streaming Video Understanding via Frame-level Semantic Carrierscited by 12
  11. 2025When Token Pruning is Worse than Random: Understanding Visual Token Information in VLLMscited by 12
  12. 2026ApET: Approximation-Error Guided Token Compression for Efficient VLMscited by 12
  13. 2025Why 1 + 1 < 1 in Visual Token Pruning: Beyond Naive Integration via Multi-Objective Balanced Coveringcited by 11
  14. 2025VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMscited by 11
  15. 2026The Model Knows Which Tokens Matter: Automatic Token Selection via Noise Gatingcited by 11
  16. 2026Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Evictioncited by 11
  17. 2026ViCA: Efficient Multimodal LLMs with Vision-Only Cross-Attentioncited by 10
  18. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospectscited by 9
  19. 2026StreamMeCo: Long-Term Agent Memory Compression for Efficient Streaming Video Understandingcited by 9
  20. 2026LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Modelscited by 9
  21. 2025QuickVideo: Real-Time Long Video Understanding with System Algorithm Co-Designcited by 8
  22. 2025SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Modelscited by 8
  23. 2025Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuningcited by 8
  24. 2026Innovator-VL: A Multimodal Large Language Model for Scientific Discoverycited by 8
  25. 2025ToDRE: Effective Visual Token Pruning via Token Diversity and Task Relevancecited by 7
  26. 2025LLMC+: Benchmarking Vision-Language Model Compression with a Plug-and-play Toolkitcited by 7
  27. 2025D2Pruner: Debiased Importance and Structural Diversity for MLLM Token Pruningcited by 7
  28. 2025VLM-Pruner: Buffering for Spatial Sparsity in an Efficient VLM Centrifugal Token Pruning Paradigmcited by 6
  29. 2026AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compressioncited by 6
  30. 2026Unified Spatiotemporal Token Compression for Video-LLMs at Ultra-Low Retentioncited by 6
  31. 2026EgoSAT: A Comprehensive Benchmark of Egocentric Streaming Interaction Understandingcited by 6
  32. 2025IPCV: Information-Preserving Compression for MLLM Visual Encoderscited by 5
  33. 2026IDPruner: Harmonizing Importance and Diversity in Visual Token Pruning for MLLMscited by 5
  34. 2026Token Reduction via Local and Global Contexts Optimization for Efficient Video Large Language Modelscited by 5
  35. 2026EvoPrune: Early-Stage Visual Token Pruning for Efficient MLLMscited by 5
  36. 2026EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labelingcited by 5
  37. 2026EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistantcited by 5
  38. 2026Improving Visual Token Reduction via Rectifying Distortions for Efficient Multimodal LLM Inferencecited by 5

50 most cited of 155 All 155 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025TrimTokenator: Towards Adaptive Visual Token Pruning for Large Multimodal Models
  2. 2025TransPrune: Token Transition Pruning for Efficient Large Vision-Language Model
  3. 2025Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?
  4. 2026How Many Visual Tokens Do Multimodal Language Models Need? Scaling Visual Token Pruning with F^3A
  5. 2026Capturing Token Tendencies for Training-Free Token Pruning in Multimodal Large Language Models
  6. 2025TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model
  7. 2025STAR: Stage-Wise Attention-Guided Token Reduction for Efficient Large Vision-Language Models Inference
  8. 2025VFlowOpt: A Token Pruning Framework for LMMs with Visual Information Flow-Guided Optimization