mlsanityData to 14 Oct 2026

Can Visual Input Be Compressed? A Visual Token Compression Benchmark for Large Multimodal Models

arXiv 2511.02650 ↗First posted 4 Nov 2025cs.CVcited by 13

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

Large multimodal models (LMMs) often suffer from severe inference inefficiency due to the large number of visual tokens introduced by image encoders. While recent token compression methods, such as pruning and merging, have shown promise in reducing redundancy, their evaluation remains fragmented and inconsistent. In this work, we present UniPruneBench, a unified and extensible benchmark for visual token pruning in multimodal LLMs. UniPruneBench provides standardized protocols across six ability dimensions and ten datasets, covering ten representative compression algorithms and three families of LMMs (LLaVA-v1.5, Intern-VL3, and Qwen2.5-VL). Beyond task accuracy, it incorporates system-level metrics such as runtime and prefilling latency to provide a holistic view. Our experiments uncover several key findings: (1) random pruning is a surprisingly strong baseline, (2) no single method consistently outperforms others across scenarios, (3) pruning sensitivity varies significantly across tasks, with OCR being most vulnerable, and (4) pruning ratio is the dominant factor governing performance degradation. We believe UniPruneBench will serve as a reliable foundation for future research on efficient multimodal modeling.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 44 of its 45 references that are in the map. First posted on arXiv 4 Nov 2025.

Builds on

44 of 45 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models4 Mar
  2. 2025Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?17 Feb
  3. 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More17 Feb
  4. 2025What Kind of Visual Tokens Do We Need? Training-free Visual Token Pruning for Multi-modal Large Language Models from the Perspective of Graph4 Jan
  5. 2024Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model16 Nov
  6. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction22 Oct
  7. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct
  8. 2024Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models16 Sep
  9. 2024Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference9 May
  10. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models22 Mar
  11. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2024Qwen2.5 Technical Reportcited by 4,942
  8. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  9. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  10. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  11. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  12. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
21 more
  1. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  3. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  4. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  5. 2022Token Merging: Your ViT But Fastercited by 1,021
  6. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  7. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  8. 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
  9. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  10. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  11. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 229
  12. 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehensioncited by 146
  13. 2024The Revolution of Multimodal Large Language Models: A Surveycited by 137
  14. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72
  15. 2024MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMscited by 62
  16. 2025PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selectioncited by 61
  17. 2024LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steeringcited by 51
  18. 2025CoT-Kinetics: A Theoretical Modeling Assessing LRM Reasoning Processcited by 51
  19. 2024A Survey on Evaluation of Multimodal Large Language Modelscited by 46
  20. 2023CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Updatecited by 43
  21. 2024MLLM-CompBench: A Comparative Reasoning Benchmark for Multimodal LLMscited by 32

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

  1. 2026OPD-V: Visual On-Policy Self-Distillation with Modality Balancecited by 8
  2. 2026What Do Visual Tokens Really Encode? Uncovering Sparsity and Redundancy in Multimodal Large Language Modelscited by 7
  3. 2026Why and When Visual Token Pruning Fails? A Study on Relevant Visual Information Shift in MLLMs Decodingcited by 7
  4. 2026ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoningcited by 6
  5. 2026MMKU-Bench: A Multimodal Update Benchmark for Diverse Visual Knowledgecited by 5
  6. 2026Set-Valued Prediction for Large Language Models with Feasibility-Aware Coverage Guaranteescited by 4
  7. 2026EchoRL: Reinforcement Learning via Rollout Echoingcited by 4
  8. 2026Beyond Relevance-Centric Retrieval: Rubric-Oriented Document Set Selection and Rankingcited by 3
  9. 2026AQuaUI: Visual Token Reduction for GUI Agents with Adaptive Quadtreescited by 1
  10. 2026Uncertainty-Aware Abstention in Large Language Models with Provable Alignment Guaranteescited by 1
  11. 2026AdaTutoRank: Learning to Rerank Document Sets via Adaptive Tutoring Optimization for RAG and Deep Researchcited by 1
  12. 2026Bifocal: Training-Free Mixed Granularity Visual Token Pruning for Multimodal LLMscited by 0
1 more
  1. 2026Why Generate a Click? Plug-in Decision Heads for Web Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Token Sequence Compression for Efficient Multimodal Computing
  2. 2025TrimTokenator-LC: Towards Adaptive Visual Token Pruning for Large Multimodal Models with Long Contexts
  3. 2026LRCP: Low-Rank Compressibility Guided Visual Token Pruning for Efficient LVLMs
  4. 2025TrimTokenator: Towards Adaptive Visual Token Pruning for Large Multimodal Models
  5. 2025Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods
  6. 2026How Many Visual Tokens Do Multimodal Language Models Need? Scaling Visual Token Pruning with F^3A
  7. 2025A Survey of Token Compression for Efficient Multimodal Large Language Models
  8. 2025A Glimpse to Compress: Dynamic Visual Token Pruning for Large Vision-Language Models