mlsanityData to 14 Oct 2026

EffiVLM-BENCH: A Comprehensive Benchmark for Evaluating Training-Free Acceleration in Large Vision-Language Models

arXiv 2506.00479 ↗First posted 31 May 2025cs.CL cs.CV cs.LGcited by 3

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

Large Vision-Language Models (LVLMs) have achieved remarkable success, yet their significant computational demands hinder practical deployment. While efforts to improve LVLM efficiency are growing, existing methods lack comprehensive evaluation across diverse backbones, benchmarks, and metrics. In this work, we systematically evaluate mainstream acceleration techniques for LVLMs, categorized into token and parameter compression. We introduce EffiVLM-Bench, a unified framework for assessing not only absolute performance but also generalization and loyalty, while exploring Pareto-optimal trade-offs. Our extensive experiments and in-depth analyses offer insights into optimal strategies for accelerating LVLMs. We open-source code and recipes for EffiVLM-Bench to foster future research.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 54 references that are in the map. First posted on arXiv 31 May 2025.

Builds on

34 of 54 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Full Static Quantization1 Feb
  2. 2024Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Context Sparsification1 Dec
  3. 2024VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration29 Oct
  4. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct
  5. 2024LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference26 Jun
  6. 2022EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning14 Oct

Other references · 28

  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  5. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  7. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  8. 2019Towards VQA Models That Can Readcited by 1,718
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  10. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  11. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  12. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
16 more
  1. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  2. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  3. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  4. 2022Token Merging: Your ViT But Fastercited by 1,021
  5. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  6. 2016A Diagram Is Worth A Dozen Imagescited by 804
  7. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  8. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 474
  9. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
  10. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 229
  11. 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 197
  12. 2022Hierarchical multimodal transformers for Multi-Page DocVQAcited by 105
  13. 2024LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillationcited by 45
  14. 2021Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compressioncited by 38
  15. 2021Distilled Dual-Encoder Model for Vision-Language Understandingcited by 22
  16. 2024CFSP: An Efficient Structured Pruning Framework for LLMs with Coarse-to-Fine Activation Informationcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 25
  2. 2025VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMscited by 11
  3. 2026Rethinking Visual Token Compression for Video Large Language Models: A Simple Yet Strong Baselinecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023ReForm-Eval: Evaluating Large Vision Language Models via Unified Re-Formulation of Task-Oriented Benchmarks
  2. 2025LLMC+: Benchmarking Vision-Language Model Compression with a Plug-and-play Toolkit
  3. 2025LVLM-Compress-Bench: Benchmarking the Broader Impact of Large Vision-Language Model Compression
  4. 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
  5. 2026Hierarchical Compression of Vision-Language Model Benchmarks
  6. 2026Empirical Recipes for Efficient and Compact Vision-Language Models
  7. 2025Resampling Benchmark for Efficient Comprehensive Evaluation of Large Vision-Language Models
  8. 2025Short-LVLM: Compressing and Accelerating Large Vision-Language Models by Pruning Redundant Layers