mlsanityData to 14 Oct 2026

A Survey on Efficient Vision-Language Models

arXiv 2504.09724 ↗First posted 13 Apr 2025cs.CVcited by 19

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-language models (VLMs) integrate visual and textual information, enabling a wide range of applications such as image captioning and visual question answering, making them crucial for modern AI systems. However, their high computational demands pose challenges for real-time applications. This has led to a growing focus on developing efficient vision language models. In this survey, we review key techniques for optimizing VLMs on edge and resource-constrained devices. We also explore compact VLM architectures, frameworks and provide detailed insights into the performance-memory trade-offs of efficient VLMs. Furthermore, we establish a GitHub repository at https://github.com/MPSCUMBC/Efficient-Vision-Language-Models-A-Survey to compile all surveyed papers, which we will actively update. Our objective is to foster deeper research in this area.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 121 of its 199 references that are in the map. First posted on arXiv 13 Apr 2025.

Builds on

121 of 199 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025NanoVLMs: How small can we go and still make coherent Vision Language Models?11 Feb
  2. 2025Advancing the Understanding and Evaluation of AR-Generated Scenes: When Vision-Language Models Shine and Stumble21 Jan
  3. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?29 Mar
  4. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions20 Feb
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  6. 2023Vision-Language Models for Vision Tasks: A Survey3 Apr
  7. 2022A Survey of Vision-Language Pre-Trained Models18 Feb

Other references · 114

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  8. 2017On Calibration of Modern Neural Networkscited by 4,960
  9. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  12. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
38 more
  1. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  2. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  3. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  4. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  5. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  7. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  8. 2019Towards VQA Models That Can Readcited by 1,718
  9. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  10. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  11. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  12. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  13. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  14. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  15. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  16. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  17. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  18. 2016A Diagram Is Worth A Dozen Imagescited by 804
  19. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 661
  20. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 597
  21. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  22. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  23. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 382
  24. 2023Fine-Tuning Language Models with Just Forward Passescited by 345
  25. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  26. 2019Counterfactual Visual Explanationscited by 313
  27. 2023Visual-Language Prompt Tuning with Knowledge-guided Context Optimizationcited by 308
  28. 2021Post-Training Quantization for Vision Transformercited by 278
  29. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
  30. 2016Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioningcited by 232
  31. 2018Textual Explanations for Self-Driving Vehiclescited by 226
  32. 2024Multi-Modal Hallucination Control by Visual Information Groundingcited by 208
  33. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
  34. 2023Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuningcited by 170
  35. 2023Physically Grounded Vision-Language Models for Robotic Manipulationcited by 165
  36. 2024TinyLLaVA: A Framework of Small-scale Large Multimodal Modelscited by 164
  37. 2024SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationcited by 160
  38. 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 155

50 most cited of 114 All 114 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 11.

19 papers.

Sort:

By area:

  1. 2025V-ReasonBench: Toward Unified Reasoning Benchmark Suite for Video Generation Modelscited by 22
  2. 2025Frame Sampling Strategies Matter: A Benchmark for small vision language modelscited by 6
  3. 2025ECVL-ROUTER: Scenario-Aware Routing for Vision-Language Modelscited by 4
  4. 2025Large Language Models for Crash Detection in Video: A Survey of Methods, Datasets, and Challengescited by 3
  5. 2025SineProject: Machine Unlearning for Stable Vision Language Alignmentcited by 3
  6. 2026Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Groundingcited by 3
  7. 2025Are Neuro-Inspired Multi-Modal Vision-Language Models Resilient to Membership Inference Privacy Leakage?cited by 2
  8. 2026Lingua-SafetyBench: A Benchmark for Safety Evaluation of Multilingual Vision-Language Modelscited by 2
  9. 2025VT-LVLM-AR: A Video-Temporal Large Vision-Language Model Adapter for Fine-Grained Action Recognition in Long-Term Videoscited by 1
  10. 2026Multimodal Benchmark for Safety Assessment in Industrial Inspection Scenarioscited by 1
  11. 2026Environment-Aware Adaptive Pruning with Interleaved Inference Orchestration for Vision-Language-Action Modelscited by 1
  12. 2025When Better Teachers Don't Make Better Students: Revisiting Knowledge Distillation for CLIP Models in VQAcited by 0
7 more
  1. 2026Integration of Object Detection and Small VLMs for Construction Safety Hazard Identificationcited by 0
  2. 2026Chain of Evidence: Pixel-Level Visual Attribution for Iterative Retrieval-Augmented Generationcited by 0
  3. 2026SafeLens: Deliberate and Efficient Video Guardrails with Fast-and-Slow Screeningcited by 0
  4. 2026MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Modelscited by 0
  5. 2026Assessing VLM Reliability for Medical Image Quality Evaluation Under Corruption and Biascited by 0
  6. 2026EG-ARSA: An Expert-Grounded Open Model for Visual Road Safety Auditing in Low-Resource Settingscited by 0
  7. 2026Exploring In-Context Learning for Handwritten Text Recognitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Vision-Language Models for Edge Networks: A Comprehensive Survey
  2. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges
  3. 2026Empirical Recipes for Efficient and Compact Vision-Language Models
  4. 2024An Introduction to Vision-Language Modeling
  5. 2025Small Vision-Language Models: A Survey on Compact Architectures and Techniques
  6. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Models
  7. 2025Response Wide Shut? Surprising Observations in Basic Vision Language Model Capabilities
  8. 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies