mlsanityData to 14 Oct 2026

Small Vision-Language Models: A Survey on Compact Architectures and Techniques

arXiv 2503.10665 ↗First posted 9 Mar 2025cs.CV cs.AI cs.CLcited by 4

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

also in 1 more
Abstract (arXiv)

The emergence of small vision-language models (sVLMs) marks a critical advancement in multimodal AI, enabling efficient processing of visual and textual data in resource-constrained environments. This survey offers a comprehensive exploration of sVLM development, presenting a taxonomy of architectures - transformer-based, mamba-based, and hybrid - that highlight innovations in compact design and computational efficiency. Techniques such as knowledge distillation, lightweight attention mechanisms, and modality pre-fusion are discussed as enablers of high performance with reduced resource requirements. Through an in-depth analysis of models like TinyGPT-V, MiniGPT-4, and VL-Mamba, we identify trade-offs between accuracy, efficiency, and scalability. Persistent challenges, including data biases and generalization to complex tasks, are critically examined, with proposed pathways for addressing them. By consolidating advancements in sVLMs, this work underscores their transformative potential for accessible AI, setting a foundation for future research into efficient multimodal systems.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 56 of its 93 references that are in the map. First posted on arXiv 9 Mar 2025.

Builds on

56 of 93 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024To Preserve or To Compress: An In-Depth Study of Connector Selection in Multimodal Large Language Models9 Oct
  2. 2024Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference21 Mar
  3. 2023TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones28 Dec
  4. 2023Improved Baselines with Visual Instruction Tuning5 Oct

Other references · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  9. 2022Generative Adversarial Networkscited by 5,627
  10. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  11. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  12. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
38 more
  1. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  2. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  3. 2015VQA: Visual Question Answeringcited by 2,506
  4. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  5. 2020GLU Variants Improve Transformercited by 1,804
  6. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  7. 2019Towards VQA Models That Can Readcited by 1,718
  8. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  9. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  10. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  11. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  12. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  13. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 649
  14. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 576
  15. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 490
  16. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 365
  17. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 360
  18. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 339
  19. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  20. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 267
  21. 2024LocalMamba: Visual State Space Model with Windowed Selective Scancited by 236
  22. 2023Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Textcited by 208
  23. 2024Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modelingcited by 174
  24. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 166
  25. 2024Zamba: A Compact 7B SSM Hybrid Modelcited by 124
  26. 2018Land-Cover Classification with High-Resolution Remote Sensing Images Using Transferable Deep Modelscited by 119
  27. 2021VLM: Task-agnostic Video-Language Model Pre-training for Video Understandingcited by 102
  28. 2023Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Modelscited by 91
  29. 2021VSR: A Unified Framework for Document Layout Analysis combining Vision, Semantics and Relationscited by 40
  30. 20242DMamba: Efficient State Space Model for Image Representation with Applications on Giga-Pixel Whole Slide Image Classificationcited by 35
  31. 2023Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understandingcited by 28
  32. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 25
  33. 2023CIFAR-10-Warehouse: Broad and More Realistic Testbeds in Model Generalization Analysiscited by 12
  34. 2024Efficient Multi-modal Large Language Models via Visual Token Groupingcited by 9
  35. 2024MaskMamba: A Hybrid Mamba-Transformer Model for Masked Image Generationcited by 8
  36. 2021Making EfficientNet More Efficient: Exploring Batch-Independent Normalization, Group Convolutions and Reduced Resolution Trainingcited by 6
  37. 2023Benchmarking a Benchmark: How Reliable is MS-COCO?cited by 6
  38. 2024ST-Mamba: Spatial-Temporal Mamba for Traffic Flow Estimation Recovery using Limited Datacited by 4

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Frame Sampling Strategies Matter: A Benchmark for small vision language modelscited by 6
  2. 2025From the Laboratory to Real-World Application: Evaluating Zero-Shot Scene Interpretation on Edge Devices for Mobile Roboticscited by 1
  3. 2025Keeping it Local, Tiny and Real: Automated Report Generation on Edge Computing Devices for Mechatronic-Based Cognitive Systemscited by 1
  4. 2026OpenGlass: A Sensing-Computing Split Architecture for Local MLLM-Driven Real-Time Visual Assistancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks
  2. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions
  3. 2025A Survey on Efficient Vision-Language Models
  4. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Models
  5. 2026Empirical Recipes for Efficient and Compact Vision-Language Models
  6. 2025Benchmarking Large and Small MLLMs
  7. 2025Mordal: Automated Pretrained Model Selection for Vision Language Models
  8. 2024An Introduction to Vision-Language Modeling