Vision Language Models: A Survey of 26K Papers
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
We present a transparent, reproducible measurement of research trends across 26,104 accepted papers from CVPR, ICLR, and NeurIPS spanning 2023-2025. Titles and abstracts are normalized, phrase-protected, and matched against a hand-crafted lexicon to assign up to 35 topical labels and mine fine-grained cues about tasks, architectures, training regimes, objectives, datasets, and co-mentioned modalities. The analysis quantifies three macro shifts: (1) a sharp rise of multimodal vision-language-LLM work, which increasingly reframes classic perception as instruction following and multi-step reasoning; (2) steady expansion of generative methods, with diffusion research consolidating around controllability, distillation, and speed; and (3) resilient 3D and video activity, with composition moving from NeRFs to Gaussian splatting and a growing emphasis on human- and agent-centric understanding. Within VLMs, parameter-efficient adaptation like prompting/adapters/LoRA and lightweight vision-language bridges dominate; training practice shifts from building encoders from scratch to instruction tuning and finetuning strong backbones; contrastive objectives recede relative to cross-entropy/ranking and distillation. Cross-venue comparisons show CVPR has a stronger 3D footprint and ICLR the highest VLM share, while reliability themes such as efficiency or robustness diffuse across areas. We release the lexicon and methodology to enable auditing and extension. Limitations include lexicon recall and abstract-only scope, but the longitudinal signals are consistent across venues and years.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 59 of its 79 references that are in the map. First posted on arXiv 10 Oct 2025.
Builds on
59 of 79 references.
Closest ancestors · 4
Same direction, newest first
- 2023Kosmos-2: Grounding Multimodal Large Language Models to the World26 Jun
- 2022Flamingo: a Visual Language Model for Few-Shot Learning29 Apr
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation28 Jan
- 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervision24 Aug
Other references · 55
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2023Segment Anythingcited by 8,158
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
38 more
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
- 2022A ConvNet for the 2020scited by 3,545
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
- 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
- 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
- 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
- 2025DINOv3cited by 1,297
- 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
- 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
- 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
- 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
- 2021Grounded Language-Image Pre-trainingcited by 1,147
- 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
- 2016Modeling Context in Referring Expressionscited by 1,054
- 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
- 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
- 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
- 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
- 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
- 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
- 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
- 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
50 most cited of 55 All 55 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions
- 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Models
- 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models
- 2026Language-Instructed Vision Embeddings for Controllable and Generalizable Perception
- 2026Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines
- 2025Hidden in plain sight: VLMs overlook their visual representations
- 2024An Introduction to Vision-Language Modeling
- 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges