mlsanityData to 7 Oct 2026

The Quest for Visual Understanding: A Journey Through the Evolution of Visual Question Answering

arXiv 2501.07109 ↗First posted 13 Jan 2025cs.CVcited by 3

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Visual Question Answering (VQA) is an interdisciplinary field that bridges the gap between computer vision (CV) and natural language processing(NLP), enabling Artificial Intelligence(AI) systems to answer questions about images. Since its inception in 2015, VQA has rapidly evolved, driven by advances in deep learning, attention mechanisms, and transformer-based models. This survey traces the journey of VQA from its early days, through major breakthroughs, such as attention mechanisms, compositional reasoning, and the rise of vision-language pre-training methods. We highlight key models, datasets, and techniques that shaped the development of VQA systems, emphasizing the pivotal role of transformer architectures and multimodal pre-training in driving recent progress. Additionally, we explore specialized applications of VQA in domains like healthcare and discuss ongoing challenges, such as dataset bias, model interpretability, and the need for common-sense reasoning. Lastly, we discuss the emerging trends in large multimodal language models and the integration of external knowledge, offering insights into the future directions of VQA. This paper aims to provide a comprehensive overview of the evolution of VQA, highlighting both its current state and potential advancements.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 50 of its 81 references that are in the map. First posted on arXiv 13 Jan 2025.

Builds on

50 of 81 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  8. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  9. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  10. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  11. 2015VQA: Visual Question Answeringcited by 2,489
  12. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
38 more
  1. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  2. 2019Towards VQA Models That Can Readcited by 1,696
  3. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  4. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  5. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  6. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  7. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  8. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  9. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  10. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
  11. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  12. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  13. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  14. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  15. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 457
  16. 2015Neural Module Networkscited by 375
  17. 2019Deep Modular Co-Attention Networks for Visual Question Answeringcited by 338
  18. 2019PlotQA: Reasoning over Scientific Plotscited by 317
  19. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 314
  20. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
  21. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
  22. 2018Scalable and accurate deep learning for electronic health recordscited by 275
  23. 2015Stacked Attention Networks for Image Question Answeringcited by 273
  24. 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
  25. 2016Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioningcited by 231
  26. 2018Compositional Attention Networks for Machine Reasoningcited by 183
  27. 2017MUTAN: Multimodal Tucker Fusion for Visual Question Answeringcited by 137
  28. 2016Dual Attention Networks for Multimodal Reasoning and Matchingcited by 113
  29. 2021Multi-modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Trainingcited by 96
  30. 2017An Analysis of Visual Question Answering Algorithmscited by 90
  31. 2015Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answeringcited by 80
  32. 2021MMBERT: Multimodal BERT Pretraining for Improved Medical VQAcited by 68
  33. 2024Sports-QA: A Large-Scale Video Question Answering Benchmark for Complex and Professional Sportscited by 41
  34. 2020MMFT-BERT: Multimodal Fusion Transformer with BERT Encodings for Visual Question Answeringcited by 30
  35. 2017Spatial Memory for Context Reasoning in Object Detectioncited by 22
  36. 2023VaQuitA: Enhancing Alignment in LLM-Assisted Video Understandingcited by 17
  37. 2016Hierarchical Memory Networkscited by 15
  38. 2021Towards Developing a Multilingual and Code-Mixed Visual Question Answering System by Knowledge Distillationcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025VQArt-Bench: A semantically rich VQA Benchmark for Art and Cultural Heritagecited by 7
  2. 2026CogniVerse: Revolutionizing Multi-Modal Retrieval-Augmented Generation with Cognitive Reflection and Geometric Reasoningcited by 3
  3. 2025MV-CoRe: Multimodal Visual-Conceptual Reasoning for Complex Visual Question Answeringcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Visual question answering: from early developments to recent advances -- a survey
  2. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities
  3. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey
  4. 2025Exploring Advanced Techniques for Visual Question Answering: A Comprehensive Comparison
  5. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithms
  6. 2024Guiding Vision-Language Model Selection for Visual Question-Answering Across Tasks, Domains, and Knowledge Types
  7. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge
  8. 2023Language Guided Visual Question Answering: Elevate Your Multimodal Language Model Using Knowledge-Enriched Prompts