mlsanityData to 7 Oct 2026

All You May Need for VQA are Image Captions

arXiv 2205.01883 ↗First posted 4 May 2022cs.CV cs.CLcited by 65

Not on the map. Why?

Cited by 24 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Visual Question Answering (VQA) has benefited from increasingly sophisticated models, but has not enjoyed the same level of engagement in terms of data creation. In this paper, we propose a method that automatically derives VQA examples at volume, by leveraging the abundance of existing image-caption annotations combined with neural models for textual question generation. We show that the resulting data is of high-quality. VQA models trained on our data improve state-of-the-art zero-shot accuracy by double digits and achieve a level of robustness that lacks in the same model trained on human-annotated VQA data.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 66 of its 79 references that are in the map. First posted on arXiv 4 May 2022.

Builds on

66 of 79 references.

References · 66

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  9. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  10. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  11. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  12. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
38 more
  1. 2015VQA: Visual Question Answeringcited by 2,489
  2. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  3. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  4. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  5. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  6. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  7. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  8. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  9. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  10. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  11. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  12. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  13. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  14. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  15. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
  16. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  17. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  18. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  19. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 337
  20. 2021An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQAcited by 298
  21. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 292
  22. 201912-in-1: Multi-Task Vision and Language Representation Learningcited by 278
  23. 2020Just Ask: Learning to Answer Questions from Millions of Narrated Videoscited by 248
  24. 2019UNITER: UNiversal Image-TExt Representation Learningcited by 225
  25. 2020Adversarial Filters of Dataset Biasescited by 159
  26. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
  27. 2015Exploring Models and Data for Image Question Answeringcited by 139
  28. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 131
  29. 2019Cycle-Consistency for Robust Visual Question Answeringcited by 105
  30. 2020Training Question Answering Models From Synthetic Datacited by 95
  31. 2017An Analysis of Visual Question Answering Algorithmscited by 90
  32. 2018Pythia v0.1: the Winning Entry to the VQA Challenge 2018cited by 62
  33. 2016Generating Natural Questions About an Imagecited by 56
  34. 2017Visual Question Generation as Dual Task of Visual Question Answeringcited by 41
  35. 2019Information Maximizing Visual Question Generationcited by 40
  36. 2018Effective Parallel Corpus Mining using Bilingual Sentence Embeddingscited by 39
  37. 2020WeaQA: Weak Supervision via Captions for Visual Question Answeringcited by 25
  38. 2020Clue: Cross-modal Coherence Modeling for Caption Generationcited by 24

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 26; 2024: 15; 2025: 11; 2026: 7.

65 papers.

Sort:

By area:

  1. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  2. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  3. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 416
  4. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  5. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
  6. 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 159
  7. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scalecited by 125
  8. 2022From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Modelscited by 114
  9. 2023Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entitiescited by 112
  10. 2023What You See is What You Read? Improving Text-Image Alignment Evaluationcited by 105
  11. 2023Encyclopedic VQA: Visual questions about detailed properties of fine-grained categoriescited by 105
  12. 2022Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Trainingcited by 88
38 more
  1. 2023Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Modelscited by 80
  2. 2023Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Imagescited by 79
  3. 2024BRAVE: Broadening the visual encoding of vision-language modelscited by 68
  4. 2024Towards A Better Metric for Text-to-Video Generationcited by 50
  5. 2024MedCoT: Medical Chain of Thought via Hierarchical Expertcited by 46
  6. 2023If at First You Don't Succeed, Try, Try Again: Faithful Diffusion-based Text-to-Image Generation by Selectioncited by 40
  7. 2024From Seconds to Hours: Reviewing MultiModal Large Language Models on Comprehensive Long Video Understandingcited by 29
  8. 2023Connecting Vision and Language with Video Localized Narrativescited by 24
  9. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
  10. 2023Unified Language Representation for Question Answering over Text, Tables, and Imagescited by 22
  11. 2024R-LLaVA: Improving Med-VQA Understanding through Visual Region of Interestcited by 21
  12. 2023Q: How to Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images!cited by 19
  13. 2023Improving Zero-shot Visual Question Answering via Large Language Models with Reasoning Question Promptscited by 19
  14. 2025DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Imagescited by 17
  15. 2024Enhancing Visual Question Answering through Ranking-Based Hybrid Training and Multimodal Fusioncited by 16
  16. 2023Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Modelscited by 14
  17. 2024Mining Fine-Grained Image-Text Alignment for Zero-Shot Captioning via Text-Only Trainingcited by 12
  18. 2023Exploring Question Decomposition for Zero-Shot VQAcited by 11
  19. 2023Pre-Training Multi-Modal Dense Retrievers for Outside-Knowledge Visual Question Answeringcited by 10
  20. 2023Multi-Sentence Grounding for Long-term Instructional Videocited by 10
  21. 2022MaXM: Towards Multilingual Visual Question Answeringcited by 9
  22. 2024CIC: A Framework for Culturally-Aware Image Captioningcited by 8
  23. 2022Modal-specific Pseudo Query Generation for Video Corpus Moment Retrievalcited by 6
  24. 2023EVJVQA Challenge: Multilingual Visual Question Answeringcited by 6
  25. 2023Architext: Language-Driven Generative Architecture Designcited by 5
  26. 2024Web-Scale Visual Entity Recognition: An LLM-Driven Data Approachcited by 5
  27. 2023MAQA: A Multimodal QA Benchmark for Negationcited by 4
  28. 2023AQUALLM: Audio Question Answering Data Generation Using Large Language Modelscited by 4
  29. 2023Enhancing Vision-Language Pre-Training with Jointly Learned Questioner and Dense Captionercited by 3
  30. 2024An Entailment Tree Generation Approach for Multimodal Multi-Hop Question Answering with Mixture-of-Experts and Iterative Feedback Mechanismcited by 3
  31. 2025Multimodal Language Models as Text-to-Image Model Evaluatorscited by 3
  32. 2025FuseLIP: Multimodal Embeddings via Early Fusion of Discrete Tokenscited by 3
  33. 2025Chart Question Answering from Real-World Analytical Narrativescited by 3
  34. 2024Language-guided Detection and Mitigation of Unknown Dataset Biascited by 2
  35. 2025SHAPE : Self-Improved Visual Preference Alignment by Iteratively Generating Holistic Winnercited by 2
  36. 2023CARTIER: Cartographic lAnguage Reasoning Targeted at Instruction Execution for Robotscited by 1
  37. 2025Capturing Visualization Design Rationalecited by 1
  38. 2025Adapting Vision-Language Models for Evaluating World Modelscited by 1

50 most cited of 65 All 65 (JSON)