mlsanityData to 7 Oct 2026

FVQA: Fact-based Visual Question Answering

arXiv 1606.05433 ↗First posted 17 Jun 2016cs.CVcited by 181

Not on the map. Why?

Cited by 31 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Visual Question Answering (VQA) has attracted a lot of attention in both Computer Vision and Natural Language Processing communities, not least because it offers insight into the relationships between two important sources of information. Current datasets, and the models built upon them, have focused on questions which are answerable by direct analysis of the question and image alone. The set of such questions that require no external information to answer is interesting, but very limited. It excludes questions which require common sense, or basic factual knowledge to answer, for example. Here we introduce FVQA, a VQA dataset which requires, and supports, much deeper reasoning. FVQA only contains questions which require external information to answer. We thus extend a conventional visual question answering dataset, which contains image-question-answerg triplets, through additional image-question-answer-supporting fact tuples. The supporting fact is represented as a structural triplet, such as <Cat,CapableOf,ClimbingTrees>. We evaluate several baseline models on the FVQA dataset, and describe a novel model which is capable of reasoning about an image on the basis of supporting facts.

Key facts. Cited by 181 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 38; 2023: 35; 2024: 37; 2025: 22; 2026: 20.

181 papers.

Sort:

By area:

  1. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  2. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  3. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  4. 2021An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQAcited by 298
  5. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 203
  6. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  7. 2021KAT: A Knowledge Augmented Transformer for Vision-and-Languagecited by 125
  8. 2022REVEAL: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memorycited by 122
  9. 2022CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoningcited by 119
  10. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  11. 2023Encyclopedic VQA: Visual questions about detailed properties of fine-grained categoriescited by 105
  12. 2023AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learncited by 97
38 more
  1. 2021Medical Visual Question Answering: A Surveycited by 93
  2. 2022Retrieval Augmented Visual Question Answering with Outside Knowledgecited by 93
  3. 2021Multi-Modal Answer Validation for Knowledge-Based VQAcited by 78
  4. 2022Multi-Modal Knowledge Graph Construction and Application: A Surveycited by 74
  5. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  6. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  7. 2022REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answeringcited by 58
  8. 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspectivecited by 56
  9. 2023Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answeringcited by 51
  10. 2021Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answeringcited by 50
  11. 2023See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoningcited by 49
  12. 2021Towards General Purpose Vision Systemscited by 46
  13. 2024A Survey on Evaluation of Multimodal Large Language Modelscited by 46
  14. 20213D Question Answeringcited by 41
  15. 2022Text2Pos: Text-to-Point-Cloud Cross-Modal Localizationcited by 40
  16. 2023Benchmarks for Automated Commonsense Reasoning: A Surveycited by 40
  17. 2024Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspectivecited by 39
  18. 2022The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoningcited by 38
  19. 2021Zero-shot Visual Question Answering using Knowledge Graphcited by 37
  20. 2022MuKEA: Multimodal Knowledge Extraction and Accumulation for Knowledge-based Visual Question Answeringcited by 36
  21. 2023CoTDet: Affordance Knowledge Prompting for Task Driven Object Detectioncited by 36
  22. 2026Vision-DeepResearch: Incentivizing DeepResearch Capability in Multimodal Large Language Modelscited by 33
  23. 2022A Review on Methods and Applications in Multimodal Deep Learningcited by 29
  24. 2023Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answeringcited by 28
  25. 2021Recent Advances and Trends in Multimodal Deep Learning: A Reviewcited by 26
  26. 2025Modality Curation: Building Universal Embeddings for Advanced Multimodal Information Retrievalcited by 26
  27. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 26
  28. 2021Image Captioning for Effective Use of Language Models in Knowledge-Based Visual Question Answeringcited by 25
  29. 2021Coarse-to-Fine Reasoning for Visual Question Answeringcited by 25
  30. 2025Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearchcited by 25
  31. 2021Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledgecited by 24
  32. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
  33. 2021Zero-shot and Few-shot Learning with Knowledge Graphs: A Comprehensive Surveycited by 23
  34. 2022Modern Question Answering Datasets and Benchmarks: A Surveycited by 23
  35. 2023End-to-end Knowledge Retrieval with Multi-modal Queriescited by 23
  36. 2023Robust Visual Question Answering: Datasets, Methods, and Future Challengescited by 23
  37. 2023UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Modelscited by 23
  38. 2023A Comprehensive Evaluation of GPT-4V on Knowledge-Intensive Visual Question Answeringcited by 23

50 most cited of 181 All 181 (JSON)