mlsanityData to 14 Oct 2026

Explicit Knowledge-based Reasoning for Visual Question Answering

arXiv 1511.02570 ↗First posted 9 Nov 2015cs.CV cs.CLcited by 85

Not on the map. Why?

Abstract (arXiv)

We describe a method for visual question answering which is capable of reasoning about contents of an image on the basis of information extracted from a large-scale knowledge base. The method not only answers natural language questions using concepts not contained in the image, but can provide an explanation of the reasoning by which it developed its answer. The method is capable of answering far more complex questions than the predominant long short-term memory-based approach, and outperforms it significantly in the testing. We also provide a dataset and a protocol by which to evaluate such methods, thus addressing one of the key issues in general visual ques- tion answering.

Key facts. Cited by 85 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 15; 2022: 19; 2023: 19; 2024: 17; 2025: 12; 2026: 3.

85 papers.

Sort:

By area:

  1. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  2. 2021An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQAcited by 301
  3. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
  4. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 170
  5. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  6. 2021KAT: A Knowledge Augmented Transformer for Vision-and-Languagecited by 126
  7. 2022REVEAL: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memorycited by 123
  8. 2022A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodalcited by 106
  9. 2021Medical Visual Question Answering: A Surveycited by 93
  10. 2024A Survey on Benchmarks of Multimodal Large Language Modelscited by 82
  11. 2021Multi-Modal Answer Validation for Knowledge-Based VQAcited by 79
  12. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 76
38 more
  1. 2022Multi-Modal Knowledge Graph Construction and Application: A Surveycited by 74
  2. 2022REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answeringcited by 59
  3. 2023See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoningcited by 51
  4. 2021Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answeringcited by 50
  5. 2023Benchmarks for Automated Commonsense Reasoning: A Surveycited by 40
  6. 2022The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoningcited by 38
  7. 2021Zero-shot Visual Question Answering using Knowledge Graphcited by 37
  8. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  9. 2022MuKEA: Multimodal Knowledge Extraction and Accumulation for Knowledge-based Visual Question Answeringcited by 36
  10. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 28
  11. 2021DISCOS: Bridging the Gap between Discourse Knowledge and Commonsense Knowledgecited by 26
  12. 2021Image Captioning for Effective Use of Language Models in Knowledge-Based Visual Question Answeringcited by 25
  13. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 25
  14. 2021Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledgecited by 24
  15. 2022Modern Question Answering Datasets and Benchmarks: A Surveycited by 23
  16. 2023A Comprehensive Evaluation of GPT-4V on Knowledge-Intensive Visual Question Answeringcited by 23
  17. 2023HRVQA: A Visual Question Answering Benchmark for High-Resolution Aerial Imagescited by 21
  18. 2021Cross-Modal Retrieval Augmentation for Multi-Modal Classificationcited by 20
  19. 2023Improving Zero-shot Visual Question Answering via Large Language Models with Reasoning Question Promptscited by 20
  20. 2022Hypergraph Transformer: Weakly-supervised Multi-hop Reasoning for Knowledge-based Visual Question Answeringcited by 15
  21. 2022VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challengescited by 14
  22. 2023A Simple Baseline for Knowledge-Based Visual Question Answeringcited by 14
  23. 2024A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysiscited by 14
  24. 2022A Thousand Words Are Worth More Than a Picture: Natural Language-Centric Outside-Knowledge Visual Question Answeringcited by 13
  25. 2022A survey on knowledge-enhanced multimodal learningcited by 11
  26. 2025Video SimpleQA: Towards Factuality Evaluation in Large Video Language Modelscited by 11
  27. 2025Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questionscited by 11
  28. 2023Combo of Thinking and Observing for Outside-Knowledge VQAcited by 9
  29. 2024GeReA: Question-Aware Prompt Captions for Knowledge-based Visual Question Answeringcited by 9
  30. 2021Learning Visual Models using a Knowledge Graph as a Trainercited by 8
  31. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithmscited by 8
  32. 2021Transferring Domain-Agnostic Knowledge in Video Question Answeringcited by 7
  33. 2022Improving Commonsense in Vision-Language Models via Knowledge Graph Riddlescited by 7
  34. 2023VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoningcited by 7
  35. 2023KAFA: Rethinking Image Ad Understanding with Knowledge-Augmented Feature Adaptation of Vision-Language Modelscited by 7
  36. 2021How to find a good image-text embedding for remote sensing visual question answering?cited by 6
  37. 2023Logical Implications for Visual Question Answering Consistencycited by 6
  38. 2025VaseVQA: Multimodal Agent and Benchmark for Ancient Greek Potterycited by 6

50 most cited of 85 All 85 (JSON)