Data to 7 Oct 2026

OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge

arXiv 1906.00067 ↗First posted 31 May 2019cs.CV cs.CLcited by 1,114

Not on the map. Why?

Cited by 576 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Visual Question Answering (VQA) in its ideal form lets us study reasoning in the joint space of vision and language and serves as a proxy for the AI task of scene understanding. However, most VQA benchmarks to date are focused on questions such as simple counting, visual attributes, and object detection that do not require reasoning or knowledge beyond what is in the image. In this paper, we address the task of knowledge-based visual question answering and provide a benchmark, called OK-VQA, where the image content is not sufficient to answer the questions, encouraging methods that rely on external knowledge resources. Our new dataset includes more than 14,000 questions that require external knowledge to answer. We show that the performance of the state-of-the-art VQA models degrades drastically in this new setting. Our analysis shows that our knowledge-based VQA task is diverse, difficult, and large compared to previous knowledge-based VQA datasets. We hope that this dataset enables researchers to open up new avenues for research in this domain. See http://okvqa.allenai.org to download and browse the dataset.

Key facts. Cited by 1,114 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 33; 2022: 70; 2023: 213; 2024: 325; 2025: 294; 2026: 179.

1,114 papers.

Sort:

By area:

2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
2023Improved Baselines with Visual Instruction Tuningcited by 4,362
2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696

38 more

2025Gemma 3 Technical Reportcited by 1,686
2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
2022A Generalist Agentcited by 802
2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
2023CogAgent: A Visual Language Model for GUI Agentscited by 692
2023VILA: On Pre-training for Visual Language Modelscited by 686
2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
2022Visual Programming: Compositional visual reasoning without trainingcited by 507
2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
2023Generative Multimodal Models are In-Context Learnerscited by 446
2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 356
2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
2024What matters when building vision-language models?cited by 321
2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
2023OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documentscited by 316

50 most cited of 1,114 All 1,114 (JSON)