mlsanityData to 14 Oct 2026

Fully Authentic Visual Question Answering Dataset from Online Communities

arXiv 2311.15562 ↗First posted 27 Nov 2023cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Visual Question Answering (VQA) entails answering questions about images. We introduce the first VQA dataset in which all contents originate from an authentic use case. Sourced from online question answering community forums, we call it VQAonline. We characterize this dataset and how it relates to eight mainstream VQA datasets. Observing that answers in our dataset tend to be much longer (i.e., a mean of 173 words) and so incompatible with standard VQA evaluation metrics, we instead utilize popular metrics for longer text evaluation for evaluating six state-of-the-art VQA models on VQAonline and report where they struggle most. Finally, we analyze which evaluation metrics align best with human judgments. To facilitate future extensions, we publicly-share the dataset at: https://vqaonline.github.io/.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 62 of its 93 references that are in the map. First posted on arXiv 27 Nov 2023.

Builds on

62 of 93 references.

References · 62

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  8. 2023Visual Instruction Tuningcited by 7,869
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  10. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
38 more
  1. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  2. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  3. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  5. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  6. 2015VQA: Visual Question Answeringcited by 2,506
  7. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  8. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  9. 2019Towards VQA Models That Can Readcited by 1,718
  10. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  11. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  12. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  13. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  14. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  15. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  16. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  17. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  18. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  19. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  20. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  21. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  22. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  23. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  24. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  25. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  26. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  27. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
  28. 2018TVQA: Localized, Compositional Video Question Answeringcited by 467
  29. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 445
  30. 2021InfographicVQAcited by 437
  31. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 252
  32. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  33. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
  34. 2020Open-Domain Question Answering Goes Conversational via Question Rewritingcited by 148
  35. 2019Taskmaster-1: Toward a Realistic and Diverse Dialog Datasetcited by 140
  36. 2016Visual Question Answering: A Survey of Methods and Datasetscited by 123
  37. 2023Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entitiescited by 113
  38. 2023A Critical Evaluation of Evaluations for Long-form Question Answeringcited by 111

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 3; 2026: 2.

9 papers.

  1. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 75
  2. 2024Long-Form Answers to Visual Questions from Blind and Low Vision Peoplecited by 18
  3. 2023Gemini Pro Defeated by GPT-4V: Evidence from Educationcited by 12
  4. 2025Towards Understanding the Use of MLLM-Enabled Applications for Visual Interpretation by Blind and Low Vision Peoplecited by 11
  5. 2025Acknowledging Focus Ambiguity in Visual Questionscited by 8
  6. 2025MORQA: Benchmarking Evaluation Metrics for Medical Open-Ended Question Answeringcited by 8
  7. 2023An Evaluation of GPT-4V and Gemini in Online VQAcited by 5
  8. 2026How Multimodal Large Language Models Support Access to Visual Information: A Diary Study With Blind and Low Vision Peoplecited by 3
  9. 2026Diagnosing Failure Modes of Shared-State Collaboration in Resource-Constrained Visual Agentscited by 0