mlsanityData to 7 Oct 2026

Modern Question Answering Datasets and Benchmarks: A Survey

arXiv 2206.15030 ↗First posted 30 Jun 2022cs.CLcited by 23

Not on the map. Why?

Abstract (arXiv)

Question Answering (QA) is one of the most important natural language processing (NLP) tasks. It aims using NLP technologies to generate a corresponding answer to a given question based on the massive unstructured corpus. With the development of deep learning, more and more challenging QA datasets are being proposed, and lots of new methods for solving them are also emerging. In this paper, we investigate influential QA datasets that have been released in the era of deep learning. Specifically, we begin with introducing two of the most common QA tasks - textual question answer and visual question answering - separately, covering the most representative datasets, and then give some current challenges of QA research.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 59 of its 77 references that are in the map. First posted on arXiv 30 Jun 2022.

Builds on

59 of 77 references.

References · 59

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  8. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  9. 2015VQA: Visual Question Answeringcited by 2,489
  10. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  12. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
38 more
  1. 2019Towards VQA Models That Can Readcited by 1,696
  2. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  3. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  4. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  5. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  6. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  7. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  8. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  9. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  10. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  11. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  12. 2018CoQA: A Conversational Question Answering Challengecited by 674
  13. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  14. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  15. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  16. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
  17. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  18. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  19. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
  20. 2018QuAC : Question Answering in Contextcited by 379
  21. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 331
  22. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  23. 2015Visual7W: Grounded Question Answering in Imagescited by 315
  24. 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 309
  25. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
  26. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 211
  27. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 199
  28. 2017Constructing Datasets for Multi-hop Reading Comprehension Across Documentscited by 190
  29. 2016FVQA: Fact-based Visual Question Answeringcited by 181
  30. 2018TallyQA: Answering Complex Counting Questionscited by 152
  31. 2015Large-scale Simple Question Answering with Memory Networkscited by 127
  32. 2021WebQA: Multihop and Multimodal QAcited by 109
  33. 2015The Goldilocks Principle: Reading Children's Books with Explicit Memory Representationscited by 108
  34. 2017An Analysis of Visual Question Answering Algorithmscited by 90
  35. 2015Explicit Knowledge-based Reasoning for Visual Question Answeringcited by 85
  36. 2018RecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipescited by 74
  37. 2017DeepStory: Video Story QA by Deep Embedded Memory Networkscited by 67
  38. 2019TWEETQA: A Social Media Focused Question Answering Datasetcited by 55

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 8; 2024: 6; 2025: 7.

23 papers.

Sort:

By area:

  1. 2023Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMscited by 103
  2. 2023On the Possibilities of AI-Generated Text Detectioncited by 88
  3. 2025RAG vs. GraphRAG: A Systematic Evaluation and Key Insightscited by 51
  4. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 50
  5. 2023Empower Large Language Model to Perform Better on Industrial Domain-Specific Question Answeringcited by 41
  6. 2023In ChatGPT We Trust? Measuring and Characterizing the Reliability of ChatGPTcited by 35
  7. 2023Benchmarking LLM powered Chatbots: Methods and Metricscited by 17
  8. 2025YESciEval: Robust LLM-as-a-Judge for Scientific Question Answeringcited by 15
  9. 2022A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialogcited by 14
  10. 2023Joyful: Joint Modality Fusion and Graph Contrastive Learning for Multimodal Emotion Recognitioncited by 14
  11. 2024RIPPLECOT: Amplifying Ripple Effect of Knowledge Editing in Language Models via Chain-of-Thought In-Context Learningcited by 5
  12. 2025ImageChain: Advancing Sequential Image-to-Text Reasoning in Multimodal Large Language Modelscited by 4
11 more
  1. 2022Can Language Representation Models Think in Bets?cited by 3
  2. 2023Chatbots as Problem Solvers: Playing Twenty Questions with Role Reversalscited by 3
  3. 2024Generative AI Toolkit -- a framework for increasing the quality of LLM-based applications over their whole life cyclecited by 2
  4. 2023FusionMind -- Improving question and answering with external context fusioncited by 1
  5. 2024Towards Robust Evaluation: A Comprehensive Taxonomy of Datasets and Metrics for Open Domain Question Answering in the Era of Large Language Modelscited by 1
  6. 2024LEGAL-UQA: A Low-Resource Urdu-English Dataset for Legal Question Answeringcited by 1
  7. 2025Meet Your New Client: Writing Reports for AI -- Benchmarking Information Loss in Market Research Deliverablescited by 1
  8. 2025Natural Context Drift Undermines the Natural Language Understanding of Large Language Modelscited by 1
  9. 2024CFMatch: Aligning Automated Answer Equivalence Evaluation with Expert Judgments For Open-Domain Question Answeringcited by 0
  10. 2024Deceiving Question-Answering Models: A Hybrid Word-Level Adversarial Approachcited by 0
  11. 2025Comparative Analysis of 47 Context-Based Question Answer Models Across 8 Diverse Datasetscited by 0