mlsanityData to 7 Oct 2026

More Than Reading Comprehension: A Survey on Datasets and Metrics of Textual Question Answering

arXiv 2109.12264 ↗First posted 25 Sep 2021cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Textual Question Answering (QA) aims to provide precise answers to user's questions in natural language using unstructured data. One of the most popular approaches to this goal is machine reading comprehension(MRC). In recent years, many novel datasets and evaluation metrics based on classical MRC tasks have been proposed for broader textual QA tasks. In this paper, we survey 47 recent textual QA benchmark datasets and propose a new taxonomy from an application point of view. In addition, We summarize 8 evaluation metrics of textual QA tasks. Finally, we discuss current trends in constructing textual QA benchmarks and suggest directions for future work.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 55 of its 81 references that are in the map. First posted on arXiv 25 Sep 2021.

Builds on

55 of 81 references.

References · 55

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  11. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  12. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
38 more
  1. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  2. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  3. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  4. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  5. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  6. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  7. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  8. 2015Teaching Machines to Read and Comprehendcited by 1,039
  9. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  10. 2017Dense-Captioning Events in Videoscited by 789
  11. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  12. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  13. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  14. 2018CoQA: A Conversational Question Answering Challengecited by 674
  15. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  16. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  17. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  18. 2019ELI5: Long Form Question Answeringcited by 567
  19. 2018QuAC : Question Answering in Contextcited by 379
  20. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  21. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  22. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  23. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
  24. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 281
  25. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 263
  26. 2016Key-Value Memory Networks for Directly Reading Documentscited by 223
  27. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
  28. 2018ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehensioncited by 198
  29. 2015A Dataset for Movie Descriptioncited by 195
  30. 2016Movie Descriptioncited by 180
  31. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  32. 2017DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applicationscited by 110
  33. 2015The Goldilocks Principle: Reading Children's Books with Explicit Memory Representationscited by 108
  34. 2019Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answeringcited by 96
  35. 2018Interpretation of Natural Language Rules in Conversational Machine Readingcited by 72
  36. 2019Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Indexcited by 64
  37. 2019TWEETQA: A Social Media Focused Question Answering Datasetcited by 55
  38. 2017Quasar: Datasets for Question Answering by Search and Readingcited by 50

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 1; 2024: 3; 2025: 1.

8 papers.

  1. 2024Large Language Models: A Surveycited by 506
  2. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 50
  3. 2024ClimaQA: An Automated Evaluation Framework for Climate Question Answering Modelscited by 21
  4. 2022A Survey on Measuring and Mitigating Reasoning Shortcuts in Machine Reading Comprehensioncited by 4
  5. 2023Chatbots as Problem Solvers: Playing Twenty Questions with Role Reversalscited by 3
  6. 2022A Comprehensive Survey on Multi-hop Machine Reading Comprehension Datasets and Metricscited by 1
  7. 2022A Comprehensive Survey on Multi-hop Machine Reading Comprehension Approachescited by 1
  8. 2024Towards Robust Evaluation: A Comprehensive Taxonomy of Datasets and Metrics for Open Domain Question Answering in the Era of Large Language Modelscited by 1