mlsanityData to 7 Oct 2026

MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension

arXiv 1910.09753 ↗First posted 22 Oct 2019cs.CLcited by 183

Not on the map. Why?

Abstract (arXiv)

We present the results of the Machine Reading for Question Answering (MRQA) 2019 shared task on evaluating the generalization capabilities of reading comprehension systems. In this task, we adapted and unified 18 distinct question answering datasets into the same format. Among them, six datasets were made available for training, six datasets were made available for development, and the final six were hidden for final evaluation. Ten teams submitted systems, which explored various ideas including data sampling, multi-task learning, adversarial training and ensembling. The best system achieved an average F1 score of 72.5 on the 12 held-out datasets, 10.7 absolute points higher than our initial baseline based on BERT.

Key facts. Cited by 183 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 43; 2022: 53; 2023: 36; 2024: 23; 2025: 18; 2026: 10.

183 papers.

Sort:

By area:

  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  2. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
  3. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  4. 2022Prompting GPT-3 To Be Reliablecited by 258
  5. 2021SPoT: Better Frozen Model Adaptation through Soft Prompt Transfercited by 235
  6. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 209
  7. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 180
  8. 2022SCROLLS: Standardized CompaRison Over Long Language Sequencescited by 170
  9. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 146
  10. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  11. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 142
  12. 2023Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learningcited by 115
38 more
  1. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 108
  2. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  3. 2023Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluationscited by 96
  4. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  5. 2022ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Promptscited by 85
  6. 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"cited by 85
  7. 2021FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Taskscited by 84
  8. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 77
  9. 2021Few-Shot Question Answering by Pretraining Span Selectioncited by 62
  10. 2021Comparing Test Sets with Item Response Theorycited by 55
  11. 2021Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarkingcited by 53
  12. 2021Zero-Shot Dialogue State Tracking via Cross-Task Transfercited by 52
  13. 2023Resolving Knowledge Conflicts in Large Language Modelscited by 49
  14. 2021Knowing More About Questions Can Help: Improving Calibration in Question Answeringcited by 47
  15. 2022Read before Generate! Faithful Long Form Question Answering with Machine Readingcited by 47
  16. 2023Universal Information Extraction as Unified Semantic Matchingcited by 47
  17. 2021Can NLI Models Verify QA Systems' Predictions?cited by 46
  18. 2023Understanding Retrieval Augmentation for Long-Form Question Answeringcited by 45
  19. 2024500xCompressor: Generalized Prompt Compression for Large Language Modelscited by 45
  20. 2021TR-BERT: Dynamic Token Reduction for Accelerating BERT Inferencecited by 40
  21. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  22. 2024Thermometer: Towards Universal Calibration for Large Language Modelscited by 38
  23. 2023DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuningcited by 34
  24. 2021Zero-shot Generalization in Dialog State Tracking through Generative Question Answeringcited by 33
  25. 2021Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistantscited by 31
  26. 2021On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Studycited by 30
  27. 2021Challenges in Generalization in Open Domain Question Answeringcited by 30
  28. 2021Learning to Generate Task-Specific Adapters from Task Descriptioncited by 28
  29. 2021Retrieval-guided Counterfactual Generation for QAcited by 28
  30. 2021MixQG: Neural Question Generation with Mixed Answer Typescited by 28
  31. 2023Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answeringcited by 28
  32. 2023ReEval: Automatic Hallucination Evaluation for Retrieval-Augmented Large Language Models via Transferable Adversarial Attackscited by 28
  33. 2024Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documentscited by 27
  34. 2021FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text modelscited by 26
  35. 2024FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decompositioncited by 26
  36. 2021Reliability Testing for Natural Language Processing Systemscited by 25
  37. 2022Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustnesscited by 23
  38. 2022Exploring The Landscape of Distributional Robustness for Question Answering Modelscited by 23

50 most cited of 183 All 183 (JSON)