mlsanityData to 14 Oct 2026

Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data

arXiv 2102.01226 ↗First posted 1 Feb 2021cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

In spite of much recent research in the area, it is still unclear whether subject-area question-answering data is useful for machine reading comprehension (MRC) tasks. In this paper, we investigate this question. We collect a large-scale multi-subject multiple-choice question-answering dataset, ExamQA, and use incomplete and noisy snippets returned by a web search engine as the relevant context for each question-answering instance to convert it into a weakly-labeled MRC instance. We then propose a self-teaching paradigm to better use the generated weakly-labeled MRC instances to improve a target MRC task. Experimental results show that we can obtain +5.1% in accuracy on a multiple-choice MRC dataset, C^3, and +3.8% in exact match on an extractive MRC dataset, CMRC 2018 over state-of-the-art MRC baselines, demonstrating the effectiveness of our framework and the usefulness of large-scale subject-area question-answering data for different types of machine reading comprehension tasks.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 47 of its 67 references that are in the map. First posted on arXiv 1 Feb 2021.

Builds on

47 of 67 references.

References · 47

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  4. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  5. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  6. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  7. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  8. 2015Teaching Machines to Read and Comprehendcited by 1,049
  9. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 861
  10. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 670
  11. 2013Do Deep Nets Really Need to be Deep?cited by 474
  12. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 285
35 more
  1. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 278
  2. 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 266
  3. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 231
  4. 2018QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehensioncited by 164
  5. 2019DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehensioncited by 156
  6. 2019Revisiting Self-Training for Neural Sequence Generationcited by 155
  7. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
  8. 2019JEC-QA: A Legal-Domain Question Answering Datasetcited by 116
  9. 2019HEAD-QA: A Healthcare Dataset for Complex Reasoningcited by 101
  10. 2020Self-training Improves Pre-training for Natural Language Understandingcited by 96
  11. 2018A Span-Extraction Dataset for Chinese Machine Reading Comprehensioncited by 84
  12. 2019Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehensioncited by 81
  13. 2020Logic-Guided Data Augmentation and Regularization for Consistent Question Answeringcited by 78
  14. 2020End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systemscited by 65
  15. 2019Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answeringcited by 63
  16. 2020EXAMS: A Multi-Subject High School Examinations Dataset for Cross-Lingual and Multilingual Question Answeringcited by 60
  17. 2019Model Compression with Two-stage Multi-teacher Knowledge Distillation for Web Question Answering Systemcited by 43
  18. 2017Machine Comprehension by Text-to-Text Neural Question Generationcited by 32
  19. 2020ERNIE-Doc: A Retrospective Long-Document Modeling Transformercited by 31
  20. 2019Align, Mask and Select: A Simple Method for Incorporating Commonsense Knowledge into Language Representation Modelscited by 28
  21. 2017Semi-Supervised QA with Generative Domain-Adaptive Netscited by 27
  22. 2017Supervised and Unsupervised Transfer Learning for Question Answeringcited by 24
  23. 2018Simple and Effective Semi-Supervised Question Answeringcited by 22
  24. 2020AMBERT: A Pre-trained Language Model with Multi-Grained Tokenizationcited by 22
  25. 2018Improving Machine Reading Comprehension with General Reading Strategiescited by 20
  26. 2019GeoSQA: A Benchmark for Scenario-based Question Answering in the Geography Domain at High School Levelcited by 17
  27. 2019Improving Question Answering with External Knowledgecited by 16
  28. 2019Unsupervised Domain Adaptation of Language Models for Reading Comprehensioncited by 15
  29. 2018Yuanfudao at SemEval-2018 Task 11: Three-way Attention and Relational Knowledge for Commonsense Machine Comprehensioncited by 12
  30. 2019Go From the General to the Particular: Multi-Domain Translation with Domain Transformation Networkscited by 12
  31. 2020Combining Self-Training and Self-Supervised Learning for Unsupervised Disfluency Detectioncited by 10
  32. 2019Learning to Ask Unanswerable Questions for Machine Reading Comprehensioncited by 9
  33. 2020Improving Machine Reading Comprehension with Contextualized Commonsense Knowledgecited by 6
  34. 2020Improved Synthetic Training for Reading Comprehensioncited by 4
  35. 2019Teach an all-rounder with experts in different domainscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 0; 2025: 3.

6 papers.

  1. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domainscited by 156
  2. 2023TrueTeacher: Learning Factual Consistency Evaluation with Large Language Modelscited by 83
  3. 2025One Token to Fool LLM-as-a-Judgecited by 60
  4. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 30
  5. 2023MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuningcited by 16
  6. 2023Friend-training: Learning from Models of Different but Related Taskscited by 0