# Improved Synthetic Training for Reading Comprehension

arXiv: https://arxiv.org/abs/2010.12776 · HTML: https://mlsanity.com/paper/2010.12776/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2020.

## Built on by (most cited first, 4 of 4)

- [Synthetic Question Value Estimation for Domain Adaptation of Question Answering](https://mlsanity.com/paper/2203.08926/) (2022)
- [Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data](https://mlsanity.com/paper/2102.01226/) (2021)
- [Not to Overfit or Underfit the Source Domains? An Empirical Study of Domain Generalization in Question Answering](https://mlsanity.com/paper/2205.07257/) (2022)
- [GAAMA 2.0: An Integrated System that Answers Boolean and Extractive Questions](https://mlsanity.com/paper/2206.08441/) (2022)

## Abstract (arXiv)

Automatically generated synthetic training examples have been shown to improve performance in machine reading comprehension (MRC). Compared to human annotated gold standard data, synthetic training data has unique properties, such as high availability at the possible expense of quality. In view of such differences, in this paper, we explore novel applications of synthetic examples to MRC. Our proposed pre-training and knowledge distillation strategies show significant improvements over existing methods. In a particularly surprising discovery, we observe that synthetic distillation often yields students that can outperform the teacher model.