# Cheap and Good? Simple and Effective Data Augmentation for Low Resource Machine Reading

arXiv: https://arxiv.org/abs/2106.04134 · HTML: https://mlsanity.com/paper/2106.04134/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 17 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Bidirectional Attention Flow for Machine Comprehension](https://mlsanity.com/paper/1611.01603/) (2016)
- [SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine](https://mlsanity.com/paper/1704.05179/) (2017)
- [MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension](https://mlsanity.com/paper/1910.09753/) (2019)
- [Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering](https://mlsanity.com/paper/1908.08167/) (2019)
- [MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension](https://mlsanity.com/paper/1905.13453/) (2019)
- [PolicyQA: A Reading Comprehension Dataset for Privacy Policies](https://mlsanity.com/paper/2010.02557/) (2020)
- [The TechQA Dataset](https://mlsanity.com/paper/1911.02984/) (2019)
- [Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering](https://mlsanity.com/paper/2005.01218/) (2020)
- [An Exploration of Data Augmentation and Sampling Techniques for Domain-Agnostic Question Answering](https://mlsanity.com/paper/1912.02145/) (2019)
- [Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space](https://mlsanity.com/paper/2010.01475/) (2020)
- [Two-Stage Synthesis Networks for Transfer Learning in Machine Comprehension](https://mlsanity.com/paper/1706.09789/) (2017)
- [Multi-Mention Learning for Reading Comprehension with Neural Cascades](https://mlsanity.com/paper/1711.00894/) (2017)

## Built on by (most cited first, 7 of 7)

- [Text Data Augmentation for Large Language Models: A Comprehensive Survey of Methods, Challenges, and Opportunities](https://mlsanity.com/paper/2501.18845/) (2025)
- [Retrieval Enhanced Data Augmentation for Question Answering on Privacy Policies](https://mlsanity.com/paper/2204.08952/) (2022)
- [How May I Help You? Using Neural Text Simplification to Improve Downstream NLP Tasks](https://mlsanity.com/paper/2109.04604/) (2021)
- [Long-Tailed Question Answering in an Open World](https://mlsanity.com/paper/2305.06557/) (2023)
- [Combining Data Generation and Active Learning for Low-Resource Question Answering](https://mlsanity.com/paper/2211.14880/) (2022)
- [Mitigating Data Scarcity for Large Language Models](https://mlsanity.com/paper/2302.01806/) (2023)
- [Expert-guided Clinical Text Augmentation via Query-Based Model Collaboration](https://mlsanity.com/paper/2509.21530/) (2025)

## Abstract (arXiv)

We propose a simple and effective strategy for data augmentation for low-resource machine reading comprehension (MRC). Our approach first pretrains the answer extraction components of a MRC system on the augmented data that contains approximate context of the correct answers, before training it on the exact answer spans. The approximate context helps the QA method components in narrowing the location of the answers. We demonstrate that our simple strategy substantially improves both document retrieval and answer extraction performance by providing larger context of the answers and additional training data. In particular, our method significantly improves the performance of BERT based retriever (15.12%), and answer extractor (4.33% F1) on TechQA, a complex, low-resource MRC task. Further, our data augmentation strategy yields significant improvements of up to 3.9% exact match (EM) and 2.7% F1 for answer extraction on PolicyQA, another practical but moderate sized QA dataset that also contains long answer spans.