# Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering

arXiv: https://arxiv.org/abs/1904.06652 · HTML: https://mlsanity.com/paper/1904.06652/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2019.

## Built on by (most cited first, 21 of 21)

- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them](https://mlsanity.com/paper/2102.07033/) (2021)
- [MDPFuzz: Testing Models Solving Markov Decision Processes](https://mlsanity.com/paper/2112.02807/) (2021)
- [Prompt-Learning for Short Text Classification](https://mlsanity.com/paper/2202.11345/) (2022)
- [Studying Catastrophic Forgetting in Neural Ranking Models](https://mlsanity.com/paper/2101.06984/) (2021)
- [Revisiting Contextual Toxicity Detection in Conversations](https://mlsanity.com/paper/2111.12447/) (2021)
- [Improved Text Classification via Test-Time Augmentation](https://mlsanity.com/paper/2206.13607/) (2022)
- [Interactive Question Answering Systems: Literature Review](https://mlsanity.com/paper/2209.01621/) (2022)
- [Robust Training for Conversational Question Answering Models with Reinforced Reformulation Generation](https://mlsanity.com/paper/2310.13505/) (2023)
- [Exploration of Masked and Causal Language Modelling for Text Generation](https://mlsanity.com/paper/2405.12630/) (2024)
- [Cyberbullying Classifiers are Sensitive to Model-Agnostic Perturbations](https://mlsanity.com/paper/2201.06384/) (2022)
- [Data Augmentation for Biomedical Factoid Question Answering](https://mlsanity.com/paper/2204.04711/) (2022)
- [Continual Learning of Long Topic Sequences in Neural Information Retrieval](https://mlsanity.com/paper/2201.03356/) (2022)
- [Data Augmentation Methods for Anaphoric Zero Pronouns](https://mlsanity.com/paper/2109.09825/) (2021)
- [Filtering with Confidence: When Data Augmentation Meets Conformal Prediction](https://mlsanity.com/paper/2509.21479/) (2025)
- [PReGAN: Answer Oriented Passage Ranking with Weakly Supervised GAN](https://mlsanity.com/paper/2207.01762/) (2022)
- [Sentiment Analysis of Movie Reviews Using BERT](https://mlsanity.com/paper/2502.18841/) (2025)
- [QUARTZ : QA-based Unsupervised Abstractive Refinement for Task-oriented Dialogue Summarization](https://mlsanity.com/paper/2509.26302/) (2025)
- [Fine-tuning BERT with Bidirectional LSTM for Fine-grained Movie Reviews Sentiment Analysis](https://mlsanity.com/paper/2502.20682/) (2025)
- [UATTA-EB: Uncertainty-Aware Test-Time Augmented Ensemble of BERTs for Classifying Common Mental Illnesses on Social Media Posts](https://mlsanity.com/paper/2304.04539/) (2023)
- [Abstractive Summarization as Augmentation for Document-Level Event Detection](https://mlsanity.com/paper/2305.18023/) (2023)

## Abstract (arXiv)

Recently, a simple combination of passage retrieval using off-the-shelf IR techniques and a BERT reader was found to be very effective for question answering directly on Wikipedia, yielding a large improvement over the previous state of the art on a standard benchmark dataset. In this paper, we present a data augmentation technique using distant supervision that exploits positive as well as negative examples. We apply a stage-wise approach to fine tuning BERT on multiple datasets, starting with data that is "furthest" from the test data and ending with the "closest". Experimental results show large gains in effectiveness over previous approaches on English QA datasets, and we establish new baselines on two recent Chinese QA datasets.