# GenAug: Data Augmentation for Finetuning Text Generators

arXiv: https://arxiv.org/abs/2010.01794 · HTML: https://mlsanity.com/paper/2010.01794/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2020.

## Built on by (most cited first, 30 of 41)

- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [A Survey on Data Augmentation for Text Classification](https://mlsanity.com/paper/2107.03158/) (2021)
- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://mlsanity.com/paper/2106.07499/) (2021)
- [The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Tasks](https://mlsanity.com/paper/2304.13861/) (2023)
- [Weakly Supervised Data Augmentation Through Prompting for Dialogue Understanding](https://mlsanity.com/paper/2210.14169/) (2022)
- [ArchivalQA: A Large-scale Benchmark Dataset for Open Domain Question Answering over Historical News Collections](https://mlsanity.com/paper/2109.03438/) (2021)
- [Generating Fake Cyber Threat Intelligence Using Transformer-Based Models](https://mlsanity.com/paper/2102.04351/) (2021)
- [Image Data Augmentation Approaches: A Comprehensive Survey and Future directions](https://mlsanity.com/paper/2301.02830/) (2023)
- [Text Data Augmentation for Large Language Models: A Comprehensive Survey of Methods, Challenges, and Opportunities](https://mlsanity.com/paper/2501.18845/) (2025)
- [Exploring New Frontiers in Agricultural NLP: Investigating the Potential of Large Language Models for Food Applications](https://mlsanity.com/paper/2306.11892/) (2023)
- [TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understanding](https://mlsanity.com/paper/2205.06153/) (2022)
- [NAREOR: The Narrative Reordering Problem](https://mlsanity.com/paper/2104.06669/) (2021)
- [To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP](https://mlsanity.com/paper/2111.09618/) (2021)
- [MDPFuzz: Testing Models Solving Markov Decision Processes](https://mlsanity.com/paper/2112.02807/) (2021)
- [Leveraging QA Datasets to Improve Generative Data Augmentation](https://mlsanity.com/paper/2205.12604/) (2022)
- [GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation](https://mlsanity.com/paper/2211.10330/) (2022)
- [Few-shot LLM Synthetic Data with Distribution Matching](https://mlsanity.com/paper/2502.08661/) (2025)
- [Retrieve, Caption, Generate: Visual Grounding for Enhancing Commonsense in Text Generation Models](https://mlsanity.com/paper/2109.03892/) (2021)
- [A Survey of Data Synthesis Approaches](https://mlsanity.com/paper/2407.03672/) (2024)
- [SAPPHIRE: Approaches for Enhanced Concept-to-Text Generation](https://mlsanity.com/paper/2108.06643/) (2021)
- [Data Augmentation for Low-Resource Keyphrase Generation](https://mlsanity.com/paper/2305.17968/) (2023)
- [CHARD: Clinical Health-Aware Reasoning Across Dimensions for Text Generation Models](https://mlsanity.com/paper/2210.04191/) (2022)
- [Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation](https://mlsanity.com/paper/2106.02833/) (2021)
- [PANCETTA: Phoneme Aware Neural Completion to Elicit Tongue Twisters Automatically](https://mlsanity.com/paper/2209.06275/) (2022)
- [Exploring Empty Spaces: Human-in-the-Loop Data Augmentation](https://mlsanity.com/paper/2410.01088/) (2024)
- [UTNLP at SemEval-2022 Task 6: A Comparative Analysis of Sarcasm Detection Using Generative-based and Mutation-based Data Augmentation](https://mlsanity.com/paper/2204.08198/) (2022)
- [Selective Text Augmentation with Word Roles for Low-Resource Text Classification](https://mlsanity.com/paper/2209.01560/) (2022)
- [Targeted Augmentation for Low-Resource Event Extraction](https://mlsanity.com/paper/2405.08729/) (2024)
- [Data Augmentation Methods for Anaphoric Zero Pronouns](https://mlsanity.com/paper/2109.09825/) (2021)
- [Filtering with Confidence: When Data Augmentation Meets Conformal Prediction](https://mlsanity.com/paper/2509.21479/) (2025)

## Abstract (arXiv)

In this paper, we investigate data augmentation for text generation, which we call GenAug. Text generation and language modeling are important tasks within natural language processing, and are especially challenging for low-data regimes. We propose and evaluate various augmentation methods, including some that incorporate external knowledge, for finetuning GPT-2 on a subset of Yelp Reviews. We also examine the relationship between the amount of augmentation and the quality of the generated text. We utilize several metrics that evaluate important aspects of the generated text including its diversity and fluency. Our experiments demonstrate that insertion of character-level synthetic noise and keyword replacement with hypernyms are effective augmentation methods, and that the quality of generations improves to a peak at approximately three times the amount of original data.