# Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification

arXiv: https://arxiv.org/abs/2109.00523 · HTML: https://mlsanity.com/paper/2109.00523/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 1 Sep 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Fast AutoAugment](https://mlsanity.com/paper/1905.00397/) (2019)
- [Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations](https://mlsanity.com/paper/1805.06201/) (2018)
- [Facebook FAIR's WMT19 News Translation Task Submission](https://mlsanity.com/paper/1907.06616/) (2019)
- [Data Augmentation using Pre-trained Transformer Models](https://mlsanity.com/paper/2003.02245/) (2020)
- [Good-Enough Compositional Data Augmentation](https://mlsanity.com/paper/1904.09545/) (2019)
- [Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules](https://mlsanity.com/paper/1905.05393/) (2019)
- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://mlsanity.com/paper/2106.07499/) (2021)
- [Faster AutoAugment: Learning Augmentation Strategies using Backpropagation](https://mlsanity.com/paper/1911.06987/) (2019)
- [Data augmentation instead of explicit regularization](https://mlsanity.com/paper/1806.03852/) (2018)
- [DADA: Differentiable Automatic Data Augmentation](https://mlsanity.com/paper/2003.03780/) (2020)
- [Learning Data Manipulation for Augmentation and Weighting](https://mlsanity.com/paper/1910.12795/) (2019)
- [Parallel Data Augmentation for Formality Style Transfer](https://mlsanity.com/paper/2005.07522/) (2020)
- [ParaBank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translation](https://mlsanity.com/paper/1901.03644/) (2019)
- [Data Manipulation: Towards Effective Instance Learning for Neural Dialogue Generation via Learning to Augment and Reweight](https://mlsanity.com/paper/2004.02594/) (2020)
- [Atalaya at TASS 2019: Data Augmentation and Robust Embeddings for Sentiment Analysis](https://mlsanity.com/paper/1909.11241/) (2019)

## Built on by (most cited first, 15 of 15)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Automated data processing and feature engineering for deep learning and big data applications: a survey](https://mlsanity.com/paper/2403.11395/) (2024)
- [A Survey of Automated Data Augmentation Algorithms for Deep Learning-based Image Classification Tasks](https://mlsanity.com/paper/2206.06544/) (2022)
- [GDA: Generative Data Augmentation Techniques for Relation Extraction Tasks](https://mlsanity.com/paper/2305.16663/) (2023)
- [PromptDA: Label-guided Data Augmentation for Prompt-based Few-shot Learners](https://mlsanity.com/paper/2205.09229/) (2022)
- [Symmetric Replay Training: Enhancing Sample Efficiency in Deep Reinforcement Learning for Combinatorial Optimization](https://mlsanity.com/paper/2306.01276/) (2023)
- [AugCSE: Contrastive Sentence Embedding with Diverse Augmentations](https://mlsanity.com/paper/2210.13749/) (2022)
- [MixPro: Simple yet Effective Data Augmentation for Prompt-based Learning](https://mlsanity.com/paper/2304.09402/) (2023)
- [UniGen: Universal Domain Generalization for Sentiment Classification via Zero-shot Dataset Generation](https://mlsanity.com/paper/2405.01022/) (2024)
- [Data Augmentation for Neural NLP](https://mlsanity.com/paper/2302.11412/) (2023)
- [GenCode: A Generic Data Augmentation Framework for Boosting Deep Learning-Based Code Understanding](https://mlsanity.com/paper/2402.15769/) (2024)
- [BootAug: Boosting Text Augmentation via Hybrid Instance Filtering Framework](https://mlsanity.com/paper/2210.02941/) (2022)
- [AutoAugment Is What You Need: Enhancing Rule-based Augmentation Methods in Low-resource Regimes](https://mlsanity.com/paper/2402.05584/) (2024)
- [Data augmentation with automated machine learning: approaches and performance comparison with classical data augmentation methods](https://mlsanity.com/paper/2403.08352/) (2024)
- [The Effects of Data Augmentation on Confidence Estimation for LLMs](https://mlsanity.com/paper/2506.11046/) (2025)

## Abstract (arXiv)

Data augmentation aims to enrich training samples for alleviating the overfitting issue in low-resource or class-imbalanced situations. Traditional methods first devise task-specific operations such as Synonym Substitute, then preset the corresponding parameters such as the substitution rate artificially, which require a lot of prior knowledge and are prone to fall into the sub-optimum. Besides, the number of editing operations is limited in the previous methods, which decreases the diversity of the augmented data and thus restricts the performance gain. To overcome the above limitations, we propose a framework named Text AutoAugment (TAA) to establish a compositional and learnable paradigm for data augmentation. We regard a combination of various operations as an augmentation policy and utilize an efficient Bayesian Optimization algorithm to automatically search for the best policy, which substantially improves the generalization capability of models. Experiments on six benchmark datasets show that TAA boosts classification accuracy in low-resource and class-imbalanced regimes by an average of 8.8% and 9.7%, respectively, outperforming strong baselines.