# What Have Been Learned & What Should Be Learned? An Empirical Study of How to Selectively Augment Text for Classification

arXiv: https://arxiv.org/abs/2109.00175 · HTML: https://mlsanity.com/paper/2109.00175/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 1 Sep 2021.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations](https://mlsanity.com/paper/1805.06201/) (2018)
- [Not Enough Data? Deep Learning to the Rescue!](https://mlsanity.com/paper/1911.03118/) (2019)
- [QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension](https://mlsanity.com/paper/1804.09541/) (2018)
- [Recurrent Neural Network for Text Classification with Multi-Task Learning](https://mlsanity.com/paper/1605.05101/) (2016)
- [Augmenting Data with Mixup for Sentence Classification: An Empirical Study](https://mlsanity.com/paper/1905.08941/) (2019)
- [Data Noising as Smoothing in Neural Network Language Models](https://mlsanity.com/paper/1703.02573/) (2017)
- [Learning Data Manipulation for Augmentation and Weighting](https://mlsanity.com/paper/1910.12795/) (2019)
- [Topic Compositional Neural Language Model](https://mlsanity.com/paper/1712.09783/) (2017)

## Built on by (most cited first, 1 of 1)

- [Feature Weaken: Vicinal Data Augmentation for Classification](https://mlsanity.com/paper/2211.10944/) (2022)

## Abstract (arXiv)

Text augmentation techniques are widely used in text classification problems to improve the performance of classifiers, especially in low-resource scenarios. Whilst lots of creative text augmentation methods have been designed, they augment the text in a non-selective manner, which means the less important or noisy words have the same chances to be augmented as the informative words, and thereby limits the performance of augmentation. In this work, we systematically summarize three kinds of role keywords, which have different functions for text classification, and design effective methods to extract them from the text. Based on these extracted role keywords, we propose STA (Selective Text Augmentation) to selectively augment the text, where the informative, class-indicating words are emphasized but the irrelevant or noisy words are diminished. Extensive experiments on four English and Chinese text classification benchmark datasets demonstrate that STA can substantially outperform the non-selective text augmentation methods.