Reprint: a randomized extrapolation based on principal components for data augmentation
Not on the map. Why?
Abstract (arXiv)
Data scarcity and data imbalance have attracted a lot of attention in many fields. Data augmentation, explored as an effective approach to tackle them, can improve the robustness and efficiency of classification models by generating new samples. This paper presents REPRINT, a simple and effective hidden-space data augmentation method for imbalanced data classification. Given hidden-space representations of samples in each class, REPRINT extrapolates, in a randomized fashion, augmented examples for target class by using subspaces spanned by principal components to summarize distribution structure of both source and target class. Consequently, the examples generated would diversify the target while maintaining the original geometry of target distribution. Besides, this method involves a label refinement component which allows to synthesize new soft labels for augmented examples. Compared with different NLP data augmentation approaches under a range of data imbalanced scenarios on four text classification benchmark, REPRINT shows prominent improvements. Moreover, through comprehensive ablation studies, we show that label refinement is better than label-preserving for augmented examples, and that our method suggests stable and consistent improvements in terms of suitable choices of principal components. Moreover, REPRINT is appealing for its easy-to-use since it contains only one hyperparameter determining the dimension of subspace and requires low computational resource.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 47 of its 67 references that are in the map. First posted on arXiv 26 Apr 2022.
Builds on
47 of 67 references.
References · 47
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
- 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
- 2018Deep contextualized word representationscited by 2,290
- 2017Random Erasing Data Augmentationcited by 1,239
- 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
35 more
- 2019When Does Label Smoothing Help?cited by 988
- 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
- 2021A Survey of Data Augmentation Approaches for NLPcited by 409
- 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 377
- 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 315
- 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 273
- 2019Representation Degeneration Problem in Training Natural Language Generation Modelscited by 255
- 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 254
- 2020MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classificationcited by 187
- 2019Not Enough Data? Deep Learning to the Rescue!cited by 175
- 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
- 2019Good-Enough Compositional Data Augmentationcited by 158
- 2020Deep Learning Based Text Classification: A Comprehensive Reviewcited by 141
- 2017Dataset Augmentation in Feature Spacecited by 124
- 2021A Survey on Data Augmentation for Text Classificationcited by 111
- 2018Semantic Parsing for Task Oriented Dialog using Hierarchical Representationscited by 101
- 2020Delving Deep into Label Smoothingcited by 94
- 2017Data Noising as Smoothing in Neural Network Language Modelscited by 78
- 2020An Analysis of Simple Data Augmentation for Named Entity Recognitioncited by 76
- 2018Sequence-to-Sequence Data Augmentation for Dialogue Language Understandingcited by 68
- 2019A Closer Look At Feature Space Data Augmentation For Few-Shot Intent Classificationcited by 45
- 2019Learning Data Manipulation for Augmentation and Weightingcited by 45
- 2017Towards Zero-Shot Frame Semantic Parsing for Domain Scalingcited by 40
- 2018Text Data Augmentation Made Simple By Leveraging NLP Cloud APIscited by 40
- 2021The Low-Dimensional Linear Geometry of Contextualized Word Representationscited by 40
- 2021Low-Dimensional Structure in the Space of Language Representations is Reflected in Brain Responsescited by 34
- 2020Snippext: Semi-supervised Opinion Mining with Augmented Datacited by 26
- 2018Feature Space Transfer for Data Augmentationcited by 21
- 2020Unsupervised Paraphrasing with Pretrained Language Modelscited by 15
- 2020Semantic Label Smoothing for Sequence to Sequence Problemscited by 13
- 2021Good-Enough Example Extrapolationcited by 5
- 2020Can We Achieve More with Less? Exploring Data Augmentation for Toxic Comment Classificationcited by 4
- 2021Similarity Based Label Smoothing For Dialogue Generationcited by 2
- 2020Overcoming Conflicting Data when Updating a Neural Semantic Parsercited by 1
Built on by
Citing papers per year: 2022: 1.
1 paper.