# FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning

arXiv: https://arxiv.org/abs/2108.06332 · HTML: https://mlsanity.com/paper/2108.06332/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 80 references that are in the map.
- First posted on arXiv 13 Aug 2021.

## Builds on (most cited first, 30 of 53)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping](https://mlsanity.com/paper/2002.06305/) (2020)
- [Meta Pseudo Labels](https://mlsanity.com/paper/2003.10580/) (2020)
- [WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations](https://mlsanity.com/paper/1808.09121/) (2018)
- [Rethinking Pre-training and Self-training](https://mlsanity.com/paper/2006.06882/) (2020)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [BAE: BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2004.01970/) (2020)
- [Billion-scale semi-supervised learning for image classification](https://mlsanity.com/paper/1905.00546/) (2019)
- [Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations](https://mlsanity.com/paper/1805.06201/) (2018)
- [ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension](https://mlsanity.com/paper/1810.12885/) (2018)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)

## Built on by (most cited first, 30 of 32)

- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions](https://mlsanity.com/paper/2306.04140/) (2023)
- [Data Augmentation Approaches in Natural Language Processing: A Survey](https://mlsanity.com/paper/2110.01852/) (2021)
- [A Survey on Data Augmentation in Large Model Era](https://mlsanity.com/paper/2401.15422/) (2024)
- [Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models](https://mlsanity.com/paper/2311.00287/) (2023)
- [FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding](https://mlsanity.com/paper/2109.12742/) (2021)
- [GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation](https://mlsanity.com/paper/2211.10330/) (2022)
- [PromptDA: Label-guided Data Augmentation for Prompt-based Few-shot Learners](https://mlsanity.com/paper/2205.09229/) (2022)
- [AutoCAD: Automatically Generating Counterfactuals for Mitigating Shortcut Learning](https://mlsanity.com/paper/2211.16202/) (2022)
- [MixPro: Simple yet Effective Data Augmentation for Prompt-based Learning](https://mlsanity.com/paper/2304.09402/) (2023)
- [MetaModulation: Learning Variational Feature Hierarchies for Few-Shot Learning with Fewer Tasks](https://mlsanity.com/paper/2305.10309/) (2023)
- [Domain Adaptive Few-Shot Open-Set Learning](https://mlsanity.com/paper/2309.12814/) (2023)
- [Improving Few-shot Generalization of Safety Classifiers via Data Augmented Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2310.16959/) (2023)
- [Enhancing Few-Shot Image Classification through Learnable Multi-Scale Embedding and Attention Mechanisms](https://mlsanity.com/paper/2409.07989/) (2024)
- [Enhancing Cross-lingual Prompting with Dual Prompt Augmentation](https://mlsanity.com/paper/2202.07255/) (2022)
- [Augmentation with Projection: Towards an Effective and Efficient Data Augmentation Paradigm for Distillation](https://mlsanity.com/paper/2210.11768/) (2022)
- [Understanding the Effect of Data Augmentation on Knowledge Distillation](https://mlsanity.com/paper/2305.12565/) (2023)
- [Rethink the Effectiveness of Text Data Augmentation: An Empirical Analysis](https://mlsanity.com/paper/2306.07664/) (2023)
- [MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teaching](https://mlsanity.com/paper/2506.02689/) (2025)
- [KnowDA: All-in-One Knowledge Mixture Model for Data Augmentation in Low-Resource NLP](https://mlsanity.com/paper/2206.10265/) (2022)
- [A Unified Knowledge Graph Augmentation Service for Boosting Domain-specific NLP Tasks](https://mlsanity.com/paper/2212.05251/) (2022)
- [Low-Resource Multi-Granularity Academic Function Recognition Based on Multiple Prompt Knowledge](https://mlsanity.com/paper/2305.03287/) (2023)
- [Unifying Token and Span Level Supervisions for Few-Shot Sequence Labeling](https://mlsanity.com/paper/2307.07946/) (2023)
- [Few-shot Image Generation via Information Transfer from the Built Geodesic Surface](https://mlsanity.com/paper/2401.01749/) (2024)
- [From Robustness to Improved Generalization and Calibration in Pre-trained Language Models](https://mlsanity.com/paper/2404.00758/) (2024)
- [Boosting Few-Shot Detection with Large Language Models and Layout-to-Image Synthesis](https://mlsanity.com/paper/2410.06841/) (2024)
- [RoPDA: Robust Prompt-based Data Augmentation for Low-Resource Named Entity Recognition](https://mlsanity.com/paper/2307.07417/) (2023)
- [A Unified Manifold Similarity Measure Enhancing Few-Shot, Transfer, and Reinforcement Learning in Manifold-Distributed Datasets](https://mlsanity.com/paper/2408.07095/) (2024)
- [Appearance Matching Adapter for Exemplar-based Semantic Image Synthesis in-the-Wild](https://mlsanity.com/paper/2412.03150/) (2024)
- [A Comprehensive Survey of Challenges and Opportunities of Few-Shot Learning Across Multiple Domains](https://mlsanity.com/paper/2504.04017/) (2025)

## Abstract (arXiv)

Most previous methods for text data augmentation are limited to simple tasks and weak baselines. We explore data augmentation on hard tasks (i.e., few-shot natural language understanding) and strong baselines (i.e., pretrained models with over one billion parameters). Under this setting, we reproduced a large number of previous augmentation methods and found that these methods bring marginal gains at best and sometimes degrade the performance much. To address this challenge, we propose a novel data augmentation method FlipDA that jointly uses a generative model and a classifier to generate label-flipped data. Central to the idea of FlipDA is the discovery that generating label-flipped data is more crucial to the performance than generating label-preserved data. Experiments show that FlipDA achieves a good tradeoff between effectiveness and robustness -- it substantially improves many tasks while not negatively affecting the others.