mlsanityData to 14 Oct 2026

FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning

arXiv 2108.06332 ↗First posted 13 Aug 2021cs.CLcited by 32

Not on the map. Why?

Abstract (arXiv)

Most previous methods for text data augmentation are limited to simple tasks and weak baselines. We explore data augmentation on hard tasks (i.e., few-shot natural language understanding) and strong baselines (i.e., pretrained models with over one billion parameters). Under this setting, we reproduced a large number of previous augmentation methods and found that these methods bring marginal gains at best and sometimes degrade the performance much. To address this challenge, we propose a novel data augmentation method FlipDA that jointly uses a generative model and a classifier to generate label-flipped data. Central to the idea of FlipDA is the discovery that generating label-flipped data is more crucial to the performance than generating label-preserved data. Experiments show that FlipDA achieves a good tradeoff between effectiveness and robustness -- it substantially improves many tasks while not negatively affecting the others.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 53 of its 80 references that are in the map. First posted on arXiv 13 Aug 2021.

Builds on

53 of 80 references.

References · 53

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  10. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  11. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  12. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
38 more
  1. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  2. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  3. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  4. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  5. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  6. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 669
  7. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
  8. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 439
  9. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 384
  10. 2020Meta Pseudo Labelscited by 374
  11. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 372
  12. 2020Rethinking Pre-training and Self-trainingcited by 352
  13. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  14. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  15. 2019Billion-scale semi-supervised learning for image classificationcited by 232
  16. 2018Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relationscited by 201
  17. 2018ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehensioncited by 200
  18. 2020MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classificationcited by 187
  19. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 187
  20. 2019Not Enough Data? Deep Learning to the Rescue!cited by 175
  21. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  22. 2018QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehensioncited by 164
  23. 2019Revisiting Self-Training for Neural Sequence Generationcited by 155
  24. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  25. 2020Contextualized Perturbation for Textual Adversarial Attackcited by 136
  26. 2019Self-Training for End-to-End Speech Recognitioncited by 129
  27. 2020SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustnesscited by 96
  28. 2020Sequence-Level Mixed Sample Data Augmentationcited by 71
  29. 2018Sequence-to-Sequence Data Augmentation for Dialogue Language Understandingcited by 68
  30. 2020Knowledge-Aware Language Model Pretrainingcited by 52
  31. 2019A Closer Look At Feature Space Data Augmentation For Few-Shot Intent Classificationcited by 45
  32. 2020How Effective is Task-Agnostic Data Augmentation for Pretrained Transformers?cited by 41
  33. 2020Local Additivity Based Data Augmentation for Semi-supervised NERcited by 31
  34. 2020Snippext: Semi-supervised Opinion Mining with Augmented Datacited by 26
  35. 2020CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detectioncited by 23
  36. 2018Data Augmentation for Spoken Language Understanding via Joint Variational Generationcited by 21
  37. 2020Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Spacecited by 19
  38. 2020Control, Generate, Augment: A Scalable Framework for Multi-Attribute Text Generationcited by 18

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 8; 2023: 11; 2024: 7; 2025: 4.

32 papers.

Sort:

By area:

  1. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 163
  2. 2023Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventionscited by 125
  3. 2021Data Augmentation Approaches in Natural Language Processing: A Surveycited by 82
  4. 2024A Survey on Data Augmentation in Large Model Eracited by 31
  5. 2023Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Modelscited by 20
  6. 2021FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understandingcited by 17
  7. 2022GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentationcited by 11
  8. 2022PromptDA: Label-guided Data Augmentation for Prompt-based Few-shot Learnerscited by 7
  9. 2022AutoCAD: Automatically Generating Counterfactuals for Mitigating Shortcut Learningcited by 4
  10. 2023MixPro: Simple yet Effective Data Augmentation for Prompt-based Learningcited by 4
  11. 2023MetaModulation: Learning Variational Feature Hierarchies for Few-Shot Learning with Fewer Taskscited by 4
  12. 2023Domain Adaptive Few-Shot Open-Set Learningcited by 4
20 more
  1. 2023Improving Few-shot Generalization of Safety Classifiers via Data Augmented Parameter-Efficient Fine-Tuningcited by 4
  2. 2024Enhancing Few-Shot Image Classification through Learnable Multi-Scale Embedding and Attention Mechanismscited by 4
  3. 2022Enhancing Cross-lingual Prompting with Dual Prompt Augmentationcited by 3
  4. 2022Augmentation with Projection: Towards an Effective and Efficient Data Augmentation Paradigm for Distillationcited by 2
  5. 2023Understanding the Effect of Data Augmentation on Knowledge Distillationcited by 2
  6. 2023Rethink the Effectiveness of Text Data Augmentation: An Empirical Analysiscited by 2
  7. 2025MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teachingcited by 2
  8. 2022KnowDA: All-in-One Knowledge Mixture Model for Data Augmentation in Low-Resource NLPcited by 1
  9. 2022A Unified Knowledge Graph Augmentation Service for Boosting Domain-specific NLP Taskscited by 1
  10. 2023Low-Resource Multi-Granularity Academic Function Recognition Based on Multiple Prompt Knowledgecited by 1
  11. 2023Unifying Token and Span Level Supervisions for Few-Shot Sequence Labelingcited by 1
  12. 2024Few-shot Image Generation via Information Transfer from the Built Geodesic Surfacecited by 1
  13. 2024From Robustness to Improved Generalization and Calibration in Pre-trained Language Modelscited by 1
  14. 2024Boosting Few-Shot Detection with Large Language Models and Layout-to-Image Synthesiscited by 1
  15. 2023RoPDA: Robust Prompt-based Data Augmentation for Low-Resource Named Entity Recognitioncited by 0
  16. 2024A Unified Manifold Similarity Measure Enhancing Few-Shot, Transfer, and Reinforcement Learning in Manifold-Distributed Datasetscited by 0
  17. 2024Appearance Matching Adapter for Exemplar-based Semantic Image Synthesis in-the-Wildcited by 0
  18. 2025A Comprehensive Survey of Challenges and Opportunities of Few-Shot Learning Across Multiple Domainscited by 0
  19. 2025Actively evaluating and learning the distinctions that matter: Vaccine safety signal detection from emergency triage notescited by 0
  20. 2025Expert-guided Clinical Text Augmentation via Query-Based Model Collaborationcited by 0