# PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks

arXiv: https://arxiv.org/abs/2202.12499 · HTML: https://mlsanity.com/paper/2202.12499/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 25 Feb 2022.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [Not Enough Data? Deep Learning to the Rescue!](https://mlsanity.com/paper/1911.03118/) (2019)
- [Data Augmentation using Pre-trained Transformer Models](https://mlsanity.com/paper/2003.02245/) (2020)
- [Conditional BERT Contextual Augmentation](https://mlsanity.com/paper/1812.06705/) (2018)
- [An Analysis of Simple Data Augmentation for Named Entity Recognition](https://mlsanity.com/paper/2010.11683/) (2020)
- [Towards Zero-Label Language Learning](https://mlsanity.com/paper/2109.09193/) (2021)
- [Sequence-to-Sequence Data Augmentation for Dialogue Language Understanding](https://mlsanity.com/paper/1807.01554/) (2018)
- [Generalized Data Augmentation for Low-Resource Translation](https://mlsanity.com/paper/1906.03785/) (2019)
- [Generative Data Augmentation for Commonsense Reasoning](https://mlsanity.com/paper/2004.11546/) (2020)
- [AUGNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation](https://mlsanity.com/paper/2106.05589/) (2021)

## Built on by (most cited first, 30 of 51)

- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [PLACES: Prompting Language Models for Social Conversation Synthesis](https://mlsanity.com/paper/2302.03269/) (2023)
- [Dipping PLMs Sauce: Bridging Structure and Text for Effective Knowledge Graph Completion via Conditional Soft Prompting](https://mlsanity.com/paper/2307.01709/) (2023)
- [Knowledge Is Flat: A Seq2Seq Generative Framework for Various Knowledge Graph Completion](https://mlsanity.com/paper/2209.07299/) (2022)
- [Geographic and Geopolitical Biases of Language Models](https://mlsanity.com/paper/2212.10408/) (2022)
- [Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generation](https://mlsanity.com/paper/2305.14386/) (2023)
- [Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions](https://mlsanity.com/paper/2504.14772/) (2025)
- [Mixture of Soft Prompts for Controllable Data Generation](https://mlsanity.com/paper/2303.01580/) (2023)
- [Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social Biases](https://mlsanity.com/paper/2307.01595/) (2023)
- [HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models](https://mlsanity.com/paper/2410.01524/) (2024)
- [Language-Guided Music Recommendation for Video via Prompt Analogies](https://mlsanity.com/paper/2306.09327/) (2023)
- [ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Information](https://mlsanity.com/paper/2409.14740/) (2024)
- [Exploring Format Consistency for Instruction Tuning](https://mlsanity.com/paper/2307.15504/) (2023)
- [A Contrastive Cross-Channel Data Augmentation Framework for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2204.07832/) (2022)
- [Localized Symbolic Knowledge Distillation for Visual Commonsense Models](https://mlsanity.com/paper/2312.04837/) (2023)
- [Can LLMs Augment Low-Resource Reading Comprehension Datasets? Opportunities and Challenges](https://mlsanity.com/paper/2309.12426/) (2023)
- [CoDa: Constrained Generation based Data Augmentation for Low-Resource NLP](https://mlsanity.com/paper/2404.00415/) (2024)
- [Prompting-based Synthetic Data Generation for Few-Shot Question Answering](https://mlsanity.com/paper/2405.09335/) (2024)
- [Detecting and Filtering Unsafe Training Data via Data Attribution with Denoised Representation](https://mlsanity.com/paper/2502.11411/) (2025)
- [ABEX: Data Augmentation for Low-Resource NLU via Expanding Abstract Descriptions](https://mlsanity.com/paper/2406.04286/) (2024)
- [Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmar](https://mlsanity.com/paper/2504.09645/) (2025)
- [PromptDA: Label-guided Data Augmentation for Prompt-based Few-shot Learners](https://mlsanity.com/paper/2205.09229/) (2022)
- [Learning to Generalize for Cross-domain QA](https://mlsanity.com/paper/2305.08208/) (2023)
- [Iterative Data Generation with Large Language Models for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2407.00341/) (2024)
- [Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs. Continual Pre-training](https://mlsanity.com/paper/2306.05278/) (2023)
- [MixPro: Simple yet Effective Data Augmentation for Prompt-based Learning](https://mlsanity.com/paper/2304.09402/) (2023)
- [Exploring Limits of Diffusion-Synthetic Training with Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2309.01369/) (2023)
- [Importance-Aware Data Augmentation for Document-Level Neural Machine Translation](https://mlsanity.com/paper/2401.15360/) (2024)
- [Controllable and Diverse Data Augmentation with Large Language Model for Low-Resource Open-Domain Dialogue Generation](https://mlsanity.com/paper/2404.00361/) (2024)
- [BLIAM: Literature-based Data Synthesis for Synergistic Drug Combination Prediction](https://mlsanity.com/paper/2302.06860/) (2023)

## Abstract (arXiv)

This paper focuses on the Data Augmentation for low-resource Natural Language Understanding (NLU) tasks. We propose Prompt-based D}ata Augmentation model (PromDA) which only trains small-scale Soft Prompt (i.e., a set of trainable vectors) in the frozen Pre-trained Language Models (PLMs). This avoids human effort in collecting unlabeled in-domain data and maintains the quality of generated synthetic data. In addition, PromDA generates synthetic data via two different views and filters out the low-quality data using NLU models. Experiments on four benchmarks show that synthetic data produced by PromDA successfully boost up the performance of NLU models which consistently outperform several competitive baseline models, including a state-of-the-art semi-supervised model using unlabeled in-domain data. The synthetic data from PromDA are also complementary with unlabeled in-domain data. The NLU models can be further improved when they are combined for training.