# ProtAugment: Unsupervised diverse short-texts paraphrasing for intent detection meta-learning

arXiv: https://arxiv.org/abs/2105.12995 · HTML: https://mlsanity.com/paper/2105.12995/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 44 references that are in the map.
- First posted on arXiv 27 May 2021.

## Builds on (most cited first, 30 of 30)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [PAWS: Paraphrase Adversaries from Word Scrambling](https://mlsanity.com/paper/1904.01130/) (2019)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification](https://mlsanity.com/paper/1908.11828/) (2019)
- [Probing Neural Network Comprehension of Natural Language Arguments](https://mlsanity.com/paper/1907.07355/) (2019)
- [Benchmarking Natural Language Understanding Services for building Conversational Agents](https://mlsanity.com/paper/1903.05566/) (2019)
- [Free Lunch for Few-shot Learning: Distribution Calibration](https://mlsanity.com/paper/2101.06395/) (2021)
- [Meta-Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1808.08437/) (2018)
- [DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue](https://mlsanity.com/paper/2009.13570/) (2020)
- [Comparison of Diverse Decoding Methods from Conditional Language Models](https://mlsanity.com/paper/1906.06362/) (2019)
- [Induction Networks for Few-Shot Text Classification](https://mlsanity.com/paper/1902.10482/) (2019)
- [Neural Syntactic Preordering for Controlled Paraphrase Generation](https://mlsanity.com/paper/2005.02013/) (2020)
- [Natural Language Understanding with the Quora Question Pairs Dataset](https://mlsanity.com/paper/1907.01041/) (2019)
- [Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks](https://mlsanity.com/paper/1908.10423/) (2019)
- [Importance of Search and Evaluation Strategies in Neural Dialogue Modeling](https://mlsanity.com/paper/1811.00907/) (2018)
- [Unsupervised Paraphrasing by Simulated Annealing](https://mlsanity.com/paper/1909.03588/) (2019)
- [Natural Language to Structured Query Generation via Meta-Learning](https://mlsanity.com/paper/1803.02400/) (2018)
- [Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model](https://mlsanity.com/paper/1605.03835/) (2016)
- [schuBERT: Optimizing Elements of BERT](https://mlsanity.com/paper/2005.06628/) (2020)
- [A Neural Few-Shot Text Classification Reality Check](https://mlsanity.com/paper/2101.12073/) (2021)
- [Refining Source Representations with Relation Networks for Neural Machine Translation](https://mlsanity.com/paper/1805.11154/) (2018)

## Built on by (most cited first, 18 of 18)

- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation](https://mlsanity.com/paper/2112.02721/) (2021)
- [Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality](https://mlsanity.com/paper/2211.00768/) (2022)
- [Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering](https://mlsanity.com/paper/2211.03433/) (2022)
- [SemEval-2024 Shared Task 6: SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakes](https://mlsanity.com/paper/2403.07726/) (2024)
- [ContrastNet: A Contrastive Learning Framework for Few-Shot Text Classification](https://mlsanity.com/paper/2305.09269/) (2023)
- [A survey of diversity quantification in natural language processing: The why, what, where and how](https://mlsanity.com/paper/2507.20858/) (2025)
- [TART: Improved Few-shot Text Classification Using Task-Adaptive Reference Transformation](https://mlsanity.com/paper/2306.02175/) (2023)
- [The Devil is in the Details: On Models and Training Regimes for Few-Shot Intent Classification](https://mlsanity.com/paper/2210.06440/) (2022)
- [Can Your Model Tell a Negation from an Implicature? Unravelling Challenges With Intent Encoders](https://mlsanity.com/paper/2403.04314/) (2024)
- [Large Language Models on Fine-grained Emotion Detection Dataset with Data Augmentation and Transfer Learning](https://mlsanity.com/paper/2403.06108/) (2024)
- [Prototypical Extreme Multi-label Classification with a Dynamic Margin Loss](https://mlsanity.com/paper/2410.20401/) (2024)
- [Adversarial Word Dilution as Text Data Augmentation in Low-Resource Regime](https://mlsanity.com/paper/2305.09287/) (2023)
- [Data Augmentation for Emotion Detection in Small Imbalanced Text Data](https://mlsanity.com/paper/2310.17015/) (2023)
- [HausaNLP at SemEval-2025 Task 3: Towards a Fine-Grained Model-Aware Hallucination Detection](https://mlsanity.com/paper/2503.19650/) (2025)
- [REFORMER: A ChatGPT-Driven Data Synthesis Framework Elevating Text-to-SQL Models](https://mlsanity.com/paper/2504.04363/) (2025)
- [ProtSi: Prototypical Siamese Network with Data Augmentation for Few-Shot Subjective Answer Evaluation](https://mlsanity.com/paper/2211.09855/) (2022)
- [Exploring Description-Augmented Dataless Intent Classification](https://mlsanity.com/paper/2407.17862/) (2024)

## Abstract (arXiv)

Recent research considers few-shot intent detection as a meta-learning problem: the model is learning to learn from a consecutive set of small tasks named episodes. In this work, we propose ProtAugment, a meta-learning algorithm for short texts classification (the intent detection task). ProtAugment is a novel extension of Prototypical Networks, that limits overfitting on the bias introduced by the few-shots classification objective at each episode. It relies on diverse paraphrasing: a conditional language model is first fine-tuned for paraphrasing, and diversity is later introduced at the decoding stage at each meta-learning episode. The diverse paraphrasing is unsupervised as it is applied to unlabelled data, and then fueled to the Prototypical Network training objective as a consistency loss. ProtAugment is the state-of-the-art method for intent detection meta-learning, at no extra labeling efforts and without the need to fine-tune a conditional language model on a given application domain.