# LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning

arXiv: https://arxiv.org/abs/2305.18169 · HTML: https://mlsanity.com/paper/2305.18169/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 47 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [One weird trick for parallelizing convolutional neural networks](https://mlsanity.com/paper/1404.5997/) (2014)
- [PTR: Prompt Tuning with Rules for Text Classification](https://mlsanity.com/paper/2105.11259/) (2021)
- [Bootstrapping Semantic Segmentation with Regional Contrast](https://mlsanity.com/paper/2104.04465/) (2021)
- [CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classification](https://mlsanity.com/paper/2204.14244/) (2022)
- [Unsupervised Paraphrasing via Deep Reinforcement Learning](https://mlsanity.com/paper/2007.02244/) (2020)
- [Unsupervised Paraphrasing without Translation](https://mlsanity.com/paper/1905.12752/) (2019)
- [Towards Unified Prompt Tuning for Few-shot Text Classification](https://mlsanity.com/paper/2205.05313/) (2022)
- [Contrastive Learning for Prompt-Based Few-Shot Language Learners](https://mlsanity.com/paper/2205.01308/) (2022)
- [Prompt Tuning for Discriminative Pre-trained Language Models](https://mlsanity.com/paper/2205.11166/) (2022)
- [Unsupervised Paraphrase Generation using Pre-trained Language Models](https://mlsanity.com/paper/2006.05477/) (2020)
- [Unsupervised Contextual Paraphrase Generation using Lexical Control and Reinforcement Learning](https://mlsanity.com/paper/2103.12777/) (2021)
- [ExtraPhrase: Efficient Data Augmentation for Abstractive Summarization](https://mlsanity.com/paper/2201.05313/) (2022)

## Built on by (most cited first, 16 of 16)

- [Heterogeneous Contrastive Learning for Foundation Models and Beyond](https://mlsanity.com/paper/2404.00225/) (2024)
- [A Comprehensive Survey of Contamination Detection Methods in Large Language Models](https://mlsanity.com/paper/2404.00699/) (2024)
- [Which Programming Language and What Features at Pre-training Stage Affect Downstream Logical Inference Performance?](https://mlsanity.com/paper/2410.06735/) (2024)
- [Building a Family of Data Augmentation Models for Low-cost LLM Fine-tuning on the Cloud](https://mlsanity.com/paper/2412.04871/) (2024)
- [Measuring Moral Inconsistencies in Large Language Models](https://mlsanity.com/paper/2402.01719/) (2024)
- [SaGE: Evaluating Moral Consistency in Large Language Models](https://mlsanity.com/paper/2402.13709/) (2024)
- [LM-mixup: Text Data Augmentation via Language Model based Mixup](https://mlsanity.com/paper/2510.20449/) (2025)
- [RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models](https://mlsanity.com/paper/2403.02271/) (2024)
- [EmoSpeech: A Corpus of Emotionally Rich and Contextually Detailed Speech Annotations](https://mlsanity.com/paper/2412.06581/) (2024)
- [Extracting Recurring Vulnerabilities from Black-Box LLM-Generated Software](https://mlsanity.com/paper/2602.04894/) (2026)
- [TRACER: A Semantic-Aware Framework for Fine-Grained Contamination Detection in Code LLMs](https://mlsanity.com/paper/2605.24079/) (2026)
- [BCAmirs at SemEval-2024 Task 4: Beyond Words: A Multimodal and Multilingual Exploration of Persuasion in Memes](https://mlsanity.com/paper/2404.03022/) (2024)
- [Cropping outperforms dropout as an augmentation strategy for self-supervised training of text embeddings](https://mlsanity.com/paper/2508.03453/) (2025)
- [Adam's Law: Textual Frequency Law on Large Language Models](https://mlsanity.com/paper/2604.02176/) (2026)
- [SciCodePile: A 128GB Corpus and Executable Benchmark for Challenging Scientific Code Generation](https://mlsanity.com/paper/2607.19104/) (2026)
- [SharP: Soft and Hard Prompt-Guided Augmentation with LLM for Low Resource Fake News Detection](https://mlsanity.com/paper/2610.13207/) (2026)

## Abstract (arXiv)

In recent years, there has been significant progress in developing pre-trained language models for NLP. However, these models often struggle when fine-tuned on small datasets. To address this issue, researchers have proposed various adaptation approaches. Prompt-based tuning is arguably the most common way, especially for larger models. Previous research shows that adding contrastive learning to prompt-based fine-tuning is effective as it helps the model generate embeddings that are more distinguishable between classes, and it can also be more sample-efficient as the model learns from positive and negative examples simultaneously. One of the most important components of contrastive learning is data augmentation, but unlike computer vision, effective data augmentation for NLP is still challenging. This paper proposes LM-CPPF, Contrastive Paraphrasing-guided Prompt-based Fine-tuning of Language Models, which leverages prompt-based few-shot paraphrasing using generative language models, especially large language models such as GPT-3 and OPT-175B, for data augmentation. Our experiments on multiple text classification benchmarks show that this augmentation method outperforms other methods, such as easy data augmentation, back translation, and multiple templates.