# Learning to Adapt to Low-Resource Paraphrase Generation

arXiv: https://arxiv.org/abs/2412.17111 · HTML: https://mlsanity.com/paper/2412.17111/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 49 references that are in the map.
- First posted on arXiv 22 Dec 2024.

## Builds on (most cited first, 30 of 31)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Towards a Unified View of Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2110.04366/) (2021)
- [AdapterFusion: Non-Destructive Task Composition for Transfer Learning](https://mlsanity.com/paper/2005.00247/) (2020)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://mlsanity.com/paper/2005.00052/) (2020)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Simple, Scalable Adaptation for Neural Machine Translation](https://mlsanity.com/paper/1909.08478/) (2019)
- [Characterizing and Avoiding Negative Transfer](https://mlsanity.com/paper/1811.09751/) (2018)
- [Meta-Transfer Learning for Zero-Shot Super-Resolution](https://mlsanity.com/paper/2002.12213/) (2020)
- [CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling](https://mlsanity.com/paper/1811.10996/) (2018)
- [Pre-training via Paraphrasing](https://mlsanity.com/paper/2006.15020/) (2020)
- [A Continuously Growing Dataset of Sentential Paraphrases](https://mlsanity.com/paper/1708.00391/) (2017)
- [Paraphrase Generation with Deep Reinforcement Learning](https://mlsanity.com/paper/1711.00279/) (2017)
- [Neural Paraphrase Generation with Stacked Residual LSTM Networks](https://mlsanity.com/paper/1610.03098/) (2016)
- [PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks](https://mlsanity.com/paper/2202.12499/) (2022)
- [Syntax-guided Controlled Generation of Paraphrases](https://mlsanity.com/paper/2005.08417/) (2020)
- [Learning to Paraphrase for Question Answering](https://mlsanity.com/paper/1708.06022/) (2017)
- [Unsupervised Paraphrasing by Simulated Annealing](https://mlsanity.com/paper/1909.03588/) (2019)
- [Decomposable Neural Paraphrase Generation](https://mlsanity.com/paper/1906.09741/) (2019)
- [Unsupervised Paraphrasing via Deep Reinforcement Learning](https://mlsanity.com/paper/2007.02244/) (2020)
- [Novelty Controlled Paraphrase Generation with Retrieval Augmented Conditional Prompt Tuning](https://mlsanity.com/paper/2202.00535/) (2022)
- [Factorising Meaning and Form for Intent-Preserving Paraphrasing](https://mlsanity.com/paper/2105.15053/) (2021)
- [Meta-Transfer Learning for Low-Resource Abstractive Summarization](https://mlsanity.com/paper/2102.09397/) (2021)
- [ConRPG: Paraphrase Generation using Contexts as Regularizer](https://mlsanity.com/paper/2109.00363/) (2021)
- [Reflective Decoding: Beyond Unidirectional Generation with Off-the-Shelf Language Models](https://mlsanity.com/paper/2010.08566/) (2020)
- [Unsupervised Neural Machine Translation for Low-Resource Domains via Meta-Learning](https://mlsanity.com/paper/2010.09046/) (2020)

## Built on by (most cited first, 2 of 2)

- [GradEscape: A Gradient-Based Evader Against AI-Generated Text Detectors](https://mlsanity.com/paper/2506.08188/) (2025)
- [Trusted Knowledge Extraction for Operations and Maintenance Intelligence](https://mlsanity.com/paper/2507.22935/) (2025)

## Abstract (arXiv)

Paraphrase generation is a longstanding NLP task and achieves great success with the aid of large corpora. However, transferring a paraphrasing model to another domain encounters the problem of domain shifting especially when the data is sparse. At the same time, widely using large pre-trained language models (PLMs) faces the overfitting problem when training on scarce labeled data. To mitigate these two issues, we propose, LAPA, an effective adapter for PLMs optimized by meta-learning. LAPA has three-stage training on three types of related resources to solve this problem: 1. pre-training PLMs on unsupervised corpora, 2. inserting an adapter layer and meta-training on source domain labeled data, and 3. fine-tuning adapters on a small amount of target domain labeled data. This method enables paraphrase generation models to learn basic language knowledge first, then learn the paraphrasing task itself later, and finally adapt to the target task. Our experimental results demonstrate that LAPA achieves state-of-the-art in supervised, unsupervised, and low-resource settings on three benchmark datasets. With only 2% of trainable parameters and 1% labeled data of the target task, our approach can achieve a competitive performance with previous work.