# Unsupervised Paraphrasing via Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2007.02244 · HTML: https://mlsanity.com/paper/2007.02244/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2020.

## Built on by (most cited first, 25 of 25)

- [RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting](https://mlsanity.com/paper/2305.15685/) (2023)
- [On the Evaluation Metrics for Paraphrase Generation](https://mlsanity.com/paper/2202.08479/) (2022)
- [Quality Controlled Paraphrase Generation](https://mlsanity.com/paper/2203.10940/) (2022)
- [Generalized Zero-shot Intent Detection via Commonsense Knowledge](https://mlsanity.com/paper/2102.02925/) (2021)
- [Text Revision by On-the-Fly Representation Optimization](https://mlsanity.com/paper/2204.07359/) (2022)
- [LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning](https://mlsanity.com/paper/2305.18169/) (2023)
- [ConRPG: Paraphrase Generation using Contexts as Regularizer](https://mlsanity.com/paper/2109.00363/) (2021)
- [Linguistically-Enriched and Context-Aware Zero-shot Slot Filling](https://mlsanity.com/paper/2101.06514/) (2021)
- [LLM-Friendly Knowledge Representation for Customer Support](https://mlsanity.com/paper/2510.10331/) (2025)
- [Reinforcement Learning for Generative AI: A Survey](https://mlsanity.com/paper/2308.14328/) (2023)
- [Unsupervised Contextual Paraphrase Generation using Lexical Control and Reinforcement Learning](https://mlsanity.com/paper/2103.12777/) (2021)
- [Pushing Paraphrase Away from Original Sentence: A Multi-Round Paraphrase Generation Approach](https://mlsanity.com/paper/2109.01862/) (2021)
- [Personalizing Task-oriented Dialog Systems via Zero-shot Generalizable Reward Function](https://mlsanity.com/paper/2303.13797/) (2023)
- [Towards an On-device Agent for Text Rewriting](https://mlsanity.com/paper/2308.11807/) (2023)
- [Proofread: Fixes All Errors with One Tap](https://mlsanity.com/paper/2406.04523/) (2024)
- [Learning to Selectively Learn for Weakly-supervised Paraphrase Generation](https://mlsanity.com/paper/2109.12457/) (2021)
- [Entailment Relation Aware Paraphrase Generation](https://mlsanity.com/paper/2203.10483/) (2022)
- [Proactive Prioritization of App Issues via Contrastive Learning](https://mlsanity.com/paper/2303.06586/) (2023)
- [Dr Genre: Reinforcement Learning from Decoupled LLM Feedback for Generic Text Rewriting](https://mlsanity.com/paper/2503.06781/) (2025)
- [RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models](https://mlsanity.com/paper/2403.02271/) (2024)
- [Learning to Adapt to Low-Resource Paraphrase Generation](https://mlsanity.com/paper/2412.17111/) (2024)
- [Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance](https://mlsanity.com/paper/2604.10590/) (2026)
- [Paraphrase Generation as Unsupervised Machine Translation](https://mlsanity.com/paper/2109.02950/) (2021)
- [Draft, Command, and Edit: Controllable Text Editing in E-Commerce](https://mlsanity.com/paper/2208.05623/) (2022)
- [Action Controlled Paraphrasing](https://mlsanity.com/paper/2405.11277/) (2024)

## Abstract (arXiv)

Paraphrasing is expressing the meaning of an input sentence in different wording while maintaining fluency (i.e., grammatical and syntactical correctness). Most existing work on paraphrasing use supervised models that are limited to specific domains (e.g., image captions). Such models can neither be straightforwardly transferred to other domains nor generalize well, and creating labeled training data for new domains is expensive and laborious. The need for paraphrasing across different domains and the scarcity of labeled training data in many such domains call for exploring unsupervised paraphrase generation methods. We propose Progressive Unsupervised Paraphrasing (PUP): a novel unsupervised paraphrase generation method based on deep reinforcement learning (DRL). PUP uses a variational autoencoder (trained using a non-parallel corpus) to generate a seed paraphrase that warm-starts the DRL model. Then, PUP progressively tunes the seed paraphrase guided by our novel reward function which combines semantic adequacy, language fluency, and expression diversity measures to quantify the quality of the generated paraphrases in each iteration without needing parallel sentences. Our extensive experimental evaluation shows that PUP outperforms unsupervised state-of-the-art paraphrasing techniques in terms of both automatic metrics and user studies on four real datasets. We also show that PUP outperforms domain-adapted supervised algorithms on several datasets. Our evaluation also shows that PUP achieves a great trade-off between semantic similarity and diversity of expression.