# Unsupervised Contextual Paraphrase Generation using Lexical Control and Reinforcement Learning

arXiv: https://arxiv.org/abs/2103.12777 · HTML: https://mlsanity.com/paper/2103.12777/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 42 references that are in the map.
- First posted on arXiv 23 Mar 2021.

## Builds on (most cited first, 28 of 28)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [PAWS: Paraphrase Adversaries from Word Scrambling](https://mlsanity.com/paper/1904.01130/) (2019)
- [PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification](https://mlsanity.com/paper/1908.11828/) (2019)
- [Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search](https://mlsanity.com/paper/1704.07138/) (2017)
- [Efficient Natural Language Response Suggestion for Smart Reply](https://mlsanity.com/paper/1705.00652/) (2017)
- [QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension](https://mlsanity.com/paper/1804.09541/) (2018)
- [Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing](https://mlsanity.com/paper/2004.14564/) (2020)
- [CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling](https://mlsanity.com/paper/1811.10996/) (2018)
- [A Study of Reinforcement Learning for Neural Machine Translation](https://mlsanity.com/paper/1808.08866/) (2018)
- [Sequence-to-Sequence Data Augmentation for Dialogue Language Understanding](https://mlsanity.com/paper/1807.01554/) (2018)
- [Target-Guided Open-Domain Conversation](https://mlsanity.com/paper/1905.11553/) (2019)
- [Paraphrase Generation with Deep Reinforcement Learning](https://mlsanity.com/paper/1711.00279/) (2017)
- [A Deep Generative Framework for Paraphrase Generation](https://mlsanity.com/paper/1709.05074/) (2017)
- [Neural Paraphrase Generation with Stacked Residual LSTM Networks](https://mlsanity.com/paper/1610.03098/) (2016)
- [A Controllable Model of Grounded Response Generation](https://mlsanity.com/paper/2005.00613/) (2020)
- [Unsupervised Paraphrasing by Simulated Annealing](https://mlsanity.com/paper/1909.03588/) (2019)
- [Unsupervised Paraphrasing via Deep Reinforcement Learning](https://mlsanity.com/paper/2007.02244/) (2020)
- [Syntax-Infused Variational Autoencoder for Text Generation](https://mlsanity.com/paper/1906.02181/) (2019)
- [Joint Copying and Restricted Generation for Paraphrase](https://mlsanity.com/paper/1611.09235/) (2016)
- [Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation](https://mlsanity.com/paper/1803.01465/) (2018)

## Built on by (most cited first, 10 of 10)

- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Quality Controlled Paraphrase Generation](https://mlsanity.com/paper/2203.10940/) (2022)
- [Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges](https://mlsanity.com/paper/2405.15604/) (2024)
- [LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning](https://mlsanity.com/paper/2305.18169/) (2023)
- [FACTIFY3M: A Benchmark for Multimodal Fact Verification with Explainability through 5W Question-Answering](https://mlsanity.com/paper/2306.05523/) (2023)
- [Entailment Relation Aware Paraphrase Generation](https://mlsanity.com/paper/2203.10483/) (2022)
- [Textual Data Distributions: Kullback Leibler Textual Distributions Contrasts on GPT-2 Generated Texts, with Supervised, Unsupervised Learning on Vaccine & Market Topics & Sentiment](https://mlsanity.com/paper/2107.02025/) (2021)
- [Learning Natural Language Generation from Scratch](https://mlsanity.com/paper/2109.09371/) (2021)
- [RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models](https://mlsanity.com/paper/2403.02271/) (2024)
- [SMCLM: Semantically Meaningful Causal Language Modeling for Autoregressive Paraphrase Generation](https://mlsanity.com/paper/2507.03415/) (2025)

## Abstract (arXiv)

Customer support via chat requires agents to resolve customer queries with minimum wait time and maximum customer satisfaction. Given that the agents as well as the customers can have varying levels of literacy, the overall quality of responses provided by the agents tend to be poor if they are not predefined. But using only static responses can lead to customer detraction as the customers tend to feel that they are no longer interacting with a human. Hence, it is vital to have variations of the static responses to reduce monotonicity of the responses. However, maintaining a list of such variations can be expensive. Given the conversation context and the agent response, we propose an unsupervised frame-work to generate contextual paraphrases using autoregressive models. We also propose an automated metric based on Semantic Similarity, Textual Entailment, Expression Diversity and Fluency to evaluate the quality of contextual paraphrases and demonstrate performance improvement with Reinforcement Learning (RL) fine-tuning using the automated metric as the reward function.