# Neural Paraphrase Generation with Stacked Residual LSTM Networks

arXiv: https://arxiv.org/abs/1610.03098 · HTML: https://mlsanity.com/paper/1610.03098/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2016.

## Built on by (most cited first, 30 of 52)

- [GODIVA: Generating Open-DomaIn Videos from nAtural Descriptions](https://mlsanity.com/paper/2104.14806/) (2021)
- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://mlsanity.com/paper/2106.07499/) (2021)
- [DINOISER: Diffused Conditional Sequence Learning by Manipulating Noises](https://mlsanity.com/paper/2302.10025/) (2023)
- [Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs](https://mlsanity.com/paper/2101.10579/) (2021)
- [A Comprehensive Survey on Data Augmentation](https://mlsanity.com/paper/2405.09591/) (2024)
- [How Large Language Models are Transforming Machine-Paraphrased Plagiarism](https://mlsanity.com/paper/2210.03568/) (2022)
- [LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging](https://mlsanity.com/paper/2209.09900/) (2022)
- [Novelty Controlled Paraphrase Generation with Retrieval Augmented Conditional Prompt Tuning](https://mlsanity.com/paper/2202.00535/) (2022)
- [Hierarchical Sketch Induction for Paraphrase Generation](https://mlsanity.com/paper/2203.03463/) (2022)
- [A Review of Text Style Transfer using Deep Learning](https://mlsanity.com/paper/2109.15144/) (2021)
- [ConRPG: Paraphrase Generation using Contexts as Regularizer](https://mlsanity.com/paper/2109.00363/) (2021)
- [Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priors](https://mlsanity.com/paper/2204.01227/) (2022)
- [Paraphrase Identification with Deep Learning: A Review of Datasets and Methods](https://mlsanity.com/paper/2212.06933/) (2022)
- [ParaQA: A Question Answering Dataset with Paraphrase Responses for Single-Turn Conversation](https://mlsanity.com/paper/2103.07771/) (2021)
- [An Empirical Comparison on Imitation Learning and Reinforcement Learning for Paraphrase Generation](https://mlsanity.com/paper/1908.10835/) (2019)
- [Towards Document-Level Paraphrase Generation with Sentence Rewriting and Reordering](https://mlsanity.com/paper/2109.07095/) (2021)
- [ParaAMR: A Large-Scale Syntactically Diverse Paraphrase Dataset by AMR Back-Translation](https://mlsanity.com/paper/2305.16585/) (2023)
- [Unsupervised Contextual Paraphrase Generation using Lexical Control and Reinforcement Learning](https://mlsanity.com/paper/2103.12777/) (2021)
- [Prophet Attention: Predicting Attention with Future Attention for Image Captioning](https://mlsanity.com/paper/2210.10914/) (2022)
- [ExtraPhrase: Efficient Data Augmentation for Abstractive Summarization](https://mlsanity.com/paper/2201.05313/) (2022)
- [Pushing Paraphrase Away from Original Sentence: A Multi-Round Paraphrase Generation Approach](https://mlsanity.com/paper/2109.01862/) (2021)
- [Math Word Problem Solving by Generating Linguistic Variants of Problem Statements](https://mlsanity.com/paper/2306.13899/) (2023)
- [Contrastive Representation Learning for Exemplar-Guided Paraphrase Generation](https://mlsanity.com/paper/2109.01484/) (2021)
- [From Solving a Problem Boldly to Cutting the Gordian Knot: Idiomatic Text Generation](https://mlsanity.com/paper/2104.06541/) (2021)
- [Simulated annealing for optimization of graphs and sequences](https://mlsanity.com/paper/2110.01384/) (2021)
- [AskIt: Unified Programming Interface for Programming with Large Language Models](https://mlsanity.com/paper/2308.15645/) (2023)
- [Fake News Detection After LLM Laundering: Measurement and Explanation](https://mlsanity.com/paper/2501.18649/) (2025)
- [Learning to Selectively Learn for Weakly-supervised Paraphrase Generation](https://mlsanity.com/paper/2109.12457/) (2021)
- [Entailment Relation Aware Paraphrase Generation](https://mlsanity.com/paper/2203.10483/) (2022)
- [N-gram Is Back: Residual Learning of Neural Text Generation with n-gram Language Model](https://mlsanity.com/paper/2210.14431/) (2022)

## Abstract (arXiv)

In this paper, we propose a novel neural approach for paraphrase generation. Conventional para- phrase generation methods either leverage hand-written rules and thesauri-based alignments, or use statistical machine learning principles. To the best of our knowledge, this work is the first to explore deep learning models for paraphrase generation. Our primary contribution is a stacked residual LSTM network, where we add residual connections between LSTM layers. This allows for efficient training of deep LSTMs. We evaluate our model and other state-of-the-art deep learning models on three different datasets: PPDB, WikiAnswers and MSCOCO. Evaluation results demonstrate that our model outperforms sequence to sequence, attention-based and bi- directional LSTM models on BLEU, METEOR, TER and an embedding-based sentence similarity metric.