# ParaBank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translation

arXiv: https://arxiv.org/abs/1901.03644 · HTML: https://mlsanity.com/paper/1901.03644/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jan 2019.

## Built on by (most cited first, 30 of 38)

- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs](https://mlsanity.com/paper/2101.10579/) (2021)
- [Guided Generation of Cause and Effect](https://mlsanity.com/paper/2107.09846/) (2021)
- [Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing](https://mlsanity.com/paper/2305.16635/) (2023)
- [Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation](https://mlsanity.com/paper/2104.06457/) (2021)
- [Factorising Meaning and Form for Intent-Preserving Paraphrasing](https://mlsanity.com/paper/2105.15053/) (2021)
- [Nugget: Neural Agglomerative Embeddings of Text](https://mlsanity.com/paper/2310.01732/) (2023)
- [Themis: A Reference-free NLG Evaluation Language Model with Flexibility and Interpretability](https://mlsanity.com/paper/2406.18365/) (2024)
- [BiSECT: Learning to Split and Rephrase Sentences with Bitexts](https://mlsanity.com/paper/2109.05006/) (2021)
- [Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification](https://mlsanity.com/paper/2109.00523/) (2021)
- [An Unsupervised Method for Building Sentence Simplification Corpora in Multiple Languages](https://mlsanity.com/paper/2109.00165/) (2021)
- [Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality](https://mlsanity.com/paper/2106.04102/) (2021)
- [ConRPG: Paraphrase Generation using Contexts as Regularizer](https://mlsanity.com/paper/2109.00363/) (2021)
- [ParaAMR: A Large-Scale Syntactically Diverse Paraphrase Dataset by AMR Back-Translation](https://mlsanity.com/paper/2305.16585/) (2023)
- [Learning to Paraphrase Sentences to Different Complexity Levels](https://mlsanity.com/paper/2308.02226/) (2023)
- [Active Evaluation: Efficient NLG Evaluation with Few Pairwise Comparisons](https://mlsanity.com/paper/2203.06063/) (2022)
- [ParaFusion: A Large-Scale LLM-Driven English Paraphrase Dataset Infused with High-Quality Lexical and Syntactic Diversity](https://mlsanity.com/paper/2404.12010/) (2024)
- [GRS: Combining Generation and Revision in Unsupervised Sentence Simplification](https://mlsanity.com/paper/2203.09742/) (2022)
- [ParaCotta: Synthetic Multilingual Paraphrase Corpora from the Most Diverse Translation Sample Pair](https://mlsanity.com/paper/2205.04651/) (2022)
- [Principled Paraphrase Generation with Parallel Corpora](https://mlsanity.com/paper/2205.12213/) (2022)
- [Exploring Diversity in Back Translation for Low-Resource Machine Translation](https://mlsanity.com/paper/2206.00564/) (2022)
- [Quantifying the Gaps Between Translation and Native Perception in Training for Multimodal, Multilingual Retrieval](https://mlsanity.com/paper/2410.02027/) (2024)
- [Better than Random: Reliable NLG Human Evaluation with Constrained Active Sampling](https://mlsanity.com/paper/2406.07967/) (2024)
- [Entailment Relation Aware Paraphrase Generation](https://mlsanity.com/paper/2203.10483/) (2022)
- [Active Learning for Natural Language Generation](https://mlsanity.com/paper/2305.15040/) (2023)
- [InFillmore: Frame-Guided Language Generation with Bidirectional Context](https://mlsanity.com/paper/2103.04941/) (2021)
- [On Neurons Invariant to Sentence Structural Changes in Neural Machine Translation](https://mlsanity.com/paper/2110.03067/) (2021)
- [Improving the Diversity of Unsupervised Paraphrasing with Embedding Outputs](https://mlsanity.com/paper/2110.13231/) (2021)
- [A Survey of Pun Generation: Datasets, Evaluations and Methodologies](https://mlsanity.com/paper/2507.04793/) (2025)

## Abstract (arXiv)

We present ParaBank, a large-scale English paraphrase dataset that surpasses prior work in both quantity and quality. Following the approach of ParaNMT, we train a Czech-English neural machine translation (NMT) system to generate novel paraphrases of English reference sentences. By adding lexical constraints to the NMT decoding procedure, however, we are able to produce multiple high-quality sentential paraphrases per source sentence, yielding an English paraphrase resource with more than 4 billion generated tokens and exhibiting greater lexical diversity. Using human judgments, we also demonstrate that ParaBank's paraphrases improve over ParaNMT on both semantic similarity and fluency. Finally, we use ParaBank to train a monolingual NMT model with the same support for lexically-constrained decoding for sentence rewriting tasks.