# ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations

arXiv: https://arxiv.org/abs/1711.05732 · HTML: https://mlsanity.com/paper/1711.05732/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Nov 2017.

## Built on by (most cited first, 30 of 44)

- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Generating Datasets with Pretrained Language Models](https://mlsanity.com/paper/2104.07540/) (2021)
- [A Review on Language Models as Knowledge Bases](https://mlsanity.com/paper/2204.06031/) (2022)
- [NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation](https://mlsanity.com/paper/2112.02721/) (2021)
- [Text Detoxification using Large Pre-trained Neural Models](https://mlsanity.com/paper/2109.08914/) (2021)
- [Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration](https://mlsanity.com/paper/2109.06304/) (2021)
- [Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation](https://mlsanity.com/paper/2104.06457/) (2021)
- [Paraphrastic Representations at Scale](https://mlsanity.com/paper/2104.15114/) (2021)
- [Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings](https://mlsanity.com/paper/2110.07385/) (2021)
- [Generate, Discriminate and Contrast: A Semi-Supervised Sentence Representation Learning Framework](https://mlsanity.com/paper/2210.16798/) (2022)
- [XtremeDistilTransformers: Task Transfer for Task-agnostic Distillation](https://mlsanity.com/paper/2106.04563/) (2021)
- [BiSECT: Learning to Split and Rephrase Sentences with Bitexts](https://mlsanity.com/paper/2109.05006/) (2021)
- [Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue System](https://mlsanity.com/paper/2205.04006/) (2022)
- [Disentangling Representations of Text by Masking Transformers](https://mlsanity.com/paper/2104.07155/) (2021)
- [An Unsupervised Method for Building Sentence Simplification Corpora in Multiple Languages](https://mlsanity.com/paper/2109.00165/) (2021)
- [Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality](https://mlsanity.com/paper/2106.04102/) (2021)
- [IGA : An Intent-Guided Authoring Assistant](https://mlsanity.com/paper/2104.07000/) (2021)
- [Paraphrase Detection: Human vs. Machine Content](https://mlsanity.com/paper/2303.13989/) (2023)
- [Towards Document-Level Paraphrase Generation with Sentence Rewriting and Reordering](https://mlsanity.com/paper/2109.07095/) (2021)
- [Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction](https://mlsanity.com/paper/2305.13981/) (2023)
- [Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation](https://mlsanity.com/paper/2106.02833/) (2021)
- [Training Naturalized Semantic Parsers with Very Little Data](https://mlsanity.com/paper/2204.14243/) (2022)
- [ParaCotta: Synthetic Multilingual Paraphrase Corpora from the Most Diverse Translation Sample Pair](https://mlsanity.com/paper/2205.04651/) (2022)
- [From Theories on Styles to their Transfer in Text: Bridging the Gap with a Hierarchical Survey](https://mlsanity.com/paper/2110.15871/) (2021)
- [Onception: Active Learning with Expert Advice for Real World Machine Translation](https://mlsanity.com/paper/2203.04507/) (2022)
- [Principled Paraphrase Generation with Parallel Corpora](https://mlsanity.com/paper/2205.12213/) (2022)
- [Exploring Diversity in Back Translation for Low-Resource Machine Translation](https://mlsanity.com/paper/2206.00564/) (2022)
- [Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operations](https://mlsanity.com/paper/2305.14599/) (2023)
- [Semantic Similarity Loss for Neural Source Code Summarization](https://mlsanity.com/paper/2308.07429/) (2023)
- [Generative Pretraining for Paraphrase Evaluation](https://mlsanity.com/paper/2107.08251/) (2021)

## Abstract (arXiv)

We describe PARANMT-50M, a dataset of more than 50 million English-English sentential paraphrase pairs. We generated the pairs automatically by using neural machine translation to translate the non-English side of a large parallel corpus, following Wieting et al. (2017). Our hope is that ParaNMT-50M can be a valuable resource for paraphrase generation and can provide a rich source of semantic knowledge to improve downstream natural language understanding tasks. To show its utility, we use ParaNMT-50M to train paraphrastic sentence embeddings that outperform all supervised systems on every SemEval semantic textual similarity competition, in addition to showing how it can be used for paraphrase generation.