# Paraphrase Generation with Latent Bag of Words

arXiv: https://arxiv.org/abs/2001.01941 · HTML: https://mlsanity.com/paper/2001.01941/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jan 2020.

## Built on by (most cited first, 27 of 27)

- [A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning](https://mlsanity.com/paper/2110.01515/) (2021)
- [PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation](https://mlsanity.com/paper/2203.09100/) (2022)
- [ParaSCI: A Large Scientific Paraphrase Dataset for Longer Paraphrase Generation](https://mlsanity.com/paper/2101.08382/) (2021)
- [Novelty Controlled Paraphrase Generation with Retrieval Augmented Conditional Prompt Tuning](https://mlsanity.com/paper/2202.00535/) (2022)
- [Factorising Meaning and Form for Intent-Preserving Paraphrasing](https://mlsanity.com/paper/2105.15053/) (2021)
- [Hierarchical Sketch Induction for Paraphrase Generation](https://mlsanity.com/paper/2203.03463/) (2022)
- [ConRPG: Paraphrase Generation using Contexts as Regularizer](https://mlsanity.com/paper/2109.00363/) (2021)
- [Paraphrase Detection: Human vs. Machine Content](https://mlsanity.com/paper/2303.13989/) (2023)
- [Non-Parallel Text Style Transfer with Self-Parallel Supervision](https://mlsanity.com/paper/2204.08123/) (2022)
- [Zero-shot point cloud segmentation by transferring geometric primitives](https://mlsanity.com/paper/2210.09923/) (2022)
- [Pushing Paraphrase Away from Original Sentence: A Multi-Round Paraphrase Generation Approach](https://mlsanity.com/paper/2109.01862/) (2021)
- [Controllable Dialogue Generation with Disentangled Multi-grained Style Specification and Attribute Consistency Reward](https://mlsanity.com/paper/2109.06717/) (2021)
- [Principled Paraphrase Generation with Parallel Corpora](https://mlsanity.com/paper/2205.12213/) (2022)
- [Unsupervised Syntactically Controlled Paraphrase Generation with Abstract Meaning Representations](https://mlsanity.com/paper/2211.00881/) (2022)
- [Fake News Detection After LLM Laundering: Measurement and Explanation](https://mlsanity.com/paper/2501.18649/) (2025)
- [Learning to Selectively Learn for Weakly-supervised Paraphrase Generation](https://mlsanity.com/paper/2109.12457/) (2021)
- [Weakly Supervised Reasoning by Neuro-Symbolic Approaches](https://mlsanity.com/paper/2309.13072/) (2023)
- [Parameter Efficient Diverse Paraphrase Generation Using Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/2404.12596/) (2024)
- [Language as a Latent Sequence: deep latent variable models for semi-supervised paraphrase generation](https://mlsanity.com/paper/2301.02275/) (2023)
- [NeuroQL: A Neuro-Symbolic Language and Dataset for Inter-Subjective Reasoning](https://mlsanity.com/paper/2303.07146/) (2023)
- [Graph Relation Distillation for Efficient Biomedical Instance Segmentation](https://mlsanity.com/paper/2401.06370/) (2024)
- [Paraphrase Generation as Unsupervised Machine Translation](https://mlsanity.com/paper/2109.02950/) (2021)
- [Visual Information Guided Zero-Shot Paraphrase Generation](https://mlsanity.com/paper/2201.09107/) (2022)
- [Fine-Grained Emotional Paraphrasing along Emotion Gradients](https://mlsanity.com/paper/2212.03297/) (2022)
- [Action Controlled Paraphrasing](https://mlsanity.com/paper/2405.11277/) (2024)
- [Lexical Substitution is not Synonym Substitution: On the Importance of Producing Contextually Relevant Word Substitutes](https://mlsanity.com/paper/2502.04173/) (2025)
- [The power of text similarity in identifying AI-LLM paraphrased documents: The case of BBC news articles and ChatGPT](https://mlsanity.com/paper/2505.12405/) (2025)

## Abstract (arXiv)

Paraphrase generation is a longstanding important problem in natural language processing. In addition, recent progress in deep generative models has shown promising results on discrete latent variables for text generation. Inspired by variational autoencoders with discrete latent structures, in this work, we propose a latent bag of words (BOW) model for paraphrase generation. We ground the semantics of a discrete latent variable by the BOW from the target sentences. We use this latent variable to build a fully differentiable content planning and surface realization model. Specifically, we use source words to predict their neighbors and model the target BOW with a mixture of softmax. We use Gumbel top-k reparameterization to perform differentiable subset sampling from the predicted BOW distribution. We retrieve the sampled word embeddings and use them to augment the decoder and guide its generation search space. Our latent BOW model not only enhances the decoder, but also exhibits clear interpretability. We show the model interpretability with regard to (i) unsupervised learning of word neighbors (ii) the step-by-step generation procedure. Extensive experiments demonstrate the transparent and effective generation process of this model.footnoteOur code can be found at urlhttps://github.com/FranxYao/dgmlatentbow