# Factorising Meaning and Form for Intent-Preserving Paraphrasing

arXiv: https://arxiv.org/abs/2105.15053 · HTML: https://mlsanity.com/paper/2105.15053/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 31 May 2021.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Hierarchical Transformers for Multi-Document Summarization](https://mlsanity.com/paper/1905.13164/) (2019)
- [Fast Decoding in Sequence Models using Discrete Latent Variables](https://mlsanity.com/paper/1803.03382/) (2018)
- [Data Noising as Smoothing in Neural Network Language Models](https://mlsanity.com/paper/1703.02573/) (2017)
- [Extractive Opinion Summarization in Quantized Transformer Spaces](https://mlsanity.com/paper/2012.04443/) (2020)
- [Generating Sentences from Disentangled Syntactic and Semantic Spaces](https://mlsanity.com/paper/1907.05789/) (2019)
- [Neural Syntactic Preordering for Controlled Paraphrase Generation](https://mlsanity.com/paper/2005.02013/) (2020)
- [Theory and Experiments on Vector Quantized Autoencoders](https://mlsanity.com/paper/1805.11063/) (2018)
- [Syntax-guided Controlled Generation of Paraphrases](https://mlsanity.com/paper/2005.08417/) (2020)
- [Learning to Paraphrase for Question Answering](https://mlsanity.com/paper/1708.06022/) (2017)
- [Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs](https://mlsanity.com/paper/2101.10579/) (2021)
- [Controllable Paraphrase Generation with a Syntactic Exemplar](https://mlsanity.com/paper/1906.00565/) (2019)
- [ParaBank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translation](https://mlsanity.com/paper/1901.03644/) (2019)
- [A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations](https://mlsanity.com/paper/1904.01173/) (2019)
- [Adversarial Domain Adaptation for Duplicate Question Detection](https://mlsanity.com/paper/1809.02255/) (2018)
- [Paraphrase Generation with Latent Bag of Words](https://mlsanity.com/paper/2001.01941/) (2020)
- [Unsupervised Paraphrasing without Translation](https://mlsanity.com/paper/1905.12752/) (2019)

## Built on by (most cited first, 22 of 22)

- [RADAR: Robust AI-Text Detection via Adversarial Learning](https://mlsanity.com/paper/2307.03838/) (2023)
- [IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languages](https://mlsanity.com/paper/2203.05437/) (2022)
- [Quality Controlled Paraphrase Generation](https://mlsanity.com/paper/2203.10940/) (2022)
- [Novelty Controlled Paraphrase Generation with Retrieval Augmented Conditional Prompt Tuning](https://mlsanity.com/paper/2202.00535/) (2022)
- [Hierarchical Sketch Induction for Paraphrase Generation](https://mlsanity.com/paper/2203.03463/) (2022)
- [Improving Biomedical Information Retrieval with Neural Retrievers](https://mlsanity.com/paper/2201.07745/) (2022)
- [Unsupervised Extractive Opinion Summarization Using Sparse Coding](https://mlsanity.com/paper/2203.07921/) (2022)
- [On The Ingredients of an Effective Zero-shot Semantic Parser](https://mlsanity.com/paper/2110.08381/) (2021)
- [Monotonic Paraphrasing Improves Generalization of Language Model Prompting](https://mlsanity.com/paper/2403.16038/) (2024)
- [What Has Been Lost with Synthetic Evaluation?](https://mlsanity.com/paper/2505.22830/) (2025)
- [LAMPAT: Low-Rank Adaption for Multilingual Paraphrasing Using Adversarial Training](https://mlsanity.com/paper/2401.04348/) (2024)
- [Modeling Thousands of Human Annotators for Generalizable Text-to-Image Person Re-identification](https://mlsanity.com/paper/2503.09962/) (2025)
- [Principled Paraphrase Generation with Parallel Corpora](https://mlsanity.com/paper/2205.12213/) (2022)
- [Exploring Diversity in Back Translation for Low-Resource Machine Translation](https://mlsanity.com/paper/2206.00564/) (2022)
- [Exploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEs](https://mlsanity.com/paper/2205.05943/) (2022)
- [Weakly Supervised Reasoning by Neuro-Symbolic Approaches](https://mlsanity.com/paper/2309.13072/) (2023)
- [Learning to Adapt to Low-Resource Paraphrase Generation](https://mlsanity.com/paper/2412.17111/) (2024)
- [Investigating the use of Paraphrase Generation for Question Reformulation in the FRANK QA system](https://mlsanity.com/paper/2206.02737/) (2022)
- [Language as a Latent Sequence: deep latent variable models for semi-supervised paraphrase generation](https://mlsanity.com/paper/2301.02275/) (2023)
- [A Quality-based Syntactic Template Retriever for Syntactically-controlled Paraphrase Generation](https://mlsanity.com/paper/2310.13262/) (2023)
- [Linguistically-Controlled Paraphrase Generation](https://mlsanity.com/paper/2410.24199/) (2024)
- [Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncoder](https://mlsanity.com/paper/2506.20083/) (2025)

## Abstract (arXiv)

We propose a method for generating paraphrases of English questions that retain the original intent but use a different surface form. Our model combines a careful choice of training objective with a principled information bottleneck, to induce a latent encoding space that disentangles meaning and form. We train an encoder-decoder model to reconstruct a question from a paraphrase with the same meaning and an exemplar with the same surface form, leading to separated encoding spaces. We use a Vector-Quantized Variational Autoencoder to represent the surface form as a set of discrete latent variables, allowing us to use a classifier to select a different surface form at test time. Crucially, our method does not require access to an external source of target exemplars. Extensive experiments and a human evaluation show that we are able to generate paraphrases with a better tradeoff between semantic preservation and syntactic novelty compared to previous methods.