# Generating Sentences by Editing Prototypes

arXiv: https://arxiv.org/abs/1709.08878 · HTML: https://mlsanity.com/paper/1709.08878/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2017.

## Built on by (most cited first, 30 of 88)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics](https://mlsanity.com/paper/2202.11705/) (2022)
- [Foundation Models and Fair Use](https://mlsanity.com/paper/2303.15715/) (2023)
- [Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning](https://mlsanity.com/paper/2106.02584/) (2021)
- [A Survey on Data Augmentation for Text Classification](https://mlsanity.com/paper/2107.03158/) (2021)
- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://mlsanity.com/paper/2106.07499/) (2021)
- [Efficient Nearest Neighbor Language Models](https://mlsanity.com/paper/2109.04212/) (2021)
- [Adaptive Semiparametric Language Models](https://mlsanity.com/paper/2102.02557/) (2021)
- [Aligning LLM Agents by Learning Latent Preference from User Edits](https://mlsanity.com/paper/2404.15269/) (2024)
- [Neural Machine Translation with Monolingual Translation Memory](https://mlsanity.com/paper/2105.11269/) (2021)
- [Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval](https://mlsanity.com/paper/2201.12431/) (2022)
- [kNN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference](https://mlsanity.com/paper/2303.13824/) (2023)
- [CoditT5: Pretraining for Source Code and Natural Language Editing](https://mlsanity.com/paper/2208.05446/) (2022)
- [DiffusER: Discrete Diffusion via Edit-based Reconstruction](https://mlsanity.com/paper/2210.16886/) (2022)
- [Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey](https://mlsanity.com/paper/2201.12438/) (2022)
- [What's in a Summary? Laying the Groundwork for Advances in Hospital-Course Summarization](https://mlsanity.com/paper/2105.00816/) (2021)
- [Controllable Semantic Parsing via Retrieval Augmentation](https://mlsanity.com/paper/2110.08458/) (2021)
- [StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer](https://mlsanity.com/paper/2104.05196/) (2021)
- [Why do Nearest Neighbor Language Models Work?](https://mlsanity.com/paper/2301.02828/) (2023)
- [PoET: A generative model of protein families as sequences-of-sequences](https://mlsanity.com/paper/2306.06156/) (2023)
- [Weaver: Foundation Models for Creative Writing](https://mlsanity.com/paper/2401.17268/) (2024)
- [Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Design](https://mlsanity.com/paper/2502.14944/) (2025)
- [AlphaD3M: Machine Learning Pipeline Synthesis](https://mlsanity.com/paper/2111.02508/) (2021)
- [Beyond In-Place Corruption: Insertion and Deletion In Denoising Probabilistic Models](https://mlsanity.com/paper/2107.07675/) (2021)
- [Few-Shot Table-to-Text Generation with Prototype Memory](https://mlsanity.com/paper/2108.12516/) (2021)
- [Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings](https://mlsanity.com/paper/2110.07385/) (2021)

## Abstract (arXiv)

We propose a new generative model of sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional models that generate from scratch either left-to-right or by first sampling a latent sentence vector, our prototype-then-edit model improves perplexity on language modeling and generates higher quality outputs according to human evaluation. Furthermore, the model gives rise to a latent edit vector that captures interpretable semantics such as sentence similarity and sentence-level analogies.