# DiffusER: Discrete Diffusion via Edit-based Reconstruction

arXiv: https://arxiv.org/abs/2210.16886 · HTML: https://mlsanity.com/paper/2210.16886/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Placed in Discrete and text diffusion, one of 14 directions in Diffusion models.
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 30 Oct 2022.

## Placed in

- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise](https://mlsanity.com/paper/2208.09392/) (2022)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Levenshtein Transformer](https://mlsanity.com/paper/1905.11006/) (2019)
- [Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement](https://mlsanity.com/paper/1802.06901/) (2018)
- [GECToR -- Grammatical Error Correction: Tag, Not Rewrite](https://mlsanity.com/paper/2005.12592/) (2020)
- [CM3: A Causal Masked Multimodal Model of the Internet](https://mlsanity.com/paper/2201.07520/) (2022)
- [Understanding Knowledge Distillation in Non-autoregressive Machine Translation](https://mlsanity.com/paper/1911.02727/) (2019)
- [Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation](https://mlsanity.com/paper/2006.10369/) (2020)
- [Step-unrolled Denoising Autoencoders for Text Generation](https://mlsanity.com/paper/2112.06749/) (2021)
- [Encode, Tag, Realize: High-Precision Text Editing](https://mlsanity.com/paper/1909.01187/) (2019)
- [Non-Autoregressive Machine Translation with Latent Alignments](https://mlsanity.com/paper/2004.07437/) (2020)
- [Generating Sentences by Editing Prototypes](https://mlsanity.com/paper/1709.08878/) (2017)
- [Politeness Transfer: A Tag and Generate Approach](https://mlsanity.com/paper/2004.14257/) (2020)
- [LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2105.08206/) (2021)
- [Learning to Represent Edits](https://mlsanity.com/paper/1810.13337/) (2018)
- [Learning to Model Editing Processes](https://mlsanity.com/paper/2205.12374/) (2022)
- [Variational Inference for Learning Representations of Natural Language Edits](https://mlsanity.com/paper/2004.09143/) (2020)

## Built on by (most cited first, 30 of 41)

- [Continuous diffusion for categorical data](https://mlsanity.com/paper/2211.15089/) (2022)
- [Protein Design with Guided Discrete Diffusion](https://mlsanity.com/paper/2305.20009/) (2023)
- [A Reparameterized Discrete Diffusion Model for Text Generation](https://mlsanity.com/paper/2302.05737/) (2023)
- [LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning](https://mlsanity.com/paper/2505.16933/) (2025)
- [Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning](https://mlsanity.com/paper/2410.14157/) (2024)
- [Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise](https://mlsanity.com/paper/2212.11685/) (2022)
- [TESS: Text-to-Text Self-Conditioned Simplex Diffusion](https://mlsanity.com/paper/2305.08379/) (2023)
- [SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers](https://mlsanity.com/paper/2212.10325/) (2022)
- [Edit Flows: Flow Matching with Edit Operations](https://mlsanity.com/paper/2506.09018/) (2025)
- [Empowering Diffusion Models on the Embedding Space for Text Generation](https://mlsanity.com/paper/2212.09412/) (2022)
- [A Cheaper and Better Diffusion Language Model with Soft-Masked Noise](https://mlsanity.com/paper/2304.04746/) (2023)
- [Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time](https://mlsanity.com/paper/2312.09193/) (2023)
- [DLM-One: Diffusion Language Models for One-Step Sequence Generation](https://mlsanity.com/paper/2506.00290/) (2025)
- [OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flows](https://mlsanity.com/paper/2510.03506/) (2025)
- [ScanDL: A Diffusion Model for Generating Synthetic Scanpaths on Texts](https://mlsanity.com/paper/2310.15587/) (2023)
- [Glauber Generative Model: Discrete Diffusion Models via Binary Classification](https://mlsanity.com/paper/2405.17035/) (2024)
- [Text-guided multi-property molecular optimization with a diffusion language model](https://mlsanity.com/paper/2410.13597/) (2024)
- [LaDiC: Are Diffusion Models Really Inferior to Autoregressive Counterparts for Image-to-Text Generation?](https://mlsanity.com/paper/2404.10763/) (2024)
- [Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning](https://mlsanity.com/paper/2509.15188/) (2025)
- [David helps Goliath: Inference-Time Collaboration Between Small Specialized and Large General Diffusion LMs](https://mlsanity.com/paper/2305.14771/) (2023)
- [DiffuDetox: A Mixed Diffusion Model for Text Detoxification](https://mlsanity.com/paper/2306.08505/) (2023)
- [InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generation](https://mlsanity.com/paper/2310.11976/) (2023)
- [Guided Discrete Diffusion for Electronic Health Record Generation](https://mlsanity.com/paper/2404.12314/) (2024)
- [Towards GUI Agents: Vision-Language Diffusion Models for GUI Grounding](https://mlsanity.com/paper/2603.26211/) (2026)
- [Anomaly Detection in Networks via Score-Based Generative Models](https://mlsanity.com/paper/2306.15324/) (2023)
- [Let's Rectify Step by Step: Improving Aspect-based Sentiment Analysis with Diffusion Models](https://mlsanity.com/paper/2402.15289/) (2024)
- [Meta-DiffuB: A Contextualized Sequence-to-Sequence Text Diffusion Model with Meta-Exploration](https://mlsanity.com/paper/2410.13201/) (2024)
- [Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction](https://mlsanity.com/paper/2512.05092/) (2025)
- [Edit-Based Refinement for Parallel Masked Diffusion Language Models](https://mlsanity.com/paper/2605.09603/) (2026)
- [AnchorDiff: Topology-Aware Masked Diffusion with Confidence-based Rewriting for Radiology Report Generation](https://mlsanity.com/paper/2605.17071/) (2026)

## Abstract (arXiv)

In text generation, models that generate text from scratch one token at a time are currently the dominant paradigm. Despite being performant, these models lack the ability to revise existing text, which limits their usability in many practical scenarios. We look to address this, with DiffusER (Diffusion via Edit-based Reconstruction), a new edit-based generative model for text based on denoising diffusion models -- a class of models that use a Markov chain of denoising steps to incrementally generate data. DiffusER is not only a strong generative model in general, rivalling autoregressive models on several tasks spanning machine translation, summarization, and style transfer; it can also perform other varieties of generation that standard autoregressive models are not well-suited for. For instance, we demonstrate that DiffusER makes it possible for a user to condition generation on a prototype, or an incomplete sequence, and continue revising based on previous edit steps.