mlsanityData to 7 Oct 2026

DiffusER: Discrete Diffusion via Edit-based Reconstruction

arXiv 2210.16886 ↗First posted 30 Oct 2022cs.CL cs.LGcited by 41

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

In text generation, models that generate text from scratch one token at a time are currently the dominant paradigm. Despite being performant, these models lack the ability to revise existing text, which limits their usability in many practical scenarios. We look to address this, with DiffusER (Diffusion via Edit-based Reconstruction), a new edit-based generative model for text based on denoising diffusion models -- a class of models that use a Markov chain of denoising steps to incrementally generate data. DiffusER is not only a strong generative model in general, rivalling autoregressive models on several tasks spanning machine translation, summarization, and style transfer; it can also perform other varieties of generation that standard autoregressive models are not well-suited for. For instance, we demonstrate that DiffusER makes it possible for a user to condition generation on a prototype, or an incomplete sequence, and continue revising based on previous edit steps.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 28 of its 34 references that are in the map. First posted on arXiv 30 Oct 2022.

Builds on

28 of 34 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  8. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  9. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  10. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  11. 2017Non-Autoregressive Neural Machine Translationcited by 457
  12. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noisecited by 275
16 more
  1. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 246
  2. 2019Levenshtein Transformercited by 197
  3. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
  4. 2020GECToR -- Grammatical Error Correction: Tag, Not Rewritecited by 177
  5. 2022CM3: A Causal Masked Multimodal Model of the Internetcited by 142
  6. 2019Understanding Knowledge Distillation in Non-autoregressive Machine Translationcited by 121
  7. 2020Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translationcited by 104
  8. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  9. 2019Encode, Tag, Realize: High-Precision Text Editingcited by 99
  10. 2020Non-Autoregressive Machine Translation with Latent Alignmentscited by 96
  11. 2017Generating Sentences by Editing Prototypescited by 88
  12. 2020Politeness Transfer: A Tag and Generate Approachcited by 78
  13. 2021LEWIS: Levenshtein Editing for Unsupervised Text Style Transfercited by 40
  14. 2018Learning to Represent Editscited by 34
  15. 2022Learning to Model Editing Processescited by 30
  16. 2020Variational Inference for Learning Representations of Natural Language Editscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 12; 2024: 7; 2025: 6; 2026: 12.

41 papers.

Sort:

By area:

  1. 2022Continuous diffusion for categorical datacited by 202
  2. 2023Protein Design with Guided Discrete Diffusioncited by 154
  3. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  4. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  5. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  6. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoisecited by 79
  7. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformerscited by 63
  8. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusioncited by 63
  9. 2025Edit Flows: Flow Matching with Edit Operationscited by 53
  10. 2022Empowering Diffusion Models on the Embedding Space for Text Generationcited by 40
  11. 2023A Cheaper and Better Diffusion Language Model with Soft-Masked Noisecited by 21
  12. 2023Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Timecited by 20
29 more
  1. 2025DLM-One: Diffusion Language Models for One-Step Sequence Generationcited by 19
  2. 2025OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flowscited by 17
  3. 2023ScanDL: A Diffusion Model for Generating Synthetic Scanpaths on Textscited by 13
  4. 2024Glauber Generative Model: Discrete Diffusion Models via Binary Classificationcited by 11
  5. 2024Text-guided multi-property molecular optimization with a diffusion language modelcited by 11
  6. 2024LaDiC: Are Diffusion Models Really Inferior to Autoregressive Counterparts for Image-to-Text Generation?cited by 9
  7. 2025Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuningcited by 8
  8. 2023David helps Goliath: Inference-Time Collaboration Between Small Specialized and Large General Diffusion LMscited by 6
  9. 2023DiffuDetox: A Mixed Diffusion Model for Text Detoxificationcited by 6
  10. 2023InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generationcited by 5
  11. 2024Guided Discrete Diffusion for Electronic Health Record Generationcited by 5
  12. 2023Anomaly Detection in Networks via Score-Based Generative Modelscited by 2
  13. 2024Let's Rectify Step by Step: Improving Aspect-based Sentiment Analysis with Diffusion Modelscited by 2
  14. 2024Meta-DiffuB: A Contextualized Sequence-to-Sequence Text Diffusion Model with Meta-Explorationcited by 2
  15. 2026Towards GUI Agents: Vision-Language Diffusion Models for GUI Groundingcited by 2
  16. 2026AnchorDiff: Topology-Aware Masked Diffusion with Confidence-based Rewriting for Radiology Report Generationcited by 2
  17. 2023GlyphDiffusion: Text Generation as Image Generationcited by 1
  18. 2023DiffuSIA: A Spiral Interaction Architecture for Encoder-Decoder Text Diffusioncited by 1
  19. 2025Foundations of Diffusion Models in General State Spaces: A Self-Contained Introductioncited by 1
  20. 2026Edit-Based Refinement for Parallel Masked Diffusion Language Modelscited by 1
  21. 2026Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attentioncited by 1
  22. 2026NI Sampling: Accelerating Discrete Diffusion Sampling by Token Order Optimizationcited by 0
  23. 2026EditSR: Enhancing Neural Symbolic Regression via Edit-based Rectificationcited by 0
  24. 2026Forward-Free Diffusion Language Models with BPTT-Free Looped Refinementcited by 0
  25. 2026A Continuous-Time Markov Chain Framework for Insertion Language Modelscited by 0
  26. 2026StereoDiffuer: Diffusion-based Progressive Geometry Modeling with Saliency Attention Perception for Stereo Matchingcited by 0
  27. 2026Spectral Feedback for Test-Time Alignment of Protein Diffusion Modelscited by 0
  28. 2026How Should Diffusion Language Models Edit Code?cited by 0
  29. 2026SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
  2. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
  3. 2023PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model
  4. 2025Unifying Continuous and Discrete Text Diffusion with Non-simultaneous Diffusion Processes
  5. 2025Discrete Diffusion Models for Language Generation
  6. 2023Diffusion Models for Non-autoregressive Text Generation: A Survey
  7. 2022Self-conditioned Embedding Diffusion for Text Generation
  8. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models