mlsanityData to 7 Oct 2026

A Reparameterized Discrete Diffusion Model for Text Generation

arXiv 2302.05737 ↗First posted 11 Feb 2023cs.CL cs.LGcited by 152

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

This work studies discrete diffusion probabilistic models with applications to natural language generation. We derive an alternative yet equivalent formulation of the sampling from discrete diffusion processes and leverage this insight to develop a family of reparameterized discrete diffusion models. The derived generic framework is highly flexible, offers a fresh perspective of the generation process in discrete diffusion models, and features more effective training and decoding techniques. We conduct extensive experiments to evaluate the text generation capability of our model, demonstrating significant improvements over existing diffusion models.

Key facts. Cited by 152 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 71 of its 99 references that are in the map. First posted on arXiv 11 Feb 2023.

Builds on

71 of 99 references.

Closest ancestors · 10

Same direction, newest first

  1. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers20 Dec
  2. 2022Latent Diffusion for Language Generation19 Dec
  3. 2022Continuous diffusion for categorical data28 Nov
  4. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov
  5. 2022Self-conditioned Embedding Diffusion for Text Generation8 Nov
  6. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control31 Oct
  7. 2022DiffusER: Discrete Diffusion via Edit-based Reconstruction30 Oct
  8. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct
  9. 2021Autoregressive Diffusion Models5 Oct
  10. 2021Beyond In-Place Corruption: Insertion and Deletion In Denoising Probabilistic Models16 Jul

Other references · 61

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2022Scalable Diffusion Models with Transformerscited by 5,931
  11. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  12. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
38 more
  1. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  2. 2016Pointer Sentinel Mixture Modelscited by 2,992
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  6. 2022Video Diffusion Modelscited by 2,280
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  9. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  10. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  11. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  12. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  13. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  14. 2022Diffusion Models in Vision: A Surveycited by 1,008
  15. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  16. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  17. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  18. 2016Sequence-Level Knowledge Distillationcited by 761
  19. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 666
  20. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  21. 2021Score-based Generative Modeling in Latent Spacecited by 634
  22. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  23. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  24. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 503
  25. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  26. 2017Non-Autoregressive Neural Machine Translationcited by 457
  27. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  28. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  29. 2022MAGVIT: Masked Generative Video Transformercited by 341
  30. 2022Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equationscited by 273
  31. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 205
  32. 2019Levenshtein Transformercited by 197
  33. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
  34. 2022Score-based Continuous-time Discrete Diffusion Modelscited by 167
  35. 2019Insertion Transformer: Flexible Sequence Generation via Insertion Operationscited by 123
  36. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  37. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  38. 2021Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codescited by 71

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 28; 2025: 63; 2026: 46.

152 papers.

Sort:

By area:

  1. 2025Large Language Diffusion Modelscited by 872
  2. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  3. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  4. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  5. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  6. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  7. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusionscited by 208
  8. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
  9. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  10. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  11. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  12. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
38 more
  1. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
  2. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
  3. 2024Diffusion Language Models Are Versatile Protein Learnerscited by 93
  4. 2024Energy-Based Diffusion Language Models for Text Generationcited by 86
  5. 2024Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generationcited by 85
  6. 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
  7. 2025Path Planning for Masked Diffusion Model Samplingcited by 79
  8. 2025A Survey on Diffusion Language Modelscited by 75
  9. 2024Think While You Generate: Discrete Diffusion with Planned Denoisingcited by 61
  10. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  11. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  12. 2024DPLM-2: A Multimodal Diffusion Protein Language Modelcited by 59
  13. 2023CDDM: Channel Denoising Diffusion Models for Wireless Semantic Communicationscited by 58
  14. 2024Informed Correctors for Discrete Diffusion Modelscited by 57
  15. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 55
  16. 2025Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Modelscited by 53
  17. 2024How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Frameworkcited by 48
  18. 2025GenMol: A Drug Discovery Generalist with Discrete Diffusioncited by 47
  19. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  20. 2025Theoretical Benefit and Limitation of Diffusion Language Modelcited by 46
  21. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  22. 2024Convergence Analysis of Discrete Diffusion Model: Exact Implementation through Uniformizationcited by 44
  23. 2023Diffusion Models for Non-autoregressive Text Generation: A Surveycited by 43
  24. 2023Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuningcited by 39
  25. 2025Fine-Tuning Masked Diffusion for Provable Self-Correctioncited by 39
  26. 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Modelscited by 38
  27. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Modelscited by 37
  28. 2023DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Modelscited by 36
  29. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 32
  30. 2025Attention Is All You Need for KV Cache in Diffusion LLMscited by 32
  31. 2025TESS 2: A Large-Scale Generalist Diffusion Language Modelcited by 30
  32. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 27
  33. 2025Target Concrete Score Matching: A Holistic Framework for Discrete Diffusioncited by 25
  34. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoningcited by 25
  35. 2024Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysiscited by 24
  36. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Modelscited by 22
  37. 2023A Cheaper and Better Diffusion Language Model with Soft-Masked Noisecited by 21
  38. 2023Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generationcited by 21

50 most cited of 152 All 152 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Discrete Diffusion Models for Language Generation
  2. 2025On the Role of Discreteness in Diffusion LLMs
  3. 2023Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!
  4. 2023InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generation
  5. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise
  6. 2025State Fourier Diffusion Language Model (SFDLM): A Scalable, Novel Iterative Approach to Language Modeling
  7. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
  8. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusion