mlsanityData to 7 Oct 2026

Continuous diffusion for categorical data

arXiv 2211.15089 ↗First posted 28 Nov 2022cs.CL cs.LGcited by 202

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion models have quickly become the go-to paradigm for generative modelling of perceptual signals (such as images and sound) through iterative refinement. Their success hinges on the fact that the underlying physical phenomena are continuous. For inherently discrete and categorical data such as language, various diffusion-inspired alternatives have been proposed. However, the continuous nature of diffusion models conveys many benefits, and in this work we endeavour to preserve it. We propose CDCD, a framework for modelling categorical data with diffusion models that are continuous both in time and input space. We demonstrate its efficacy on several language modelling tasks.

Key facts. Cited by 202 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 78 of its 110 references that are in the map. First posted on arXiv 28 Nov 2022.

Builds on

78 of 110 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Self-conditioned Embedding Diffusion for Text Generation8 Nov
  2. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control31 Oct
  3. 2022DiffusER: Discrete Diffusion via Edit-based Reconstruction30 Oct
  4. 2021Autoregressive Diffusion Models5 Oct

Other references · 74

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  12. 2022Classifier-Free Diffusion Guidancecited by 5,717
38 more
  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  6. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  8. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  9. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  10. 2022Video Diffusion Modelscited by 2,280
  11. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  12. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  13. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  14. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  15. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  16. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  17. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  18. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  19. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  20. 2021Variational Diffusion Modelscited by 1,145
  21. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  22. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  23. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  24. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  25. 2020On Layer Normalization in the Transformer Architecturecited by 906
  26. 2016Sequence-Level Knowledge Distillationcited by 761
  27. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
  28. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  29. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  30. 2019Neural Spline Flowscited by 590
  31. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  32. 2020Jukebox: A Generative Model for Musiccited by 535
  33. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  34. 2017Non-Autoregressive Neural Machine Translationcited by 457
  35. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  36. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  37. 2016Using the Output Embedding to Improve Language Modelscited by 266
  38. 2022On Neural Differential Equationscited by 264

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 28; 2024: 34; 2025: 59; 2026: 77.

202 papers.

Sort:

By area:

  1. 2025Large Language Diffusion Modelscited by 872
  2. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  3. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  4. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  5. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  6. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  7. 2024Discrete Flow Matchingcited by 301
  8. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  9. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 293
  10. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  11. 2023DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimizationcited by 216
  12. 2024Scaling up Masked Diffusion Models on Textcited by 195
38 more
  1. 2022Scalable Adaptive Computation for Iterative Generationcited by 190
  2. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 173
  3. 2023Protein Design with Guided Discrete Diffusioncited by 154
  4. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  5. 2023Likelihood-Based Diffusion Language Modelscited by 147
  6. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  7. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  8. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  9. 2025The Diffusion Dualitycited by 124
  10. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 113
  11. 2022Latent Diffusion for Language Generationcited by 110
  12. 2024Dirichlet Flow Matching with Applications to DNA Sequence Designcited by 110
  13. 2024Diffusion Language Models Are Versatile Protein Learnerscited by 93
  14. 2023DiffuRec: A Diffusion Model for Sequential Recommendationcited by 91
  15. 2024Energy-Based Diffusion Language Models for Text Generationcited by 86
  16. 2023Reflected Diffusion Modelscited by 81
  17. 2024Variational Flow Matching for Graph Generationcited by 78
  18. 2023SODA: Bottleneck Diffusion Models for Representation Learningcited by 76
  19. 2024Dual Diffusion for Unified Image Generation and Understandingcited by 76
  20. 2025A Survey on Diffusion Language Modelscited by 75
  21. 2025A Survey on Latent Reasoningcited by 71
  22. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 70
  23. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformerscited by 63
  24. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusioncited by 63
  25. 2023Bayesian Flow Networkscited by 62
  26. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  27. 2025LLaDA-MoE: A Sparse MoE Diffusion Language Modelcited by 54
  28. 2024Mixed Continuous and Categorical Flow Matching for 3D De Novo Molecule Generationcited by 53
  29. 2024Discrete Copula Diffusioncited by 53
  30. 2023DINOISER: Diffused Conditional Sequence Learning by Manipulating Noisescited by 50
  31. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  32. 2023Closed-Form Diffusion Modelscited by 46
  33. 2025Theoretical Benefit and Limitation of Diffusion Language Modelcited by 46
  34. 2023Diffusion Models for Non-autoregressive Text Generation: A Surveycited by 43
  35. 2023Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuningcited by 39
  36. 2024CleanDiffuser: An Easy-to-use Modularized Library for Diffusion Models in Decision Makingcited by 38
  37. 2026Categorical Flow Mapscited by 37
  38. 2024Time Series Diffusion in the Frequency Domaincited by 35

50 most cited of 202 All 202 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Optimality of FSQ Tokens for Continuous Diffusion for Categorical Data with Application to Text-to-Speech
  2. 2024Discrete Modeling via Boundary Conditional Diffusion Processes
  3. 2024Improving Discrete Diffusion Models via Structured Preferential Generation
  4. 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
  5. 2026Consistent Diffusion Language Models
  6. 2026Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling
  7. 2026Jumping up and down: Denoiser diffusion models for discrete ordinal data
  8. 2026Generative Frontiers: Why Evaluation Matters for Diffusion Language Models