mlsanityData to 7 Oct 2026

Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time

arXiv 2312.09193 ↗First posted 14 Dec 2023cs.LG cs.AI stat.MLcited by 20

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Discrete diffusion models have emerged as powerful tools for high-quality data generation. Despite their success in discrete spaces, such as text generation tasks, the acceleration of discrete diffusion models remains under-explored. In this paper, we propose discrete non-Markov diffusion models (DNDM), which naturally induce the predetermined transition time set. This enables a training-free sampling algorithm that significantly reduces the number of function evaluations (i.e., calls to the neural network), making the sampling process much faster. Furthermore, we study the transition from finite to infinite step sampling, offering new insights into bridging the gap between discrete and continuous-time processes for discrete diffusion models. Extensive experiments on natural language generation and machine translation tasks demonstrate the superior performance of our method in terms of both generation speed and sample quality compared to existing methods for discrete diffusion models.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 44 of its 56 references that are in the map. First posted on arXiv 14 Dec 2023.

Builds on

44 of 56 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning23 Aug
  2. 2023A Reparameterized Discrete Diffusion Model for Text Generation11 Feb
  3. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov
  4. 2022DiffusER: Discrete Diffusion via Edit-based Reconstruction30 Oct
  5. 2021Autoregressive Diffusion Models5 Oct

Other references · 39

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  9. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  10. 2023Consistency Modelscited by 1,778
  11. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  12. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
27 more
  1. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  2. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  3. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  4. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  5. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 695
  6. 2021Score-based Generative Modeling in Latent Spacecited by 634
  7. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  8. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  9. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 526
  10. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  11. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Modelscited by 331
  12. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contractioncited by 306
  13. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusioncited by 230
  14. 2021Gotta Go Fast When Generating Data with Score-Based Modelscited by 218
  15. 2021On Fast Sampling of Diffusion Probabilistic Modelscited by 170
  16. 2022Score-based Continuous-time Discrete Diffusion Modelscited by 167
  17. 2023Protein Design with Guided Discrete Diffusioncited by 154
  18. 2021Learning to Efficiently Sample from Diffusion Probabilistic Modelscited by 129
  19. 2022GENIE: Higher-Order Denoising Diffusion Solverscited by 127
  20. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  21. 2021Noise Estimation for Generative Diffusion Modelscited by 94
  22. 2021Non Gaussian Denoising Diffusion Modelscited by 43
  23. 2013Deep Generative Stochastic Networks Trainable by Backpropcited by 36
  24. 2017Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Netcited by 30
  25. 2023Synthesizing Mixed-type Electronic Health Records using Diffusion Modelscited by 17
  26. 2017Learning to Generate Samples from Noise through Infusion Trainingcited by 15
  27. 2015GSNs : Generative Stochastic Networkscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 6; 2026: 8.

20 papers.

Sort:

By area:

  1. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  2. 2024Think While You Generate: Discrete Diffusion with Planned Denoisingcited by 61
  3. 2024Diffusion Bridge Implicit Modelscited by 47
  4. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  5. 2024textitJump Your Steps: Optimizing Sampling Schedule of Discrete Diffusion Modelscited by 40
  6. 2024Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysiscited by 24
  7. 2025Discrete Diffusion Models: Novel Analysis and New Sampler Guaranteescited by 20
  8. 2024Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplerscited by 14
  9. 2026Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulationcited by 11
  10. 2025Learnable Sampler Distillation for Discrete Diffusion Modelscited by 8
  11. 2025Non-Markovian Discrete Diffusion with Causal Language Modelscited by 5
  12. 2025Generative Modeling with Bayesian Sample Inferencecited by 4
8 more
  1. 2026Unrestrained Simplex Denoising for Discrete Data. A Non-Markovian Approach Applied to Graph Generationcited by 2
  2. 2025Deterministic Discrete Denoisingcited by 1
  3. 2026Accelerating Discrete Diffusion Models with Parallel-In-Time Samplingcited by 1
  4. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniquescited by 1
  5. 2026Unlocking High-Fidelity Molecular Generation from Mass Spectra via Dual-Stream Line Graph Diffusioncited by 0
  6. 2026Efficient One-to-Many Translation with Joint Multi-Stream Diffusioncited by 0
  7. 2026Structured Residual Connectivity Matters for Diffusion Transformerscited by 0
  8. 2026Rethinking Soft Tokens for Parallel Decoding in Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Discrete Diffusion Models for Language Generation
  2. 2025FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models
  3. 2023Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!
  4. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling
  5. 2025The Diffusion Duality
  6. 2025Accelerated Diffusion Models via Speculative Sampling
  7. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Models
  8. 2025Latent-Augmented Discrete Diffusion Models