mlsanityData to 7 Oct 2026

Generalized Discrete Diffusion from Snapshots

arXiv 2603.21342 ↗First posted 22 Mar 2026stat.ML cs.AI cs.CLcited by 3

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We introduce Generalized Discrete Diffusion from Snapshots (GDDS), a unified framework for discrete diffusion modeling that supports arbitrary noising processes over large discrete state spaces. Our formulation encompasses all existing discrete diffusion approaches, while allowing significantly greater flexibility in the choice of corruption dynamics. The forward noising process relies on uniformization and enables fast arbitrary corruption. For the reverse process, we derive a simple evidence lower bound (ELBO) based on snapshot latents, instead of the entire noising path, that allows efficient training of standard generative modeling architectures with clear probabilistic interpretation. Our experiments on large-vocabulary discrete generation tasks suggest that the proposed framework outperforms existing discrete diffusion methods in terms of training efficiency and generation quality, and beats autoregressive models for the first time at this scale. We provide the code along with a blog post on the project page : hrefhttps://oussamazekri.fr/gddshttps://oussamazekri.fr/gdds.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 43 of its 61 references that are in the map. First posted on arXiv 22 Mar 2026.

Builds on

43 of 61 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction4 Dec
  2. 2025Demystifying Diffusion Objectives: Reweighted Losses are Better Variational Bounds24 Nov
  3. 2025The Principles of Diffusion Models24 Oct
  4. 2025Why Masking Diffusion Works: Condition on the Jump Schedule for Improved Discrete Diffusion10 Jun
  5. 2024Simple Guidance Mechanisms for Discrete Diffusion Models13 Dec
  6. 2022Scalable Diffusion Models with Transformers19 Dec
  7. 2022A Continuous Time Framework for Discrete Denoising Models30 May
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  10. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  11. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2016Pointer Sentinel Mixture Modelscited by 2,992
  5. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  7. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  8. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  9. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  10. 2025Large Language Diffusion Modelscited by 872
  11. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  12. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
20 more
  1. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  2. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
  3. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  4. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  5. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  6. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 302
  7. 2021Autoregressive Diffusion Modelscited by 208
  8. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusionscited by 208
  9. 2025Video models are zero-shot learners and reasonerscited by 208
  10. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 204
  11. 2023Likelihood-Based Diffusion Language Modelscited by 147
  12. 2025The Diffusion Dualitycited by 124
  13. 2013Generalized Denoising Auto-Encoders as Generative Modelscited by 113
  14. 2025A Survey on Diffusion Language Modelscited by 75
  15. 2019Discrete Flows: Invertible Generative Models of Discrete Datacited by 68
  16. 2025Generalized Interpolating Discrete Diffusioncited by 68
  17. 2019Latent Normalizing Flows for Discrete Sequencescited by 66
  18. 2022Training and Inference on Any-Order Autoregressive Models the Right Waycited by 64
  19. 2023Bayesian Flow Networkscited by 62
  20. 2025Scaling Behavior of Discrete Diffusion Language Modelscited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Spherical Flows for Sampling Categorical Datacited by 4
  2. 2026Semantic DLM+: Improving Diffusion Language Models through Bias-variance Trade-off in Transition Kernel Designcited by 0
  3. 2026Subliminal Clocks: Latent Time Modelling in Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Forward-Learned Discrete Diffusion: Learning how to noise to denoise faster
  2. 2023Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces
  3. 2026From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Models
  4. 2026Discrete Diffusion Models: A Unified Framework from Tokenization to Generation
  5. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models
  6. 2026Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?
  7. 2025Diffusion Models: A Mathematical Introduction
  8. 2025Non-Asymptotic Convergence of Discrete Diffusion Models: Masked and Random Walk dynamics