mlsanityData to 13 Oct 2026

Simple Guidance Mechanisms for Discrete Diffusion Models

arXiv 2412.10193 ↗First posted 13 Dec 2024cs.LGcited by 151

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models for continuous data gained widespread adoption owing to their high quality generation and control mechanisms. However, controllable diffusion on discrete data faces challenges given that continuous guidance methods do not directly apply to discrete diffusion. Here, we provide a straightforward derivation of classifier-free and classifier-based guidance for discrete diffusion, as well as a new class of diffusion models that leverage uniform noise and that are more guidable because they can continuously edit their outputs. We improve the quality of these models with a novel continuous-time variational lower bound that yields state-of-the-art performance, especially in settings involving guidance or fast generation. Empirically, we demonstrate that our guidance mechanisms combined with uniform noise diffusion improve controllable generation relative to autoregressive and diffusion baselines on several discrete data domains, including genomic sequences, small molecule design, and discretized image generation.

Key facts. Cited by 151 papers in the map (data through 13 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 52 of its 68 references that are in the map. First posted on arXiv 13 Dec 2024.

Builds on

52 of 68 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023Diffusion Models With Learned Adaptive Noise20 Dec
  2. 2023InfoDiffusion: Representation Learning Using Information Maximizing Diffusion Models14 Jun
  3. 2022Scalable Diffusion Models with Transformers19 Dec
  4. 2022Score-based Continuous-time Discrete Diffusion Models30 Nov
  5. 2022Classifier-Free Diffusion Guidance26 Jul
  6. 2022A Continuous Time Framework for Discrete Denoising Models30 May
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  8. 2021Variational Diffusion Models1 Jul
  9. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  10. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 42

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,656
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,023
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,339
  4. 2022Flow Matching for Generative Modelingcited by 5,542
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,658
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,628
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,077
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,605
  9. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,019
  10. 2016Improved Techniques for Training GANscited by 2,946
  11. 2015Character-level Convolutional Networks for Text Classificationcited by 2,319
  12. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,835
30 more
  1. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,012
  2. 2024Simple and Effective Masked Diffusion Language Modelscited by 816
  3. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 797
  4. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 614
  5. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 570
  6. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 509
  7. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 427
  8. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 377
  9. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 375
  10. 2024Discrete Flow Matchingcited by 309
  11. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 306
  12. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 299
  13. 2018Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Predictioncited by 225
  14. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
  15. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 185
  16. 2023Protein Design with Guided Discrete Diffusioncited by 156
  17. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  18. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 146
  19. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 128
  20. 2024Dirichlet Flow Matching with Applications to DNA Sequence Designcited by 113
  21. 2022Latent Diffusion for Language Generationcited by 111
  22. 2024Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modelingcited by 110
  23. 2021Oops I Took A Gradient: Scalable Sampling for Discrete Distributionscited by 92
  24. 2023Stay on topic with Classifier-Free Guidancecited by 90
  25. 2019Discrete Flows: Invertible Generative Models of Discrete Datacited by 68
  26. 2019Latent Normalizing Flows for Discrete Sequencescited by 66
  27. 2022Training and Inference on Any-Order Autoregressive Models the Right Waycited by 64
  28. 2023Bayesian Flow Networkscited by 63
  29. 2021OmniNet: Omnidirectional Representations from Transformerscited by 23
  30. 2023Classifier-free graph diffusion for molecular property targetingcited by 14

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 63; 2026: 88.

151 papers.

Sort:

By area:

  1. 2025Large Language Diffusion Modelscited by 888
  2. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 396
  3. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusionscited by 211
  4. 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 185
  5. 2025The Diffusion Dualitycited by 130
  6. 2025Path Planning for Masked Diffusion Model Samplingcited by 79
  7. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  8. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  9. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Modelscited by 46
  10. 2026Categorical Flow Mapscited by 42
  11. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMscited by 34
  12. 2025Scaling Behavior of Discrete Diffusion Language Modelscited by 34
38 more
  1. 2026LangFlow: Continuous Diffusion Rivals Discrete in Language Modelingcited by 33
  2. 2025CANDI: Hybrid Discrete-Continuous Diffusion Modelscited by 31
  3. 2025d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimationcited by 27
  4. 2025Target Concrete Score Matching: A Holistic Framework for Discrete Diffusioncited by 25
  5. 2025Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modelingcited by 24
  6. 2026Scaling Beyond Masked Diffusion Language Modelscited by 24
  7. 2025Absorb and Converge: Provable Convergence Guarantee for Absorbing Discrete Diffusion Modelscited by 20
  8. 2025MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Controlcited by 20
  9. 2025Discrete Diffusion Models: Novel Analysis and New Sampler Guaranteescited by 20
  10. 2025Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spacescited by 19
  11. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 18
  12. 2025CtrlDiff: Boosting Large Diffusion Language Models with Dynamic Block Prediction and Controllable Generationcited by 17
  13. 2025Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Stepcited by 17
  14. 2025Constrained Discrete Diffusioncited by 16
  15. 2025Feedback Guidance of Diffusion Modelscited by 16
  16. 2025Ultra-Fast Language Generation via Discrete Diffusion Divergence Instructcited by 16
  17. 2026Continuous Diffusion Scales Competitively with Discrete Diffusion for Languagecited by 16
  18. 2025Inference-Time Scaling of Discrete Diffusion Models via Importance Weighting and Optimal Proposal Designcited by 14
  19. 2025Planner Aware Path Learning in Diffusion Language Models Trainingcited by 14
  20. 2025TR2-D2: Tree Search Guided Trajectory-Aware Fine-Tuning for Discrete Diffusioncited by 14
  21. 2026dLLM: Simple Diffusion Language Modelingcited by 14
  22. 2025ReDi: Rectified Discrete Flowcited by 13
  23. 2025Fast and Accurate Causal Parallel Decoding using Jacobi Forcingcited by 12
  24. 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Modelscited by 12
  25. 2026The Diffusion Duality, Chapter II: Psi-Samplerscited by 12
  26. 2026Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulationcited by 12
  27. 2025Improving Classifier-Free Guidance in Masked Diffusion: Low-Dim Theoretical Insights with High-Dim Impactcited by 11
  28. 2025Corrective Diffusion Language Modelscited by 11
  29. 2025Adaptive Diffusion Guidance via Stochastic Optimal Controlcited by 10
  30. 2025Neuro-Symbolic Generative Diffusion Models for Physically Grounded, Robust, and Safe Generationcited by 10
  31. 2026Adaptation to Intrinsic Dependence in Diffusion Language Modelscited by 9
  32. 2026IDLM: Inverse-distilled Diffusion Language Modelscited by 8
  33. 2026DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmaskingcited by 8
  34. 2025Steering Generative Models with Experimental Data for Protein Fitness Optimizationcited by 7
  35. 2025Inference-Time Scaling of Diffusion Language Models via Trajectory Refinementcited by 7
  36. 2025Next Semantic Scale Prediction via Hierarchical Diffusion Language Modelscited by 7
  37. 2025Loopholing Discrete Diffusion: Deterministic Bypass of the Sampling Wallcited by 7
  38. 2025Towards a unified framework for guided diffusion modelscited by 7

50 most cited of 151 All 151 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Discriminator Guidance for Autoregressive Diffusion Models
  2. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decoding
  3. 2023Elucidating The Design Space of Classifier-Guided Diffusion Generation
  4. 2023Bridging the Gap: Addressing Discrepancies in Diffusion Model Training for Classifier-Free Guidance
  5. 2025Stochastic Self-Guidance for Training-Free Enhancement of Diffusion Models
  6. 2025Why Masking Diffusion Works: Condition on the Jump Schedule for Improved Discrete Diffusion
  7. 2025Efficient Controllable Diffusion via Optimal Classifier Guidance
  8. 2025Token Perturbation Guidance for Diffusion Models