mlsanityData to 14 Oct 2026

Constrained Discrete Diffusion

arXiv 2503.09790 ↗First posted 12 Mar 2025cs.CL cs.LGcited by 17

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Discrete diffusion models are a class of generative models that construct sequences by progressively denoising samples from a categorical noise distribution. Beyond their rapidly growing ability to generate coherent natural language, these models present a new and important opportunity to enforce sequence-level constraints, a capability that current autoregressive models cannot natively provide. This paper capitalizes on this opportunity by introducing Constrained Discrete Diffusion (CDD), a novel integration of differentiable constraint optimization within the diffusion process to ensure adherence to constraints, logic rules, or safety requirements for generated sequences. Unlike conventional text generators that often rely on post-hoc filtering or model retraining for controllable generation, CDD directly imposes constraints into the discrete diffusion sampling process, resulting in a training-free and effective approach. Experiments in toxicity-controlled text generation, property-constrained molecule design, and instruction-constrained text completion demonstrate that CDD achieves zero constraint violations in a diverse array of tasks while preserving fluency, novelty, and coherence while outperforming autoregressive and existing discrete diffusion approaches.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 50 of its 66 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

50 of 66 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning18 Oct
  2. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  3. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  4. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  5. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 45

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2022Classifier-Free Diffusion Guidancecited by 5,847
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  9. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  10. 2021Convex optimizationcited by 2,515
  11. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  12. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
33 more
  1. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  2. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  3. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
  4. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  5. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  6. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 224
  7. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 197
  8. 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Proverscited by 168
  9. 2021Constrained Language Models Yield Few-Shot Semantic Parserscited by 163
  10. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  11. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  12. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 155
  13. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 154
  14. 2020NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraintscited by 149
  15. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 148
  16. 2024Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Makingcited by 144
  17. 2023AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkerscited by 113
  18. 2017Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimizationcited by 97
  19. 2018CGMH: Constrained Sentence Generation by Metropolis-Hastings Samplingcited by 94
  20. 2024Constrained Synthesis with Projected Diffusion Modelscited by 89
  21. 2020Lagrangian Duality for Constrained Deep Learningcited by 74
  22. 2023COLLIE: Systematic Construction of Constrained Text Generation Taskscited by 70
  23. 2023Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agentscited by 69
  24. 2022Gradient-Based Constrained Sampling from Language Modelscited by 64
  25. 2024Speculative Diffusion Decoding: Accelerating Language Generation through Diffusioncited by 57
  26. 2023Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environmentscited by 53
  27. 2024Decision-Making Behavior Evaluation Framework for LLMs under Uncertain Contextcited by 41
  28. 2023Improved Logical Reasoning of Language Models via Differentiable Symbolic Programmingcited by 18
  29. 2024Sketch-Guided Constrained Decoding for Boosting Blackbox Large Language Models without Logit Accesscited by 9
  30. 2023A Block Metropolis-Hastings Sampler for Controllable Energy-based Text Generationcited by 6
  31. 2024Local and Global Decoding in Text Generationcited by 5
  32. 2023Structured Voronoi Samplingcited by 4
  33. 2024Logic Agent: Enhancing Validity with Logic Rule Invocationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 16.

17 papers.

Sort:

By area:

  1. 2025Calibrating Generative Models to Distributional Constraintscited by 6
  2. 2026Simple Self-Conditioning Adaptation for Masked Diffusion Modelscited by 4
  3. 2026Primal-Dual Guided Decoding for Constrained Discrete Diffusioncited by 4
  4. 2026Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correctioncited by 3
  5. 2026Search-Augmented Masked Diffusion Models for Constrained Generationcited by 1
  6. 2026Large Neighborhood Search meets Iterative Neural Constraint Heuristicscited by 1
  7. 2026Constrained Code Generation with Discrete Diffusioncited by 1
  8. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniquescited by 1
  9. 2026Focus on the Core: Empowering Diffusion Large Language Models by Self-Contrastcited by 0
  10. 2026Masked Diffusion Modeling for Anomaly Detectioncited by 0
  11. 2026Beyond the Autoregressive Horizon: A Comprehensive Survey of Diffusion Models, World Modelling, and State Space Models for Codecited by 0
  12. 2026Ready to Speak: Aligning LLMs for TTS-Friendly Text Generationcited by 0
5 more
  1. 2026Let It Go or Learn to Self-Correct: Continuous Diffusion for Constrained Discrete Taskscited by 0
  2. 2026Extracting Arguments, Not Just Classifying Them: Instruction-Tuned LLMs for Generative Component Detectioncited by 0
  3. 2026Grab a Coffee: Future-Aware Guidance for Discrete Diffusion with Compiled Objectivescited by 0
  4. 2026pCoMole: Pareto-Constrained Molecule Editing with Discrete Flowscited by 0
  5. 2026Only What You've Signed up For: Secure Workflows with Constrained Diffusion Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Continuous Diffusion Models Can Obey Formal Syntax
  2. 2023Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time
  3. 2025Non-Markovian Discrete Diffusion with Causal Language Models
  4. 2025Discrete Diffusion Models for Language Generation
  5. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling
  6. 2025Latent-Augmented Discrete Diffusion Models
  7. 2024Think While You Generate: Discrete Diffusion with Planned Denoising
  8. 2026Drifting Objectives for Refining Discrete Diffusion Language Models