# Constrained Diffusion Models via Dual Training

arXiv: https://arxiv.org/abs/2408.15094 · HTML: https://mlsanity.com/paper/2408.15094/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 60 of its 81 references that are in the map.
- First posted on arXiv 27 Aug 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 60)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Training Diffusion Models with Reinforcement Learning](https://mlsanity.com/paper/2305.13301/) (2023)
- [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022)
- [Directly Fine-Tuning Diffusion Models on Differentiable Rewards](https://mlsanity.com/paper/2309.17400/) (2023)
- [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022)
- [Universal Guidance for Diffusion Models](https://mlsanity.com/paper/2302.07121/) (2023)
- [Aligning Text-to-Image Models using Human Feedback](https://mlsanity.com/paper/2302.12192/) (2023)
- [Understanding Diffusion Models: A Unified Perspective](https://mlsanity.com/paper/2208.11970/) (2022)
- [Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation](https://mlsanity.com/paper/2303.00848/) (2023)
- [Human Preference Score: Better Aligning Text-to-Image Models with Human Preference](https://mlsanity.com/paper/2303.14420/) (2023)
- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [A Survey on Generative Diffusion Model](https://mlsanity.com/paper/2209.02646/) (2022)
- [Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization](https://mlsanity.com/paper/2308.03686/) (2023)
- [Stable Bias: Analyzing Societal Representations in Diffusion Models](https://mlsanity.com/paper/2303.11408/) (2023)
- [Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC](https://mlsanity.com/paper/2302.11552/) (2023)
- [Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions](https://mlsanity.com/paper/2211.01916/) (2022)

## Built on by (most cited first, 20 of 20)

- [Conditional Diffusion Guidance under Hard Constraint: A Stochastic Analysis Approach](https://mlsanity.com/paper/2602.05533/) (2026)
- [Composition and Alignment of Diffusion Models using Constrained Learning](https://mlsanity.com/paper/2508.19104/) (2025)
- [Mirror Flow Matching with Heavy-Tailed Priors for Generative Modeling on Convex Domains](https://mlsanity.com/paper/2510.08929/) (2025)
- [Calibrating Generative Models to Distributional Constraints](https://mlsanity.com/paper/2510.10020/) (2025)
- [Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing](https://mlsanity.com/paper/2502.09564/) (2025)
- [Constrained Flow Optimization via Sequential Fine Tuning for Molecular Design](https://mlsanity.com/paper/2605.30610/) (2026)
- [Constraint-Aware Flow Matching via Randomized Exploration](https://mlsanity.com/paper/2508.13316/) (2025)
- [DOS: Directional Object Separation in Text Embeddings for Multi-Object Image Generation](https://mlsanity.com/paper/2510.14376/) (2025)
- [Score-based constrained generative modeling via Langevin diffusions with boundary conditions](https://mlsanity.com/paper/2510.23985/) (2025)
- [Logic-Guided Vector Fields for Constrained Generative Modeling](https://mlsanity.com/paper/2602.02009/) (2026)
- [Unlearning in Diffusion Models: A Unified Framework with KL Divergence and Likelihood Constraints](https://mlsanity.com/paper/2605.30825/) (2026)
- [Constrained Diffusion Models with Primal-Dual Inference](https://mlsanity.com/paper/2606.17192/) (2026)
- [How Bias Binds: Measuring Hidden Associations for Bias Control in Text-to-Image Compositions](https://mlsanity.com/paper/2511.07091/) (2025)
- [Softly Constrained Denoisers for Diffusion Models Applied to Partial Differential Equations](https://mlsanity.com/paper/2512.14980/) (2025)
- [ADLGen: Synthesizing Symbolic, Event-Triggered Sensor Sequences for Human Activity Modeling](https://mlsanity.com/paper/2505.17987/) (2025)
- [A Constrained Optimization Perspective of Unrolled Transformers](https://mlsanity.com/paper/2601.17257/) (2026)
- [Should the Boundary Term Be Learned in Reflected Diffusion? Conormal Trace and Reflection Masking](https://mlsanity.com/paper/2608.03469/) (2026)
- [Self-Augmented Diffusion Guidance for Physics-Informed Generation](https://mlsanity.com/paper/2608.26748/) (2026)
- [Denoising as Projection: Constrained Optimization with Gradient-Guided Diffusion](https://mlsanity.com/paper/2608.29507/) (2026)
- [Primal-Dual Flow Matching for Sample-Wise Constrained Generation](https://mlsanity.com/paper/2610.12699/) (2026)

## Abstract (arXiv)

Diffusion models have attained prominence for their ability to synthesize a probability distribution for a given dataset via a diffusion process, enabling the generation of new data points with high fidelity. However, diffusion processes are prone to generating samples that reflect biases in a training dataset. To address this issue, we develop constrained diffusion models by imposing diffusion constraints based on desired distributions that are informed by requirements. Specifically, we cast the training of diffusion models under requirements as a constrained distribution optimization problem that aims to reduce the distribution difference between original and generated data while obeying constraints on the distribution of generated data. We show that our constrained diffusion models generate new data from a mixture data distribution that achieves the optimal trade-off among objective and constraints. To train constrained diffusion models, we develop a dual training algorithm and characterize the optimality of the trained constrained diffusion model. We empirically demonstrate the effectiveness of our constrained models in two constrained generation tasks: (i) we consider a dataset with one or more underrepresented classes where we train the model with constraints to ensure fairly sampling from all classes during inference; (ii) we fine-tune a pre-trained diffusion model to sample from a new dataset while avoiding overfitting.