# Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints

arXiv: https://arxiv.org/abs/2402.18012 · HTML: https://mlsanity.com/paper/2402.18012/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 71 of its 131 references that are in the map.
- First posted on arXiv 28 Feb 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 71)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon](https://mlsanity.com/paper/1811.06128/) (2018)
- [Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling](https://mlsanity.com/paper/2106.01357/) (2021)
- [Learning Combinatorial Optimization Algorithms over Graphs](https://mlsanity.com/paper/1704.01665/) (2017)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022)
- [Exact Combinatorial Optimization with Graph Convolutional Neural Networks](https://mlsanity.com/paper/1906.01629/) (2019)
- [Stochastic Gradient Descent as Approximate Bayesian Inference](https://mlsanity.com/paper/1704.04289/) (2017)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis](https://mlsanity.com/paper/1702.03849/) (2017)
- [Convergence of denoising diffusion models under the manifold hypothesis](https://mlsanity.com/paper/2208.05314/) (2022)
- [Reinforcement Learning for Combinatorial Optimization: A Survey](https://mlsanity.com/paper/2003.03600/) (2020)
- [Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization](https://mlsanity.com/paper/2308.03686/) (2023)
- [Practical and Asymptotically Exact Conditional Sampling in Diffusion Models](https://mlsanity.com/paper/2306.17775/) (2023)
- [DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimization](https://mlsanity.com/paper/2302.08224/) (2023)
- [Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC](https://mlsanity.com/paper/2302.11552/) (2023)
- [Hierarchical Generation of Molecular Graphs using Structural Motifs](https://mlsanity.com/paper/2002.03230/) (2020)
- [Learning to Optimize](https://mlsanity.com/paper/1606.01885/) (2016)
- [Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search](https://mlsanity.com/paper/1810.10659/) (2018)
- [End-to-End Constrained Optimization Learning: A Survey](https://mlsanity.com/paper/2103.16378/) (2021)
- [Protein Design with Guided Discrete Diffusion](https://mlsanity.com/paper/2305.20009/) (2023)

## Built on by (most cited first, 30 of 35)

- [Reasoning with Sampling: Your Base Model is Smarter Than You Think](https://mlsanity.com/paper/2510.14901/) (2025)
- [Fine-tuning of diffusion models via stochastic control: entropy regularization and beyond](https://mlsanity.com/paper/2403.06279/) (2024)
- [The Superposition of Diffusion Models Using the Ito Density Estimator](https://mlsanity.com/paper/2412.17762/) (2024)
- [Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNets](https://mlsanity.com/paper/2412.07775/) (2024)
- [Improving GFlowNets for Text-to-Image Diffusion Alignment](https://mlsanity.com/paper/2406.00633/) (2024)
- [Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models](https://mlsanity.com/paper/2405.19673/) (2024)
- [ParetoFlow: Guided Flows in Multi-Objective Optimization](https://mlsanity.com/paper/2412.03718/) (2024)
- [RNE: plug-and-play diffusion inference-time control and energy-based training](https://mlsanity.com/paper/2506.05668/) (2025)
- [Guided Trajectory Generation with Diffusion Models for Offline Model-based Optimization](https://mlsanity.com/paper/2407.01624/) (2024)
- [Generative Models in Decision Making: A Survey](https://mlsanity.com/paper/2502.17100/) (2025)
- [Learn to Guide Your Diffusion Model](https://mlsanity.com/paper/2510.00815/) (2025)
- [Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization](https://mlsanity.com/paper/2502.16824/) (2025)
- [CREPE: Controlling Diffusion with Replica Exchange](https://mlsanity.com/paper/2509.23265/) (2025)
- [Composite Flow Matching for Reinforcement Learning with Shifted-Dynamics Data](https://mlsanity.com/paper/2505.23062/) (2025)
- [Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddings](https://mlsanity.com/paper/2506.07109/) (2025)
- [Empower Structure-Based Molecule Optimization with Gradient Guided Bayesian Flow Networks](https://mlsanity.com/paper/2411.13280/) (2024)
- [Robust Optimization with Diffusion Models for Green Security](https://mlsanity.com/paper/2503.05730/) (2025)
- [Posterior Inference in Latent Space for Scalable Constrained Black-box Optimization](https://mlsanity.com/paper/2507.00480/) (2025)
- [Diffusion-DFL: Decision-focused Diffusion Models for Stochastic Optimization](https://mlsanity.com/paper/2510.11590/) (2025)
- [Convergence and Stability Analysis of Self-Consuming Generative Models with Heterogeneous Human Curation](https://mlsanity.com/paper/2511.09002/) (2025)
- [Constrained Flow Optimization via Sequential Fine Tuning for Molecular Design](https://mlsanity.com/paper/2605.30610/) (2026)
- [A Gradient Guided Diffusion Framework for Chance Constrained Programming](https://mlsanity.com/paper/2510.12238/) (2025)
- [Constrained Diffusion Models with Primal-Dual Inference](https://mlsanity.com/paper/2606.17192/) (2026)
- [Diffusion-based learning framework for Constrained Nonconvex Optimization with Weighted Bootstrapped Refinement](https://mlsanity.com/paper/2502.10330/) (2025)
- [Learning Design-Score Manifold to Guide Diffusion Models for Offline Optimization](https://mlsanity.com/paper/2506.05680/) (2025)
- [LLM Flow Processes for Text-Conditioned Regression](https://mlsanity.com/paper/2601.06147/) (2026)
- [Logical Guidance for the Exact Composition of Diffusion Models](https://mlsanity.com/paper/2602.05549/) (2026)
- [Recipes for Steering and Scaling LLMs via Sampling](https://mlsanity.com/paper/2608.26120/) (2026)
- [GaiaFlow: Semantic-Guided Diffusion Tuning for Carbon-Frugal Search](https://mlsanity.com/paper/2602.15423/) (2026)
- [Sobolev Regularized Score Difference Estimation in Diffusion Models](https://mlsanity.com/paper/2608.18237/) (2026)

## Abstract (arXiv)

Addressing real-world optimization problems becomes particularly challenging when analytic objective functions or constraints are unavailable. While numerous studies have addressed the issue of unknown objectives, limited research has focused on scenarios where feasibility constraints are not given explicitly. Overlooking these constraints can lead to spurious solutions that are unrealistic in practice. To deal with such unknown constraints, we propose to perform optimization within the data manifold using diffusion models. To constrain the optimization process to the data manifold, we reformulate the original optimization problem as a sampling problem from the product of the Boltzmann distribution defined by the objective function and the data distribution learned by the diffusion model. Depending on the differentiability of the objective function, we propose two different sampling methods. For differentiable objectives, we propose a two-stage framework that begins with a guided diffusion process for warm-up, followed by a Langevin dynamics stage for further correction. For non-differentiable objectives, we propose an iterative importance sampling strategy using the diffusion model as the proposal distribution. Comprehensive experiments on a synthetic dataset, six real-world black-box optimization datasets, and a multi-objective molecule optimization dataset show that our method achieves better or comparable performance with previous state-of-the-art baselines.