mlsanityData to 7 Oct 2026

Diffusion models as plug-and-play priors

arXiv 2206.09012 ↗First posted 17 Jun 2022cs.LG cs.CVcited by 230

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We consider the problem of inferring high-dimensional data x in a model that consists of a prior p(x) and an auxiliary differentiable constraint c(x,y) on x given some additional information y. In this paper, the prior is an independently trained denoising diffusion generative model. The auxiliary constraint is expected to have a differentiable form, but can come from diverse sources. The possibility of such inference turns diffusion models into plug-and-play modules, thereby allowing a range of potential applications in adapting models to new domains and tasks, such as conditional generation or image segmentation. The structure of diffusion models allows us to perform approximate inference by iterating differentiation through the fixed denoising network enriched with different amounts of noise at each step. Considering many noised versions of x in evaluation of its fitness is a novel search mechanism that may lead to new algorithms for solving combinatorial optimization problems.

Key facts. Cited by 230 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 29 of its 56 references that are in the map. First posted on arXiv 17 Jun 2022.

Builds on

29 of 56 references.

Closest ancestors · 6

Same direction, newest first

  1. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  2. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models6 Aug
  3. 2021Score-based Generative Modeling in Latent Space10 Jun
  4. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  5. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  6. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2013Auto-Encoding Variational Bayescited by 3,980
  8. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  9. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  10. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 666
  11. 2018Attention, Learn to Solve Routing Problems!cited by 646
  12. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputationcited by 577
11 more
  1. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  2. 2021Solving Inverse Problems in Medical Imaging with Score-Based Generative Modelscited by 514
  3. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  4. 2022Diffusion Models for Adversarial Purificationcited by 469
  5. 2021SegDiff: Image Segmentation with Diffusion Probabilistic Modelscited by 279
  6. 2019An Efficient Graph Convolutional Network Technique for the Travelling Salesman Problemcited by 265
  7. 2021SNIPS: Solving Noisy Inverse Problems Stochasticallycited by 191
  8. 2021D2C: Diffusion-Denoising Models for Few-shot Conditional Generationcited by 96
  9. 2021The Transformer Network for the Traveling Salesman Problemcited by 59
  10. 2017Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Modelscited by 39
  11. 2022Resolving label uncertainty with implicit posterior modelscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 12; 2023: 62; 2024: 78; 2025: 50; 2026: 28.

230 papers.

Sort:

By area:

  1. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  2. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  3. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  4. 2022Diffusion Models in Vision: A Surveycited by 1,008
  5. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  6. 2023Universal Guidance for Diffusion Modelscited by 426
  7. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  8. 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
  9. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  10. 2023A Variational Perspective on Solving Inverse Problems with Diffusion Modelscited by 217
  11. 2023DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimizationcited by 216
  12. 2023Protein Design with Guided Discrete Diffusioncited by 154
38 more
  1. 2023Manifold Preserving Guided Diffusioncited by 153
  2. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learningcited by 145
  3. 2024Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Controlcited by 136
  4. 2023Score-Based Diffusion Models as Principled Priors for Inverse Imagingcited by 126
  5. 2023Collaborative Diffusion for Multi-Modal Face Generation and Editingcited by 116
  6. 2022VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Modelscited by 112
  7. 2024PnP-Flow: Plug-and-Play Image Restoration with Flow Matchingcited by 92
  8. 2023Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecastingcited by 86
  9. 2023ReVersion: Diffusion-Based Relation Inversion from Imagescited by 82
  10. 2023DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimationcited by 81
  11. 2023Robust Classification via a Single Diffusion Modelcited by 79
  12. 2024Provably Robust Score-Based Diffusion Posterior Sampling for Plug-and-Play Image Reconstructioncited by 73
  13. 2024Diffusion Models in Low-Level Vision: A Surveycited by 73
  14. 2023Training-free Linear Image Inverses via Flowscited by 70
  15. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 70
  16. 2024Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Priorcited by 60
  17. 2024Gradient Guidance for Diffusion Models: An Optimization Perspectivecited by 59
  18. 2024Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstructioncited by 56
  19. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  20. 2024Feedback Efficient Online Fine-Tuning of Diffusion Modelscited by 52
  21. 2022Efficient Diffusion Models for Vision: A Surveycited by 50
  22. 2023Reward-Directed Conditional Diffusion: Provable Distribution Estimation and Reward Improvementcited by 49
  23. 2023Customizing 360-Degree Panoramas through Text-to-Image Diffusion Modelscited by 47
  24. 2023DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detectioncited by 46
  25. 2023Visual Anagrams: Generating Multi-View Optical Illusions with Diffusion Modelscited by 46
  26. 2025Fast T2T: Optimization Consistency Speeds Up Diffusion-Based Training-to-Testing Solving for Combinatorial Optimizationcited by 43
  27. 2024Text-to-Image Rectified Flow as Plug-and-Play Priorscited by 38
  28. 2023Decomposing the Enigma: Subgoal-based Demonstration Learning for Formal Theorem Provingcited by 37
  29. 2022Peekaboo: Text to Image Diffusion Models are Zero-Shot Segmentorscited by 34
  30. 2024Lossy Image Compression with Foundation Diffusion Modelscited by 34
  31. 2024SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesiscited by 34
  32. 2023Addressing Negative Transfer in Diffusion Modelscited by 33
  33. 2023An Ensemble Score Filter for Tracking High-Dimensional Nonlinear Dynamical Systemscited by 33
  34. 2024Ambient Diffusion Posterior Sampling: Solving Inverse Problems with Diffusion Models Trained on Corrupted Datacited by 32
  35. 2024Motion Planning Diffusion: Learning and Adapting Robot Motion Planning with Diffusion Modelscited by 32
  36. 2023DiffBEV: Conditional Diffusion Model for Bird's Eye View Perceptioncited by 30
  37. 2023User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systemscited by 30
  38. 2025Does FLUX Already Know How to Perform Physically Plausible Image Composition?cited by 30

50 most cited of 230 All 230 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Diffusion Models: A Mathematical Introduction
  2. 2025Low-dimensional adaptation of diffusion models: Convergence in total variation
  3. 2022Image Embedding for Denoising Generative Models
  4. 2022Understanding Diffusion Models: A Unified Perspective
  5. 2023Reflected Diffusion Models
  6. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision
  7. 2024Statistical guarantees for denoising reflected diffusion models
  8. 2023To smooth a cloud or to pin it down: Guarantees and Insights on Score Matching in Denoising Diffusion Models