mlsanityData to 7 Oct 2026

Universal Guidance for Diffusion Models

arXiv 2302.07121 ↗First posted 14 Feb 2023cs.CV cs.LGcited by 426

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Typical diffusion models are trained to accept a particular form of conditioning, most commonly text, and cannot be conditioned on other modalities without retraining. In this work, we propose a universal guidance algorithm that enables diffusion models to be controlled by arbitrary guidance modalities without the need to retrain any use-specific components. We show that our algorithm successfully generates quality images with guidance functions including segmentation, face recognition, object detection, and classifier signals. Code is available at https://github.com/arpitbansal297/Universal-Guided-Diffusion.

Key facts. Cited by 426 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 27 of its 31 references that are in the map. First posted on arXiv 14 Feb 2023.

Builds on

27 of 31 references.

Closest ancestors · 9

Same direction, newest first

  1. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise19 Aug
  2. 2022Classifier-Free Diffusion Guidance26 Jul
  3. 2022Diffusion models as plug-and-play priors17 Jun
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  8. 2020Denoising Diffusion Implicit Models6 Oct
  9. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  6. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  7. 2019Searching for MobileNetV3cited by 1,984
  8. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  9. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  10. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  11. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  12. 2022Denoising Diffusion Restoration Modelscited by 951
6 more
  1. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 895
  2. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Modelcited by 574
  3. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 501
  4. 2021Deblurring via Stochastic Refinementcited by 248
  5. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesiscited by 136
  6. 2021Benchmarking Detection Transfer Learning with Vision Transformerscited by 130

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 62; 2024: 112; 2025: 135; 2026: 117.

426 papers.

Sort:

By area:

  1. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  2. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
  3. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  4. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
  5. 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
  6. 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 293
  7. 2023FreeDoM: Training-Free Energy-Guided Conditional Diffusion Modelcited by 290
  8. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 280
  9. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 224
  10. 2023Practical and Asymptotically Exact Conditional Sampling in Diffusion Modelscited by 218
  11. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 193
  12. 2023DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusioncited by 167
38 more
  1. 2023Grounded Text-to-Image Synthesis with Attention Refocusingcited by 163
  2. 2023Dense Text-to-Image Generation with Attention Modulationcited by 162
  3. 2023Manifold Preserving Guided Diffusioncited by 153
  4. 2024Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Controlcited by 136
  5. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 127
  6. 2024Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Controlcited by 117
  7. 2024PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AIcited by 116
  8. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimizationcited by 115
  9. 2023End-to-End Diffusion Latent Optimization Improves Classifier Guidancecited by 107
  10. 2024TFG: Unified Training-Free Guidance for Diffusion Modelscited by 103
  11. 2024Arc2Face: A Foundation Model for ID-Consistent Human Facescited by 101
  12. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 95
  13. 2023Generative Image Dynamicscited by 93
  14. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Reviewcited by 92
  15. 2023Expressive Text-to-Image Generation with Rich Textcited by 89
  16. 2025Test-time Alignment of Diffusion Models without Reward Over-optimizationcited by 89
  17. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 82
  18. 2024Classifier-Free Guidance is a Predictor-Correctorcited by 79
  19. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  20. 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Designcited by 72
  21. 2024All-in-one simulation-based inferencecited by 71
  22. 2024Guidance with Spherical Gaussian Constraint for Conditional Diffusioncited by 69
  23. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 66
  24. 2023Zero-shot spatial layout conditioning for text-to-image diffusion modelscited by 64
  25. 2024DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalizationcited by 60
  26. 2024Learning Diffusion Priors from Observations by Expectation Maximizationcited by 60
  27. 2024Video Diffusion Models are Training-free Motion Interpreter and Controllercited by 60
  28. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  29. 2024Gradient Guidance for Diffusion Models: An Optimization Perspectivecited by 59
  30. 2023LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Promptscited by 53
  31. 2023FaceStudio: Put Your Face Everywhere in Secondscited by 53
  32. 2023Readout Guidance: Learning Control from Diffusion Featurescited by 52
  33. 2024Feedback Efficient Online Fine-Tuning of Diffusion Modelscited by 52
  34. 2024No Training, No Problem: Rethinking Classifier-Free Guidance for Diffusion Modelscited by 52
  35. 2024Balancing Act: Distribution-Guided Debiasing in Diffusion Modelscited by 50
  36. 2025Inference-time Scaling of Diffusion Models through Classical Searchcited by 48
  37. 2024Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusioncited by 47
  38. 2024Distillation of Discrete Diffusion through Dimensional Correlationscited by 46

50 most cited of 426 All 426 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025How Much To Guide: Revisiting Adaptive Guidance in Classifier-Free Guidance Text-to-Vision Diffusion Models
  2. 2023Guided Diffusion from Self-Supervised Diffusion Features
  3. 2024Plug-and-Play Diffusion Distillation
  4. 2022Self-Guided Diffusion Models
  5. 2026Improving Diffusion Generalization with Weak-to-Strong Segmented Guidance
  6. 2024Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models
  7. 2025Diffusion Models without Classifier-free Guidance
  8. 2026Learned End-to-End Guidance Schedules for Diffusion Models