mlsanityData to 7 Oct 2026

Censored Sampling of Diffusion Models Using 3 Minutes of Human Feedback

arXiv 2307.02770 ↗First posted 6 Jul 2023cs.CV cs.AIcited by 12

On the map: Diffusion models → Robustness, privacy and adversarial aspects

also in 1 more
Abstract (arXiv)

Diffusion models have recently shown remarkable success in high-quality image generation. Sometimes, however, a pre-trained diffusion model exhibits partial misalignment in the sense that the model can generate good images, but it sometimes outputs undesirable images. If so, we simply need to prevent the generation of the bad images, and we call this task censoring. In this work, we present censored generation with a pre-trained diffusion model using a reward model trained on minimal human feedback. We show that censoring can be accomplished with extreme human feedback efficiency and that labels generated with a mere few minutes of human feedback are sufficient. Code available at: https://github.com/tetrzim/diffusion-human-feedback.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 34 of its 50 references that are in the map. First posted on arXiv 6 Jul 2023.

Builds on

34 of 50 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models30 Mar
  2. 2023Erasing Concepts from Diffusion Models13 Mar
  3. 2023Extracting Training Data from Diffusion Models30 Jan

Other references · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2022Classifier-Free Diffusion Guidancecited by 5,717
  12. 2017Deep reinforcement learning from human preferencescited by 3,846
19 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  3. 2020Learning to summarize from human feedbackcited by 2,562
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  5. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  6. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  7. 2021Score-based Generative Modeling in Latent Spacecited by 634
  8. 2023Universal Guidance for Diffusion Modelscited by 426
  9. 2023Aligning Text-to-Image Models using Human Feedbackcited by 418
  10. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  11. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  12. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  13. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  14. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 214
  15. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 167
  16. 2022Refining Generative Process with Discriminator Guidance in Score-based Diffusion Modelscited by 98
  17. 2023Don't Play Favorites: Minority Guidance for Diffusion Modelscited by 37
  18. 2022Enhancing Diffusion-Based Image Synthesis with Robust Classifier Guidancecited by 33
  19. 2017Improving image generative models with human interactionscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 6; 2026: 1.

12 papers.

  1. 2025Test-time Alignment of Diffusion Models without Reward Over-optimizationcited by 89
  2. 2024Margin-aware Preference Optimization for Aligning Diffusion Models without Referencecited by 45
  3. 2024Training-free Diffusion Model Alignment with Sampling Demonscited by 30
  4. 2024Adversarial Environment Design via Regret-Guided Diffusion Modelscited by 9
  5. 2024Localize, Understand, Collaborate: Semantic-Aware Dragging via Intention Reasonercited by 7
  6. 2023Non-Cross Diffusion for Semantic Consistencycited by 2
  7. 2025Sample and Computationally Efficient Continuous-Time Reinforcement Learning with General Function Approximationcited by 2
  8. 2026Personalized Image Generation via Human-in-the-loop Bayesian Optimizationcited by 2
  9. 2025Instance-Dependent Continuous-Time Reinforcement Learning via Maximum Likelihood Estimationcited by 1
  10. 2025Scaling Multi-Agent Environment Co-Design with Diffusion Modelscited by 1
  11. 2025Towards SFW sampling for diffusion models via external conditioningcited by 0
  12. 2025Censored Sampling for Topology Design: Guiding Diffusion with Human Preferencescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ImageReFL: Balancing Quality and Diversity in Human-Aligned Diffusion Models
  2. 2025Positive-Unlabeled Diffusion Models for Preventing Sensitive Data Generation
  3. 2023Compensation Sampling for Improved Convergence in Diffusion Models
  4. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models
  5. 2024Conditional Image Synthesis with Diffusion Models: A Survey
  6. 2024Model-Agnostic Human Preference Inversion in Diffusion Models
  7. 2022Markup-to-Image Diffusion Models with Scheduled Sampling
  8. 2023Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation