mlsanityData to 14 Oct 2026

Self-Guided Diffusion Models

arXiv 2210.06462 ↗First posted 12 Oct 2022cs.CV cs.AI cs.LGcited by 37

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have demonstrated remarkable progress in image generation quality, especially when guidance is used to control the generative process. However, guidance requires a large amount of image-annotation pairs for training and is thus dependent on their availability, correctness and unbiasedness. In this paper, we eliminate the need for such annotation by instead leveraging the flexibility of self-supervision signals to design a framework for self-guided diffusion models. By leveraging a feature extraction function and a self-annotation function, our method provides guidance signals at various image granularities: from the level of holistic images to object boxes and even segmentation masks. Our experiments on single-label and multi-label image datasets demonstrate that self-labeled guidance always outperforms diffusion models without guidance and may even surpass guidance based on ground-truth labels, especially on unbalanced data. When equipped with self-supervised box or mask proposals, our method further generates visually diverse yet semantically consistent images, without the need for any class, box, or segment label annotation. Self-guided diffusion is simple, flexible and expected to profit from deployment at scale. Source code will be at: https://taohu.me/sgdm/

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 61 of its 76 references that are in the map. First posted on arXiv 12 Oct 2022.

Builds on

61 of 76 references.

Closest ancestors · 9

Same direction, newest first

  1. 2022Classifier-Free Diffusion Guidance26 Jul
  2. 2022Few-Shot Diffusion Models30 May
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  4. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representation30 Nov
  5. 2021Variational Diffusion Models1 Jul
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  8. 2020Denoising Diffusion Implicit Models6 Oct
  9. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2020End-to-End Object Detection with Transformerscited by 7,414
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
38 more
  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  2. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  4. 2016Improved Techniques for Training GANscited by 2,970
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  7. 2014Conditional Generative Adversarial Netscited by 2,162
  8. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  9. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  10. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  11. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  12. 2019Contrastive Multiview Codingcited by 1,301
  13. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
  14. 2019Self-labelling via simultaneous clustering and representation learningcited by 528
  15. 2021Solving Inverse Problems in Medical Imaging with Score-Based Generative Modelscited by 521
  16. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  17. 2022Masked Siamese Networks for Label-Efficient Learningcited by 302
  18. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  19. 2022Diffusion Probabilistic Modeling for Video Generationcited by 238
  20. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 230
  21. 2018A Note on the Inception Scorecited by 226
  22. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  23. 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
  24. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  25. 2022KNN-Diffusion: Image Generation via Large-Scale Retrievalcited by 137
  26. 2021SOFT: Softmax-free Transformer with Linear Complexitycited by 128
  27. 2021LAFITE: Towards Language-Free Training for Text-to-Image Generationcited by 128
  28. 2016Discrete Variational Autoencoderscited by 125
  29. 2021Instance-Conditioned GANcited by 121
  30. 2021Large-scale Unsupervised Semantic Segmentationcited by 101
  31. 2018Self-Supervised GANs via Auxiliary Rotation Losscited by 94
  32. 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90
  33. 2021High Fidelity Visualization of What Your Self-Supervised Representation Knows Aboutcited by 66
  34. 2020Toward unsupervised, multi-object discovery in large-scale image collectionscited by 65
  35. 2021Large-Scale Unsupervised Object Discoverycited by 54
  36. 2019High-Fidelity Image Generation With Fewer Labelscited by 52
  37. 2017Oversampling for Imbalanced Learning Based on K-Means and SMOTEcited by 38
  38. 2021Partition-Guided GANscited by 9

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 11; 2024: 13; 2025: 8; 2026: 5.

37 papers.

Sort:

By area:

  1. 2023Return of Unconditional Generation: A Self-supervised Representation Generation Methodcited by 98
  2. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  3. 2024ZigMa: A DiT-style Zigzag Mamba Diffusion Modelcited by 84
  4. 2024DepthFM: Fast Monocular Depth Estimation with Flow Matchingcited by 76
  5. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 60
  6. 2024Diffusion Models and Representation Learning: A Surveycited by 54
  7. 2023Learned representation-guided diffusion models for large-image generationcited by 40
  8. 2023Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labelscited by 37
  9. 2023Latent Space Editing in Transformer-Based Flow Matchingcited by 32
  10. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  11. 2025TREAD: Token Routing for Efficient Architecture-agnostic Diffusion Trainingcited by 21
  12. 2024DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latentscited by 20
25 more
  1. 2023ProtoDiff: Learning to Learn Prototypical Networks by Task-Guided Diffusioncited by 16
  2. 2024Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modelingcited by 13
  3. 2023Visual Chain-of-Thought Diffusion Modelscited by 12
  4. 2024PrimeDepth: Efficient Monocular Depth Estimation with a Stable Diffusion Preimagecited by 12
  5. 2024[MASK] is All You Needcited by 11
  6. 2023Guided Diffusion from Self-Supervised Diffusion Featurescited by 9
  7. 2024Cross-Domain Graph Data Scaling: A Showcase with Diffusion Modelscited by 8
  8. 2024Retinex-Diffusion: On Controlling Illumination Conditions in Diffusion Models via Retinex Theorycited by 8
  9. 2023ScribbleGen: Generative Data Augmentation Improves Scribble-supervised Semantic Segmentationcited by 5
  10. 2024Rethinking cluster-conditioned diffusion models for label-free image synthesiscited by 5
  11. 2024TreeDiffusion: Hierarchical Generative Clustering for Conditional Diffusioncited by 3
  12. 2025Conditioning Residuals for Diffusion Models via Representation Feedbackcited by 3
  13. 2025Compositional Discrete Latent Code for High Fidelity, Productive Diffusion Modelscited by 3
  14. 2024Prompt Diffusion Robustifies Any-Modality Prompt Learningcited by 2
  15. 2024Nested Diffusion Models Using Hierarchical Latent Priorscited by 2
  16. 2025A Comprehensive Review of Diffusion Models in Smart Agriculture: Progress, Applications, and Challengescited by 2
  17. 2025Composable Score-based Graph Diffusion Model for Multi-Conditional Molecular Generationcited by 2
  18. 2026Exploiting Completeness Perception with Diffusion Transformer for Unified 3D MRI Synthesiscited by 1
  19. 2025Guidance-base Diffusion Models for Improving Photoacoustic Image Qualitycited by 0
  20. 2025Extendable Generalization Self-Supervised Diffusion for Low-Dose CT Reconstructioncited by 0
  21. 2025Align & Invert: Solving Inverse Problems with Diffusion and Flow-based Models via Representation Alignmentcited by 0
  22. 2026CausalGDP: Causality-Guided Diffusion Policies for Reinforcement Learningcited by 0
  23. 2026CoFi-UCGen: Coarse-to-Fine Unsupervised Conditional Generation without Label Priorscited by 0
  24. 2026Learning A Unified Template for Gait Recognitioncited by 0
  25. 2026From Pixels, Without Pre-training: Joint Generative and Self-Supervised Representation Learning in One Modelcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Diffusion Self-Guidance for Controllable Image Generation
  2. 2024Self-Guidance: Boosting Flow and Diffusion Generation on Their Own
  3. 2024Plug-and-Play Diffusion Distillation
  4. 2023Elucidating The Design Space of Classifier-Guided Diffusion Generation
  5. 2022On Distillation of Guided Diffusion Models
  6. 2026Towards Controllable Image Generation through Representation-Conditioned Diffusion Models
  7. 2024Diffusion Self-Distillation for Zero-Shot Customized Image Generation
  8. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidance