mlsanityData to 7 Oct 2026

Rethinking cluster-conditioned diffusion models for label-free image synthesis

arXiv 2403.00570 ↗First posted 1 Mar 2024cs.CV cs.AI cs.LGcited by 5

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion-based image generation models can enhance image quality when conditioned on ground truth labels. Here, we conduct a comprehensive experimental study on image-level conditioning for diffusion models using cluster assignments. We investigate how individual clustering determinants, such as the number of clusters and the clustering method, impact image synthesis across three different datasets. Given the optimal number of clusters with respect to image synthesis, we show that cluster-conditioning can achieve state-of-the-art performance, with an FID of 1.67 for CIFAR10 and 2.17 for CIFAR100, along with a strong increase in training sample efficiency. We further propose a novel empirical method to estimate an upper bound for the optimal number of clusters. Unlike existing approaches, we find no significant association between clustering performance and the corresponding cluster-conditional FID scores. The code is available at https://github.com/HHU-MMBS/cedm-official-wavc2025.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 73 of its 102 references that are in the map. First posted on arXiv 1 Mar 2024.

Builds on

73 of 102 references.

Closest ancestors · 20

Same direction, newest first

  1. 2023Guided Diffusion from Self-Supervised Diffusion Features14 Dec
  2. 2023Analyzing and Improving the Training Dynamics of Diffusion Models5 Dec
  3. 2023CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Sampling26 Oct
  4. 2023Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation?22 May
  5. 2023Universal Guidance for Diffusion Models14 Feb
  6. 2022Scalable Diffusion Models with Transformers19 Dec
  7. 2022Why Are Conditional Generative Models Better Than Unconditional Ones?1 Dec
  8. 2022Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models28 Nov
  9. 2022Self-Guided Diffusion Models12 Oct
  10. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidance3 Oct
  11. 2022Classifier-Free Diffusion Guidance26 Jul
  12. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
8 more
  1. 2021Conditional Image Generation with Score-Based Diffusion Models26 Nov
  2. 2021Variational Diffusion Models1 Jul
  3. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  4. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  5. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  7. 2020Denoising Diffusion Implicit Models6 Oct
  8. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 53

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  11. 2014Going Deeper with Convolutionscited by 4,231
  12. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
38 more
  1. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  2. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  3. 2014Conditional Generative Adversarial Netscited by 2,152
  4. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  5. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  6. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  7. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  8. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  9. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 763
  10. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
  11. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  12. 2019Self-labelling via simultaneous clustering and representation learningcited by 525
  13. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  14. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 454
  15. 2020SCAN: Learning to Classify Images without Labelscited by 351
  16. 2020Are we done with ImageNet?cited by 329
  17. 2023Rethinking FID: Towards a Better Evaluation Metric for Image Generationcited by 303
  18. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 215
  19. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 211
  20. 2023Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelscited by 199
  21. 2019Effectively Unbiased FID and Inception Score and where to find themcited by 161
  22. 2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 143
  23. 2022KNN-Diffusion: Image Generation via Large-Scale Retrievalcited by 135
  24. 2023CLIPN for Zero-Shot OOD Detection: Teaching CLIP to Say Nocited by 130
  25. 2021LAFITE: Towards Language-Free Training for Text-to-Image Generationcited by 126
  26. 2021Instance-Conditioned GANcited by 119
  27. 2020Hyper-Parameter Optimization: A Review of Algorithms and Applicationscited by 114
  28. 2023What Do Self-Supervised Vision Transformers Learn?cited by 90
  29. 2023PFGM++: Unlocking the Potential of Physics-Inspired Generative Modelscited by 73
  30. 2021High Fidelity Visualization of What Your Self-Supervised Representation Knows Aboutcited by 64
  31. 2019Do We Train on Test Data? Purging CIFAR of Near-Duplicatescited by 61
  32. 2021Learning Representation for Clustering via Prototype Scattering and Positive Samplingcited by 58
  33. 2020Diverse Image Generation via Self-Conditioned GANscited by 54
  34. 2022Denoising Likelihood Score Matching for Conditional Score-based Data Generationcited by 45
  35. 2023Exploring the Limits of Deep Image Clustering using Pretrained Modelscited by 33
  36. 2021Self-Supervised Classification Networkcited by 27
  37. 2023Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Modelscited by 22
  38. 2022Deep Clustering with Features from Self-Supervised Pretrainingcited by 14

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 0; 2026: 2.

5 papers.

  1. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  2. 2024Guiding a diffusion model using sliding windowscited by 2
  3. 2026Diffusion Models Memorize in Training -- and Generalize in Inferencecited by 2
  4. 2024Scaling Up Deep Clustering Methods Beyond ImageNet-1Kcited by 1
  5. 2026Flow Matching with Optimized Subclass Priors for Medical Image Augmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Conditional Image Synthesis with Diffusion Models: A Survey
  2. 2024On Improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models
  3. 2022On Conditioning the Input Noise for Controlled Image Generation with Diffusion Models
  4. 2023Modulating Pretrained Diffusion Models for Multimodal Image Synthesis
  5. 2023Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models
  6. 2024A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models
  7. 2023Masked Diffusion Models Are Fast Distribution Learners
  8. 2024Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion