mlsanityData to 7 Oct 2026

Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels

arXiv 2302.10586 ↗First posted 21 Feb 2023cs.CV cs.AI cs.LGcited by 37

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

In an effort to further advance semi-supervised generative and classification tasks, we propose a simple yet effective training strategy called dual pseudo training (DPT), built upon strong semi-supervised learners and diffusion models. DPT operates in three stages: training a classifier on partially labeled data to predict pseudo-labels; training a conditional generative model using these pseudo-labels to generate pseudo images; and retraining the classifier with a mix of real and pseudo images. Empirically, DPT consistently achieves SOTA performance of semi-supervised generation and classification across various settings. In particular, with one or two labels per class, DPT achieves a Fr'echet Inception Distance (FID) score of 3.08 or 2.52 on ImageNet 256x256. Besides, DPT outperforms competitive semi-supervised baselines substantially on ImageNet classification tasks, achieving top-1 accuracies of 59.0 (+2.8), 69.5 (+3.0), and 74.4 (+2.0) with one, two, or five labels per class, respectively. Notably, our results demonstrate that diffusion can generate realistic images with only a few labels (e.g., <0.1%) and generative augmentation remains viable for semi-supervised classification. Our code is available at https://github.com/ML-GSAI/DPT.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 95 of its 119 references that are in the map. First posted on arXiv 21 Feb 2023.

Builds on

95 of 119 references.

Closest ancestors · 20

Same direction, newest first

  1. 2022Scalable Diffusion Models with Transformers19 Dec
  2. 2022Why Are Conditional Generative Models Better Than Unconditional Ones?1 Dec
  3. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  4. 2022Self-Guided Diffusion Models12 Oct
  5. 2022Classifier-Free Diffusion Guidance26 Jul
  6. 2022EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations14 Jul
  7. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models15 Jun
  8. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  10. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  11. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  12. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
8 more
  1. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  2. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models6 Aug
  3. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  4. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  5. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  7. 2020Denoising Diffusion Implicit Models6 Oct
  8. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 75

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  11. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  12. 2017Squeeze-and-Excitation Networkscited by 4,763
38 more
  1. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  2. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  6. 2016Improved Techniques for Training GANscited by 2,938
  7. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  9. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  10. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  11. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  12. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  13. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  14. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  15. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  16. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  17. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  18. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  19. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  20. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  21. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  22. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 666
  23. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 537
  24. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
  25. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  26. 2020Meta Pseudo Labelscited by 372
  27. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  28. 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
  29. 2019Label Propagation for Deep Semi-supervised Learningcited by 287
  30. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  31. 2021Generating Images with Sparse Representationscited by 262
  32. 2015Semi-Supervised Learning with Ladder Networkscited by 246
  33. 2022FreeMatch: Self-adaptive Thresholding for Semi-supervised Learningcited by 232
  34. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 211
  35. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
  36. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
  37. 2020CoMatch: Semi-supervised Learning with Contrastive Graph Regularizationcited by 167
  38. 2021Dash: Semi-Supervised Learning with Dynamic Thresholdingcited by 147

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 11; 2025: 10; 2026: 5.

37 papers.

Sort:

By area:

  1. 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
  2. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  3. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  4. 2023MissDiff: Training Diffusion Models on Tabular Data with Missing Valuescited by 38
  5. 2023Fill-Up: Balancing Long-Tailed Data with Generative Modelscited by 34
  6. 2023Feedback-guided Data Synthesis for Imbalanced Classificationcited by 33
  7. 2024Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Modelcited by 27
  8. 2023Toward Understanding Generative Data Augmentationcited by 25
  9. 2023ScaleLong: Towards More Stable Training of Diffusion Model via Scaling Network Long Skip Connectioncited by 25
  10. 2023Boosting Human-Object Interaction Detection with Text-to-Image Diffusion Modelcited by 21
  11. 2023AI-Generated Images as Data Source: The Dawn of Synthetic Eracited by 13
  12. 2024MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generationcited by 10
25 more
  1. 2023Guided Diffusion from Self-Supervised Diffusion Featurescited by 9
  2. 2025Generate, Refine, and Encode: Leveraging Synthesized Novel Samples for On-the-Fly Fine-Grained Category Discoverycited by 9
  3. 2024Cross-Domain Graph Data Scaling: A Showcase with Diffusion Modelscited by 8
  4. 2024Dual Conditional Diffusion Models for Sequential Recommendationcited by 8
  5. 2023DiffAug: A Diffuse-and-Denoise Augmentation for Training Robust Classifierscited by 5
  6. 2024Are Images Indistinguishable to Humans Also Indistinguishable to Classifiers?cited by 5
  7. 2024Diffusion-Refined VQA Annotations for Semi-Supervised Gaze Followingcited by 4
  8. 2025MatWheel: Addressing Data Scarcity in Materials Science Through Synthetic Datacited by 2
  9. 2025DIVE: Inverting Conditional Diffusion Models for Discriminative Taskscited by 2
  10. 2025Unsupervised Learning for Class Distribution Mismatchcited by 2
  11. 2025Robust Learning of Diffusion Models with Extremely Noisy Conditionscited by 2
  12. 2025Enhancing Semi-supervised Learning with Zero-shot Pseudolabelscited by 1
  13. 2025SARFormer -- An Acquisition Parameter Aware Vision Transformer for Synthetic Aperture Radar Datacited by 1
  14. 2025LoFT: LoRA-fused Training Dataset Generation with Few-shot Guidancecited by 1
  15. 2025Generative Data Augmentation for Object Point Cloud Segmentationcited by 1
  16. 2024Prompt-Driven Feature Diffusion for Open-World Semi-Supervised Learningcited by 0
  17. 2024LayerMatch: Do Pseudo-labels Benefit All Layers?cited by 0
  18. 2024Input-Adaptive Generative Dynamics in Diffusion Modelscited by 0
  19. 2024SemiDFL: A Semi-Supervised Paradigm for Decentralized Federated Learningcited by 0
  20. 2025Vision Generalist Model: A Surveycited by 0
  21. 2026Mitigating Instance Entanglement in Instance-Dependent Partial Label Learningcited by 0
  22. 2026What Makes Synthetic Data Effective in Image Segmentationcited by 0
  23. 2026Multi-Objective Learning for Diffusion Models: A Statistical Theory under Semi-Supervised Learningcited by 0
  24. 2026Semi-Supervised Conditional Diffusion via Label Augmentationcited by 0
  25. 2026Revisiting Spectral Representations in Generative Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Synthetic Data from Diffusion Models Improves ImageNet Classification
  2. 2023Denoising Diffusion Autoencoders are Unified Self-supervised Learners
  3. 2023Conditional Generation from Unconditional Diffusion Models using Denoiser Representations
  4. 2024On Improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models
  5. 2023Diffusion Models Beat GANs on Image Classification
  6. 2023Your Diffusion Model is Secretly a Zero-Shot Classifier
  7. 2023Learning Data Representations with Joint Diffusion Models
  8. 2025Learning Robust Diffusion Models from Imprecise Supervision