mlsanityData to 7 Oct 2026

Synthetic Data from Diffusion Models Improves ImageNet Classification

arXiv 2304.08466 ↗First posted 17 Apr 2023cs.CV cs.AI cs.CLcited by 326

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Deep generative models are becoming increasingly powerful, now generating diverse high fidelity photo-realistic samples given text prompts. Have they reached the point where models of natural images can be used for generative data augmentation, helping to improve challenging discriminative tasks? We show that large-scale text-to image diffusion models can be fine-tuned to produce class conditional models with SOTA FID (1.76 at 256x256 resolution) and Inception Score (239 at 256x256). The model also yields a new SOTA in Classification Accuracy Scores (64.96 for 256x256 generative samples, improving to 69.24 for 1024x1024 samples). Augmenting the ImageNet training set with samples from the resulting models yields significant improvements in ImageNet classification accuracy over strong ResNet and Vision Transformer baselines.

Key facts. Cited by 326 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 62 of its 73 references that are in the map. First posted on arXiv 17 Apr 2023.

Builds on

62 of 73 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Effective Data Augmentation With Diffusion Models7 Feb
  2. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting13 Dec
  3. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  7. 2021Palette: Image-to-Image Diffusion Models10 Nov

Other references · 55

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  8. 2020Denoising Diffusion Implicit Modelscited by 9,655
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  10. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  11. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  12. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
38 more
  1. 2022Classifier-Free Diffusion Guidancecited by 5,717
  2. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  3. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  5. 2016Improved Techniques for Training GANscited by 2,938
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  7. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  8. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  9. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  10. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  11. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  12. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  13. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  14. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  15. 2021Scaling Vision Transformerscited by 993
  16. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  17. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  18. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  19. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  20. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  21. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  22. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  23. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 413
  24. 2017Learning from Synthetic Humanscited by 358
  25. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  26. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 285
  27. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
  28. 2021Improving Robustness using Generated Datacited by 233
  29. 2023On the Importance of Noise Scheduling for Diffusion Modelscited by 193
  30. 2022Scalable Adaptive Computation for Iterative Generationcited by 190
  31. 2021Revisiting ResNets: Improved Training and Scaling Strategiescited by 158
  32. 2021Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalizationcited by 141
  33. 2019Classification Accuracy Score for Conditional Generative Modelscited by 134
  34. 2018Learning Semantic Segmentation from Synthetic Data: A Geometrically Guided Input-Output Adaptation Approachcited by 119
  35. 2021AutoFlow: Learning a Better Training Set for Optical Flowcited by 102
  36. 2022BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotationscited by 99
  37. 2022Better plain ViT baselines for ImageNet-1kcited by 97
  38. 2023Leaving Reality to Imagination: Robust Classification via Generated Datasetscited by 91

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 85; 2024: 129; 2025: 72; 2026: 40.

326 papers.

Sort:

By area:

  1. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
  2. 2023Self-Consuming Generative Models Go MADcited by 239
  3. 2023StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learnerscited by 177
  4. 2024Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modelingcited by 156
  5. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 127
  6. 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
  7. 2023Scaling Laws of Synthetic Images for Model Training ... for Nowcited by 107
  8. 2024URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Imagescited by 106
  9. 2023Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentationcited by 100
  10. 2023Improving Multimodal Datasets with Image Captioningcited by 93
  11. 2023Efficient Dataset Distillation via Minimax Diffusioncited by 81
  12. 2023Generating images of rare concepts using pre-trained diffusion modelscited by 78
38 more
  1. 2024DiffuseMix: Label-Preserving Data Augmentation with Diffusion Modelscited by 78
  2. 2023Dream the Impossible: Outlier Imagination with Diffusion Modelscited by 77
  3. 2023Towards Unified Text-based Person Retrieval: A Large-scale Multi-Attribute and Language Search Benchmarkcited by 75
  4. 2023Learning Vision from Models Rivals Learning Vision from Datacited by 71
  5. 2023Generative AI for Medical Imaging: extending the MONAI Frameworkcited by 68
  6. 2024SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?cited by 64
  7. 2024DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curationcited by 64
  8. 2023FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Modelscited by 61
  9. 2023Towards Understanding the Interplay of Generative Artificial Intelligence and the Internetcited by 54
  10. 2023PathLDM: Text conditioned Latent Diffusion Model for Histopathologycited by 51
  11. 2023Training on Thin Air: Improve Image Classification with Generated Datacited by 50
  12. 2023Analyzing Bias in Diffusion-based Face Generation Modelscited by 45
  13. 2023DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Controlcited by 44
  14. 2024Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verificationcited by 43
  15. 2023The Journey, Not the Destination: How Data Guides Diffusion Modelscited by 42
  16. 2024SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentationcited by 41
  17. 2024RF-Diffusion: Radio Signal Generation via Time-Frequency Diffusioncited by 41
  18. 2024MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generationcited by 41
  19. 2024Is Synthetic Data all We Need? Benchmarking the Robustness of Models Trained with Synthetic Imagescited by 41
  20. 2023MissDiff: Training Diffusion Models on Tabular Data with Missing Valuescited by 38
  21. 2024Diffusion Feedback Helps CLIP See Bettercited by 37
  22. 2024A Vision Check-up for Language Modelscited by 36
  23. 2023Real-Fake: Effective Training Data Synthesis Through Distribution Matchingcited by 35
  24. 2024Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferencescited by 35
  25. 2023Fill-Up: Balancing Long-Tailed Data with Generative Modelscited by 34
  26. 2024Self-Correcting Self-Consuming Loops for Generative Model Trainingcited by 34
  27. 2024Enhance Image Classification via Inter-Class Image Mixup with Diffusion Modelcited by 34
  28. 2023Feedback-guided Data Synthesis for Imbalanced Classificationcited by 33
  29. 2024SubjectDrive: Scaling Generative Data in Autonomous Driving via Subject Controlcited by 32
  30. 2024Towards Theoretical Understandings of Self-Consuming Generative Modelscited by 28
  31. 2024DataDream: Few-shot Guided Dataset Generationcited by 28
  32. 2023Diffusion-TTA: Test-time Adaptation of Discriminative Models via Generative Feedbackcited by 27
  33. 2023DriveSceneGen: Generating Diverse and Realistic Driving Scenarios from Scratchcited by 26
  34. 2023Toward Understanding Generative Data Augmentationcited by 25
  35. 2024Do Generated Data Always Help Contrastive Learning?cited by 25
  36. 2024DreamDA: Generative Data Augmentation with Diffusion Modelscited by 24
  37. 2024Would Deep Generative Models Amplify Bias in Future Models?cited by 24
  38. 2024DiffClass: Diffusion-Based Class Incremental Learningcited by 23

50 most cited of 326 All 326 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Text-to-Image Diffusion Models are Zero-Shot Classifiers
  2. 2023Your Diffusion Model is Secretly a Zero-Shot Classifier
  3. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clones
  4. 2023Diffusion Models Beat GANs on Image Classification
  5. 2023Discriminative Class Tokens for Text-to-Image Diffusion Models
  6. 2024On Improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models
  7. 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesis
  8. 2026When Pretty Isn't Useful: Investigating Why Modern Text-to-Image Models Fail as Reliable Training Data Generators