mlsanityData to 7 Oct 2026

Training on Thin Air: Improve Image Classification with Generated Data

arXiv 2305.15316 ↗First posted 24 May 2023cs.CV cs.LGcited by 50

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Acquiring high-quality data for training discriminative models is a crucial yet challenging aspect of building effective predictive systems. In this paper, we present Diffusion Inversion, a simple yet effective method that leverages the pre-trained generative model, Stable Diffusion, to generate diverse, high-quality training data for image classification. Our approach captures the original data distribution and ensures data coverage by inverting images to the latent space of Stable Diffusion, and generates diverse novel training images by conditioning the generative model on noisy versions of these vectors. We identify three key components that allow our generated images to successfully supplant the original dataset, leading to a 2-3x enhancement in sample complexity and a 6.5x decrease in sampling time. Moreover, our approach consistently outperforms generic prompt-based steering methods and KNN retrieval baseline across a wide range of datasets. Additionally, we demonstrate the compatibility of our approach with widely-used data augmentation techniques, as well as the reliability of the generated data in supporting various neural architectures and enhancing few-shot learning.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 57 of its 67 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

57 of 67 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Synthetic Data from Diffusion Models Improves ImageNet Classification17 Apr
  2. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clones16 Dec
  3. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  7. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  10. 2022Generative Adversarial Networkscited by 5,603
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  12. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
38 more
  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  5. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  6. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  7. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  8. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  9. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  10. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  11. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  12. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  13. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  14. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  15. 2022On Distillation of Guided Diffusion Modelscited by 721
  16. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 682
  17. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  18. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  19. 2021GAN Inversion: A Surveycited by 366
  20. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 361
  21. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 355
  22. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  23. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 326
  24. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  25. 2016Learning from Simulated and Unsupervised Images through Adversarial Trainingcited by 287
  26. 2017Data Augmentation Generative Adversarial Networkscited by 278
  27. 2021Improving Robustness using Generated Datacited by 233
  28. 2016Inverting The Generator Of A Generative Adversarial Networkcited by 140
  29. 2019Classification Accuracy Score for Conditional Generative Modelscited by 134
  30. 2020Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustnesscited by 121
  31. 2023Leaving Reality to Imagination: Robust Classification via Generated Datasetscited by 91
  32. 2021Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Developmentcited by 85
  33. 2020StyleGAN2 Distillation for Feed-forward Image Manipulationcited by 78
  34. 2022Synthesizing Informative Training Samples with GANcited by 77
  35. 2021Ensembling with Deep Generative Viewscited by 53
  36. 2020DeepFake Detection: Current Challenges and Next Stepscited by 44
  37. 2017Data Augmentation in Emotion Classification Using Generative Adversarial Networkscited by 19
  38. 2022How good are deep models in understanding the generated images?cited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 21; 2025: 15; 2026: 5.

50 papers.

Sort:

By area:

  1. 2023Efficient Dataset Distillation via Minimax Diffusioncited by 81
  2. 2023Learning Vision from Models Rivals Learning Vision from Datacited by 71
  3. 2024SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?cited by 64
  4. 2024Is Synthetic Data all We Need? Benchmarking the Robustness of Models Trained with Synthetic Imagescited by 41
  5. 2023Real-Fake: Effective Training Data Synthesis Through Distribution Matchingcited by 35
  6. 2024DataDream: Few-shot Guided Dataset Generationcited by 28
  7. 2024DreamDA: Generative Data Augmentation with Diffusion Modelscited by 24
  8. 2023Federated Generative Learning with Foundation Modelscited by 23
  9. 2024Distribution-Aware Data Expansion with Diffusion Modelscited by 20
  10. 2023HiPA: Enabling One-Step Text-to-Image Diffusion Models via High-Frequency-Promoting Adaptationcited by 16
  11. 2023AI-Generated Images as Data Source: The Dawn of Synthetic Eracited by 13
  12. 2023Diversified in-domain synthesis with efficient fine-tuning for few-shot classificationcited by 11
38 more
  1. 2024Inversion Circle Interpolation: Diffusion-based Image Augmentation for Data-scarce Classificationcited by 11
  2. 2024Muffin or Chihuahua? Challenging Multimodal Large Language Models with Multipanel VQAcited by 9
  3. 2024Personalized Representation from Personalized Generationcited by 9
  4. 2025Generate, Refine, and Encode: Leveraging Synthesized Novel Samples for On-the-Fly Fine-Grained Category Discoverycited by 9
  5. 2024Active Generation for Image Classificationcited by 8
  6. 2024GenView: Enhancing View Quality with Pretrained Generative Model for Self-Supervised Learningcited by 8
  7. 2024Can Medical Vision-Language Pre-training Succeed with Purely Synthetic Data?cited by 6
  8. 2024Decoupled Data Augmentation for Improving Image Classificationcited by 6
  9. 2026When Pretty Isn't Useful: Investigating Why Modern Text-to-Image Models Fail as Reliable Training Data Generatorscited by 6
  10. 2024Are Images Indistinguishable to Humans Also Indistinguishable to Classifiers?cited by 5
  11. 2023A Unified Framework for Generative Data Augmentation: A Comprehensive Surveycited by 4
  12. 2024Robust Disaster Assessment from Aerial Imagery Using Text-to-Image Synthetic Datacited by 4
  13. 2025Training Data Provenance Verification: Did Your Model Use Synthetic Data from My Generative Model for Training?cited by 4
  14. 2025Diffusion on Graph: Augmentation of Graph Structure for Node Classificationcited by 4
  15. 2025Generative Data Mining with Longtail-Guided Diffusioncited by 3
  16. 2025Training-Free Synthetic Data Generation with Dual IP-Adapter Guidancecited by 3
  17. 2024Difficulty Controlled Diffusion Model for Synthesizing Effective Training Datacited by 2
  18. 2024Merging synthetic and real embryo data for advanced AI predictionscited by 2
  19. 2025D-Feat Occlusions: Diffusion Features for Robustness to Partial Visual Occlusions in Object Recognitioncited by 2
  20. 2025The Role of Video Generation in Enhancing Data-Limited Action Understandingcited by 2
  21. 2025Do We Need All the Synthetic Data? Targeted Image Augmentation via Diffusion Modelscited by 2
  22. 2023Image Captions are Natural Prompts for Text-to-Image Modelscited by 1
  23. 2024Boosting Few-Shot Detection with Large Language Models and Layout-to-Image Synthesiscited by 1
  24. 2024Will the Inclusion of Generated Data Amplify Bias Across Generations in Future Image Classification Models?cited by 1
  25. 2025Can Text-to-Video Generation help Video-Language Alignment?cited by 1
  26. 2025SPIE: Semantic and Structural Post-Training of Image Editing Diffusion Models with AI feedbackcited by 1
  27. 2025LoFT: LoRA-fused Training Dataset Generation with Few-shot Guidancecited by 1
  28. 2025SoK: Can Synthetic Images Replace Real Data? A Survey of Utility and Privacy of Synthetic Image Generationcited by 1
  29. 2024On the Surprising Efficacy of Distillation as an Alternative to Pre-Training Small Modelscited by 0
  30. 2024Reinforcement Learning with Generative Models for Compact Support Setscited by 0
  31. 2024If you can describe it, they can see it: Cross-Modal Learning of Visual Concepts from Textual Descriptionscited by 0
  32. 2025SpinML: Customized Synthetic Data Generation for Private Training of Specialized ML Modelscited by 0
  33. 2025Salient Concept-Aware Generative Data Augmentationcited by 0
  34. 2025Beyond Objects: Contextual Synthetic Data Generation for Fine-Grained Classificationcited by 0
  35. 2026VisionFoundry: Teaching VLMs Visual Perception with Synthetic Imagescited by 0
  36. 2026Towards Controllable Image Generation through Representation-Conditioned Diffusion Modelscited by 0
  37. 2026Post-Generation Curation of Synthetic Images via Homogeneous-Heterogeneous Splittingcited by 0
  38. 2026Breaking the Synthetic-Real Domain Shortcut for Training-Free Generative Replay-based Class Incremental Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Representation-Conditioned Diffusion Models for Guided Training Data Generation
  2. 2023Reverse Stable Diffusion: What prompt was used to generate this image?
  3. 2022Evaluating a Synthetic Image Dataset Generated with Stable Diffusion
  4. 2024On Improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models
  5. 2023Image retrieval outperforms diffusion models on data augmentation
  6. 2023Conditional Generation from Unconditional Diffusion Models using Denoiser Representations
  7. 2025Accelerating Diffusion Model Training under Minimal Budgets: A Condensation-Based Perspective
  8. 2024Stable Diffusion Dataset Generation for Downstream Classification Tasks