# DataDream: Few-shot Guided Dataset Generation

arXiv: https://arxiv.org/abs/2407.10910 · HTML: https://mlsanity.com/paper/2407.10910/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 36 of its 50 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 36)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [Synthetic Data from Diffusion Models Improves ImageNet Classification](https://mlsanity.com/paper/2304.08466/) (2023)
- [Is synthetic data from generative models ready for image recognition?](https://mlsanity.com/paper/2210.07574/) (2022)
- [Self-Consuming Generative Models Go MAD](https://mlsanity.com/paper/2307.01850/) (2023)
- [More Control for Free! Image Synthesis with Semantic Diffusion Guidance](https://mlsanity.com/paper/2112.05744/) (2021)
- [StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners](https://mlsanity.com/paper/2306.00984/) (2023)
- [Fake it till you make it: Learning transferable representations from synthetic ImageNet clones](https://mlsanity.com/paper/2212.08420/) (2022)
- [Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentation](https://mlsanity.com/paper/2305.16289/) (2023)
- [Leaving Reality to Imagination: Robust Classification via Generated Datasets](https://mlsanity.com/paper/2302.02503/) (2023)
- [Diversity is Definitely Needed: Improving Model-Agnostic Zero-shot Classification via Stable Diffusion](https://mlsanity.com/paper/2302.03298/) (2023)
- [SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?](https://mlsanity.com/paper/2402.01832/) (2024)

## Built on by (most cited first, 29 of 29)

- [A Large Scale Analysis of Gender Biases in Text-to-Image Generative Models](https://mlsanity.com/paper/2503.23398/) (2025)
- [CoEmoGen: Towards Semantically-Coherent and Scalable Emotional Image Content Generation](https://mlsanity.com/paper/2508.03535/) (2025)
- [Diffusion Curriculum: Synthetic-to-Real Data Curriculum via Image-Guided Diffusion](https://mlsanity.com/paper/2410.13674/) (2024)
- [SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutions](https://mlsanity.com/paper/2507.23784/) (2025)
- [Exploring Cross-Modal Flows for Few-Shot Learning](https://mlsanity.com/paper/2510.14543/) (2025)
- [Does Feasibility Matter? Understanding the Impact of Feasibility on Synthetic Training Data](https://mlsanity.com/paper/2505.10551/) (2025)
- [Context-Aware Multimodal Pretraining](https://mlsanity.com/paper/2411.15099/) (2024)
- [Provably Improving Generalization of Few-Shot Models with Synthetic Data](https://mlsanity.com/paper/2505.24190/) (2025)
- [Training-Free Synthetic Data Generation with Dual IP-Adapter Guidance](https://mlsanity.com/paper/2509.22635/) (2025)
- [Stitch: Training-Free Position Control in Multimodal Diffusion Transformers](https://mlsanity.com/paper/2509.26644/) (2025)
- [Active Learning Inspired ControlNet Guidance for Augmenting Semantic Segmentation Datasets](https://mlsanity.com/paper/2503.09221/) (2025)
- [Improving Knowledge Distillation Under Unknown Covariate Shift Through Confidence-Guided Data Augmentation](https://mlsanity.com/paper/2506.02294/) (2025)
- [CNS-Bench: Benchmarking Image Classifier Robustness Under Continuous Nuisance Shifts](https://mlsanity.com/paper/2507.17651/) (2025)
- [UtilGen: Utility-Centric Generative Data Augmentation with Dual-Level Task Adaptation](https://mlsanity.com/paper/2510.24262/) (2025)
- [Envisioning Beyond the Few: Disentangled Semantics and Primitives for Few-Shot Atypical Layout-to-Image Generation](https://mlsanity.com/paper/2605.31266/) (2026)
- [On the Difficulty of Learning a Meta-network for Training Data Selection](https://mlsanity.com/paper/2606.00571/) (2026)
- [LoFT: LoRA-fused Training Dataset Generation with Few-shot Guidance](https://mlsanity.com/paper/2505.11703/) (2025)
- [CytoDiff: AI-Driven Cytomorphology Image Synthesis for Medical Diagnostics](https://mlsanity.com/paper/2507.05063/) (2025)
- [Beyond Words: Augmenting Discriminative Richness via Diffusions in Unsupervised Prompt Learning](https://mlsanity.com/paper/2504.11930/) (2025)
- [Masked Language Prompting for Generative Data Augmentation in Few-shot Fashion Style Recognition](https://mlsanity.com/paper/2504.19455/) (2025)
- [AdaptaGen: Domain-Specific Image Generation through Hierarchical Semantic Optimization Framework](https://mlsanity.com/paper/2507.05621/) (2025)
- [Beyond Objects: Contextual Synthetic Data Generation for Fine-Grained Classification](https://mlsanity.com/paper/2510.24078/) (2025)
- [ChimeraLoRA: Multi-Head LoRA-Guided Synthetic Datasets](https://mlsanity.com/paper/2602.19708/) (2026)
- [Diffusion-Based Data Augmentation for Image Recognition: A Systematic Analysis and Evaluation](https://mlsanity.com/paper/2603.08364/) (2026)
- [LiBaGS: Lightweight Boundary Gap Synthesis for Targeted Synthetic Data Selection](https://mlsanity.com/paper/2605.11231/) (2026)
- [Post-Generation Curation of Synthetic Images via Homogeneous-Heterogeneous Splitting](https://mlsanity.com/paper/2607.02637/) (2026)
- [Steering Diffusion Models via Class-Contrastive Influence for Few-Shot Medical Classification](https://mlsanity.com/paper/2607.12464/) (2026)
- [PCFlow: Physics-Conditioned Flow Matching for GPR B-Scan Image Synthesis](https://mlsanity.com/paper/2609.07300/) (2026)
- [Discovering Undisclosed Concepts in LoRA Adapters for Diffusion-Based AIGC Models](https://mlsanity.com/paper/2610.18990/) (2026)

## Abstract (arXiv)

While text-to-image diffusion models have been shown to achieve state-of-the-art results in image synthesis, they have yet to prove their effectiveness in downstream applications. Previous work has proposed to generate data for image classifier training given limited real data access. However, these methods struggle to generate in-distribution images or depict fine-grained features, thereby hindering the generalization of classification models trained on synthetic datasets. We propose DataDream, a framework for synthesizing classification datasets that more faithfully represents the real data distribution when guided by few-shot examples of the target classes. DataDream fine-tunes LoRA weights for the image generation model on the few real images before generating the training data using the adapted model. We then fine-tune LoRA weights for CLIP using the synthetic data to improve downstream image classification over previous approaches on a large variety of datasets. We demonstrate the efficacy of DataDream through extensive experiments, surpassing state-of-the-art classification accuracy with few-shot data across 7 out of 10 datasets, while being competitive on the other 3. Additionally, we provide insights into the impact of various factors, such as the number of real-shot and generated images as well as the fine-tuning compute on model performance. The code is available at https://github.com/ExplainableML/DataDream.