# Expanding Small-Scale Datasets with Guided Imagination

arXiv: https://arxiv.org/abs/2211.13976 · HTML: https://mlsanity.com/paper/2211.13976/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Builds on 80 of its 125 references that are in the map.
- First posted on arXiv 25 Nov 2022.

## Builds on (most cited first, 30 of 80)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)

## Built on by (most cited first, 30 of 62)

- [Matrix-game 2.0: An open-source real-time and streaming interactive world model](https://mlsanity.com/paper/2508.13009/) (2025)
- [Leaving Reality to Imagination: Robust Classification via Generated Datasets](https://mlsanity.com/paper/2302.02503/) (2023)
- [Dream the Impossible: Outlier Imagination with Diffusion Models](https://mlsanity.com/paper/2309.13415/) (2023)
- [Dataset Quantization](https://mlsanity.com/paper/2308.10524/) (2023)
- [Does Graph Distillation See Like Vision Dataset Counterpart?](https://mlsanity.com/paper/2310.09192/) (2023)
- [A Comprehensive Survey on Data Augmentation](https://mlsanity.com/paper/2405.09591/) (2024)
- [MEMO: Memory-Guided Diffusion for Expressive Talking Video Generation](https://mlsanity.com/paper/2412.04448/) (2024)
- [Visual Large Language Models for Generalized and Specialized Applications](https://mlsanity.com/paper/2501.02765/) (2025)
- [DreamDA: Generative Data Augmentation with Diffusion Models](https://mlsanity.com/paper/2403.12803/) (2024)
- [Would Deep Generative Models Amplify Bias in Future Models?](https://mlsanity.com/paper/2404.03242/) (2024)
- [Implicit Concept Removal of Diffusion Models](https://mlsanity.com/paper/2310.05873/) (2023)
- [Distribution-Aware Data Expansion with Diffusion Models](https://mlsanity.com/paper/2403.06741/) (2024)
- [Improving the Scaling Laws of Synthetic Data with Deliberate Practice](https://mlsanity.com/paper/2502.15588/) (2025)
- [Semantic-Guided Generative Image Augmentation Method with Diffusion Models for Image Classification](https://mlsanity.com/paper/2302.02070/) (2023)
- [TP2O: Creative Text Pair-to-Object Generation using Balance Swap-Sampling](https://mlsanity.com/paper/2310.01819/) (2023)
- [HiPA: Enabling One-Step Text-to-Image Diffusion Models via High-Frequency-Promoting Adaptation](https://mlsanity.com/paper/2311.18158/) (2023)
- [MacDiff: Unified Skeleton Modeling with Masked Conditional Diffusion](https://mlsanity.com/paper/2409.10473/) (2024)
- [Image retrieval outperforms diffusion models on data augmentation](https://mlsanity.com/paper/2304.10253/) (2023)
- [DriveGEN: Generalized and Robust 3D Detection in Driving via Controllable Text-to-Image Diffusion Generation](https://mlsanity.com/paper/2503.11122/) (2025)
- [Can pre-trained models assist in dataset distillation?](https://mlsanity.com/paper/2310.03295/) (2023)
- [Inversion Circle Interpolation: Diffusion-based Image Augmentation for Data-scarce Classification](https://mlsanity.com/paper/2408.16266/) (2024)
- [Human-Guided Image Generation for Expanding Small-Scale Training Image Datasets](https://mlsanity.com/paper/2412.16839/) (2024)
- [Beyond Isolated Words: Diffusion Brush for Handwritten Text-Line Generation](https://mlsanity.com/paper/2508.03256/) (2025)
- [GenView: Enhancing View Quality with Pretrained Generative Model for Self-Supervised Learning](https://mlsanity.com/paper/2403.12003/) (2024)
- [Forte : Finding Outliers with Representation Typicality Estimation](https://mlsanity.com/paper/2410.01322/) (2024)
- [PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning](https://mlsanity.com/paper/2310.09183/) (2023)
- [One-to-Normal: Anomaly Personalization for Few-shot Anomaly Detection](https://mlsanity.com/paper/2502.01201/) (2025)
- [Generative Active Learning for Long-tail Trajectory Prediction via Controllable Diffusion Model](https://mlsanity.com/paper/2507.22615/) (2025)
- [Increasing the Utility of Synthetic Images through Chamfer Guidance](https://mlsanity.com/paper/2508.10631/) (2025)
- [ReCon: Region-Controllable Data Augmentation with Rectification and Alignment for Object Detection](https://mlsanity.com/paper/2510.15783/) (2025)

## Abstract (arXiv)

The power of DNNs relies heavily on the quantity and quality of training data. However, collecting and annotating data on a large scale is often expensive and time-consuming. To address this issue, we explore a new task, termed dataset expansion, aimed at expanding a ready-to-use small dataset by automatically creating new labeled samples. To this end, we present a Guided Imagination Framework (GIF) that leverages cutting-edge generative models like DALL-E2 and Stable Diffusion (SD) to "imagine" and create informative new data from the input seed data. Specifically, GIF conducts data imagination by optimizing the latent features of the seed data in the semantically meaningful space of the prior model, resulting in the creation of photo-realistic images with new content. To guide the imagination towards creating informative samples for model training, we introduce two key criteria, i.e., class-maintained information boosting and sample diversity promotion. These criteria are verified to be essential for effective dataset expansion: GIF-SD obtains 13.5% higher model accuracy on natural image datasets than unguided expansion with SD. With these essential criteria, GIF successfully expands small datasets in various scenarios, boosting model accuracy by 36.9% on average over six natural image datasets and by 13.5% on average over three medical datasets. The source code is available at https://github.com/Vanint/DatasetExpansion.