# Few-Shot Synthetic Data Generation with Diffusion Models for Downstream Vision Tasks

arXiv: https://arxiv.org/abs/2605.11898 · HTML: https://mlsanity.com/paper/2605.11898/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 12 of its 14 references that are in the map.
- First posted on arXiv 12 May 2026.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Synthetic Data from Diffusion Models Improves ImageNet Classification](https://mlsanity.com/paper/2304.08466/) (2023)
- [Effective Data Augmentation With Diffusion Models](https://mlsanity.com/paper/2302.07944/) (2023)
- [RoentGen: Vision-Language Foundation Model for Chest X-ray Generation](https://mlsanity.com/paper/2211.12737/) (2022)
- [Augmenting medical image classifiers with synthetic data from latent diffusion models](https://mlsanity.com/paper/2308.12453/) (2023)
- [You Don't Have to Be Perfect to Be Amazing: Unveil the Utility of Synthetic Images](https://mlsanity.com/paper/2305.18337/) (2023)
- [Controllable Image Synthesis of Industrial Data Using Stable Diffusion](https://mlsanity.com/paper/2401.03152/) (2024)
- [Are Synthetic Corruptions A Reliable Proxy For Real-World Corruptions?](https://mlsanity.com/paper/2505.04835/) (2025)
- [Bounding Box-Guided Diffusion for Synthesizing Industrial Images and Segmentation Map](https://mlsanity.com/paper/2505.03623/) (2025)

## Abstract (arXiv)

Class imbalance is a persistent challenge in visual recognition, particularly in safety-critical domains where collecting positive examples is expensive and rare events are inherently underrepresented. We propose a lightweight synthetic data augmentation pipeline that fine-tunes a LoRA adapter on as few as 20-50 real images of a rare class and uses a pretrained diffusion model to generate synthetic samples for training. We systematically vary the synthetic-to-real ratio and evaluate the approach across two structurally different domains: chest X-ray pathology classification (NIH ChestX-ray14) and industrial surface crack detection (Magnetic Tile Defect dataset). All evaluations are performed on held-out sets of real images only. Across both domains, synthetic augmentation consistently improves rare-class recall and F1 compared to training with real data alone. Performance improves with moderate synthetic augmentation and shows diminishing returns as the synthetic ratio increases. These results suggest that LoRA-adapted diffusion models provide a simple and scalable mechanism for augmenting rare classes, enabling effective learning in data-scarce scenarios across heterogeneous visual domains.