# Few-shot Image Generation with Diffusion Models

arXiv: https://arxiv.org/abs/2211.03264 · HTML: https://mlsanity.com/paper/2211.03264/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 72 of its 80 references that are in the map.
- First posted on arXiv 7 Nov 2022.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 72)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)

## Built on by (most cited first, 21 of 21)

- [Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.17591/) (2023)
- [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023)
- [The Euclidean Space is Evil: Hyperbolic Attribute Editing for Few-shot Image Generation](https://mlsanity.com/paper/2211.12347/) (2022)
- [A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shot](https://mlsanity.com/paper/2307.14397/) (2023)
- [Phasic Content Fusing Diffusion Model with Directional Distribution Consistency for Few-Shot Model Adaption](https://mlsanity.com/paper/2309.03729/) (2023)
- [Transfer Learning for Diffusion Models](https://mlsanity.com/paper/2405.16876/) (2024)
- [Image Synthesis under Limited Data: A Survey and Taxonomy](https://mlsanity.com/paper/2307.16879/) (2023)
- [Few-Shot Image Generation by Conditional Relaxing Diffusion Inversion](https://mlsanity.com/paper/2407.07249/) (2024)
- [DogFit: Domain-guided Fine-tuning for Efficient Transfer Learning of Diffusion Models](https://mlsanity.com/paper/2508.05685/) (2025)
- [Efficient Transfer Learning in Diffusion Models via Adversarial Noise](https://mlsanity.com/paper/2308.11948/) (2023)
- [Masked Diffusion Language Models with Frequency-Informed Training](https://mlsanity.com/paper/2509.05056/) (2025)
- [Universal Few-Shot Spatial Control for Diffusion Models](https://mlsanity.com/paper/2509.07530/) (2025)
- [Uni-DAD: Unified Distillation and Adaptation of Diffusion Models for Few-step Few-shot Image Generation](https://mlsanity.com/paper/2511.18281/) (2025)
- [Targeted Image Reconstruction by Sampling Pre-trained Diffusion Model](https://mlsanity.com/paper/2301.07557/) (2023)
- [Few-shot 3D Shape Generation](https://mlsanity.com/paper/2305.11664/) (2023)
- [Conditional Distribution Modelling for Few-Shot Image Synthesis with Diffusion Models](https://mlsanity.com/paper/2404.16556/) (2024)
- [DomainGallery: Few-shot Domain-driven Image Generation by Attribute-centric Finetuning](https://mlsanity.com/paper/2411.04571/) (2024)
- [GUST: Quantifying Free-Form Geometric Uncertainty of Metamaterials Using Small Data](https://mlsanity.com/paper/2506.12051/) (2025)
- [SinSEMI: A One-Shot Image Generation Model and Data-Efficient Evaluation Framework for Semiconductor Inspection Equipment](https://mlsanity.com/paper/2511.06740/) (2025)
- [A Turn Toward Better Alignment: Few-Shot Generative Adaptation with Equivariant Feature Rotation](https://mlsanity.com/paper/2512.21174/) (2025)
- [Improving Combined Detection and Classification of TEM Defects via Mask-Conditioned Latent Diffusion Augmentation](https://mlsanity.com/paper/2606.02532/) (2026)

## Abstract (arXiv)

Denoising diffusion probabilistic models (DDPMs) have been proven capable of synthesizing high-quality images with remarkable diversity when trained on large amounts of data. However, to our knowledge, few-shot image generation tasks have yet to be studied with DDPM-based approaches. Modern approaches are mainly built on Generative Adversarial Networks (GANs) and adapt models pre-trained on large source domains to target domains using a few available samples. In this paper, we make the first attempt to study when do DDPMs overfit and suffer severe diversity degradation as training data become scarce. Then we fine-tune DDPMs pre-trained on large source domains to solve the overfitting problem when training data is limited. Although the directly fine-tuned models accelerate convergence and improve generation quality and diversity compared with training from scratch, they still fail to retain some diverse features and can only produce coarse images. Therefore, we design a DDPM pairwise adaptation (DDPM-PA) approach to optimize few-shot DDPM domain adaptation. DDPM-PA efficiently preserves information learned from source domains by keeping the relative pairwise distances between generated samples during adaptation. Besides, DDPM-PA enhances the learning of high-frequency details from source models and limited training data. DDPM-PA further improves generation quality and diversity and achieves results better than current state-of-the-art GAN-based approaches. We demonstrate the effectiveness of our approach on a series of few-shot image generation tasks qualitatively and quantitatively.