# Few-Shot Diffusion Models

arXiv: https://arxiv.org/abs/2205.15463 · HTML: https://mlsanity.com/paper/2205.15463/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 95 of its 124 references that are in the map.
- First posted on arXiv 30 May 2022.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 95)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)

## Built on by (most cited first, 30 of 38)

- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.17591/) (2023)
- [A Survey on Generative Diffusion Model](https://mlsanity.com/paper/2209.02646/) (2022)
- [PTQD: Accurate Post-Training Quantization for Diffusion Models](https://mlsanity.com/paper/2305.10657/) (2023)
- [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023)
- [Aligning Optimization Trajectories with Diffusion Models for Constrained Design Generation](https://mlsanity.com/paper/2305.18470/) (2023)
- [Efficient Diffusion Models for Vision: A Survey](https://mlsanity.com/paper/2210.09292/) (2022)
- [Class-Balancing Diffusion Models](https://mlsanity.com/paper/2305.00562/) (2023)
- [Self-Guided Diffusion Models](https://mlsanity.com/paper/2210.06462/) (2022)
- [Group Diffusion Transformers are Unsupervised Multitask Learners](https://mlsanity.com/paper/2410.15027/) (2024)
- [Few-shot Image Generation with Diffusion Models](https://mlsanity.com/paper/2211.03264/) (2022)
- [Diffusion Model for Dense Matching](https://mlsanity.com/paper/2305.19094/) (2023)
- [ADIR: Adaptive Diffusion for Image Reconstruction](https://mlsanity.com/paper/2212.03221/) (2022)
- [A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shot](https://mlsanity.com/paper/2307.14397/) (2023)
- [DiffusionPen: Towards Controlling the Style of Handwritten Text Generation](https://mlsanity.com/paper/2409.06065/) (2024)
- [Diffusion Models in Bioinformatics: A New Wave of Deep Learning Revolution in Action](https://mlsanity.com/paper/2302.10907/) (2023)
- [DomainStudio: Fine-Tuning Diffusion Models for Domain-Driven Image Generation using Limited Data](https://mlsanity.com/paper/2306.14153/) (2023)
- [Data Augmentation for Seizure Prediction with Generative Diffusion Model](https://mlsanity.com/paper/2306.08256/) (2023)
- [Diffusion Models as Artists: Are we Closing the Gap between Humans and Machines?](https://mlsanity.com/paper/2301.11722/) (2023)
- [Diffusing the Optimal Topology: A Generative Optimization Approach](https://mlsanity.com/paper/2303.09760/) (2023)
- [Image Synthesis under Limited Data: A Survey and Taxonomy](https://mlsanity.com/paper/2307.16879/) (2023)
- [EEG Synthetic Data Generation Using Probabilistic Diffusion Models](https://mlsanity.com/paper/2303.06068/) (2023)
- [From Generation to Generalization: Emergent Few-Shot Learning in Video Diffusion Models](https://mlsanity.com/paper/2506.07280/) (2025)
- [UDPM: Upsampling Diffusion Probabilistic Models](https://mlsanity.com/paper/2305.16269/) (2023)
- [Provable Sample-Efficient Transfer Learning Conditional Diffusion Models via Representation Learning](https://mlsanity.com/paper/2502.04491/) (2025)
- [Improving the Effectiveness of Deep Generative Data](https://mlsanity.com/paper/2311.03959/) (2023)
- [Learning to Reconstruct Accelerated MRI Through K-space Cold Diffusion without Noise](https://mlsanity.com/paper/2311.10162/) (2023)
- [Enforcing Conditional Independence for Fair Representation Learning and Causal Image Generation](https://mlsanity.com/paper/2404.13798/) (2024)
- [Diffusion-Refined VQA Annotations for Semi-Supervised Gaze Following](https://mlsanity.com/paper/2406.02774/) (2024)
- [On quantifying and improving realism of images generated with diffusion](https://mlsanity.com/paper/2309.14756/) (2023)

## Abstract (arXiv)

Denoising diffusion probabilistic models (DDPM) are powerful hierarchical latent variable models with remarkable sample generation quality and training stability. These properties can be attributed to parameter sharing in the generative hierarchy, as well as a parameter-free diffusion-based inference procedure. In this paper, we present Few-Shot Diffusion Models (FSDM), a framework for few-shot generation leveraging conditional DDPMs. FSDMs are trained to adapt the generative process conditioned on a small set of images from a given class by aggregating image patch information using a set-based Vision Transformer (ViT). At test time, the model is able to generate samples from previously unseen classes conditioned on as few as 5 samples from that class. We empirically show that FSDM can perform few-shot generation and transfer to new datasets. We benchmark variants of our method on complex vision datasets for few-shot learning and compare to unconditional and conditional DDPM baselines. Additionally, we show how conditioning the model on patch-based input set information improves training convergence.