# DifFSS: Diffusion Model for Few-Shot Semantic Segmentation

arXiv: https://arxiv.org/abs/2307.00773 · HTML: https://mlsanity.com/paper/2307.00773/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 34 of its 38 references that are in the map.
- First posted on arXiv 3 Jul 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 30 of 34)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [One-Shot Learning for Semantic Segmentation](https://mlsanity.com/paper/1709.03410/) (2017)
- [Prior Guided Feature Enrichment Network for Few-Shot Segmentation](https://mlsanity.com/paper/2008.01449/) (2020)
- [CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning](https://mlsanity.com/paper/1903.02351/) (2019)
- [Adaptive Prototype Learning and Allocation for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01893/) (2021)
- [Hypercorrelation Squeeze for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01538/) (2021)
- [Prototype Mixture Models for Few-shot Semantic Segmentation](https://mlsanity.com/paper/2008.03898/) (2020)
- [Feature Weighting and Boosting for Few-Shot Segmentation](https://mlsanity.com/paper/1909.13140/) (2019)
- [SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation](https://mlsanity.com/paper/1810.09091/) (2018)
- [Part-aware Prototype Network for Few-shot Semantic Segmentation](https://mlsanity.com/paper/2007.06309/) (2020)
- [FSS-1000: A 1000-Class Dataset for Few-Shot Segmentation](https://mlsanity.com/paper/1907.12347/) (2019)
- [Few-Shot Segmentation via Cycle-Consistent Transformer](https://mlsanity.com/paper/2106.02320/) (2021)
- [Learning What Not to Segment: A New Perspective on Few-Shot Segmentation](https://mlsanity.com/paper/2203.07615/) (2022)
- [CRNet: Cross-Reference Networks for Few-Shot Segmentation](https://mlsanity.com/paper/2003.10658/) (2020)
- [Hierarchical Dense Correlation Distillation for Few-Shot Segmentation](https://mlsanity.com/paper/2303.14652/) (2023)
- [Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation](https://mlsanity.com/paper/2207.08549/) (2022)

## Built on by (most cited first, 22 of 22)

- [SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation](https://mlsanity.com/paper/2403.16605/) (2024)
- [Interpretable Diffusion via Information Decomposition](https://mlsanity.com/paper/2310.07972/) (2023)
- [Image Segmentation in Foundation Model Era: A Survey](https://mlsanity.com/paper/2408.12957/) (2024)
- [3DiffTection: 3D Object Detection with Geometry-Aware Diffusion Features](https://mlsanity.com/paper/2311.04391/) (2023)
- [Distillation of Diffusion Features for Semantic Correspondence](https://mlsanity.com/paper/2412.03512/) (2024)
- [Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentation](https://mlsanity.com/paper/2410.02369/) (2024)
- [AI-Generated Images as Data Source: The Dawn of Synthetic Era](https://mlsanity.com/paper/2310.01830/) (2023)
- [Explore In-Context Segmentation via Latent Diffusion Models](https://mlsanity.com/paper/2403.09616/) (2024)
- [MaskDiffusion: Exploiting Pre-trained Diffusion Models for Semantic Segmentation](https://mlsanity.com/paper/2403.11194/) (2024)
- [ControlUDA: Controllable Diffusion-assisted Unsupervised Domain Adaptation for Cross-Weather Semantic Segmentation](https://mlsanity.com/paper/2402.06446/) (2024)
- [AgMTR: Agent Mining Transformer for Few-shot Segmentation in Remote Sensing](https://mlsanity.com/paper/2409.17453/) (2024)
- [FBSDiff: Plug-and-Play Frequency Band Substitution of Diffusion Features for Highly Controllable Text-Driven Image Translation](https://mlsanity.com/paper/2408.00998/) (2024)
- [Data Augmentation via Latent Diffusion for Saliency Prediction](https://mlsanity.com/paper/2409.07307/) (2024)
- [DODA: Adapting Object Detectors to Dynamic Agricultural Environments in Real-Time with Diffusion](https://mlsanity.com/paper/2403.18334/) (2024)
- [Auto Cherry-Picker: Learning from High-quality Generative Data Driven by Language](https://mlsanity.com/paper/2406.20085/) (2024)
- [Conditional Latent Diffusion Models for Zero-Shot Instance Segmentation](https://mlsanity.com/paper/2508.04122/) (2025)
- [Group-On: Boosting One-Shot Segmentation with Supportive Query](https://mlsanity.com/paper/2404.11871/) (2024)
- [Denoising Diffusion Models for Anomaly Localization in Medical Images](https://mlsanity.com/paper/2410.23834/) (2024)
- [O3Afford: One-Shot 3D Object-to-Object Affordance Grounding for Generalizable Robotic Manipulation](https://mlsanity.com/paper/2509.06233/) (2025)
- [Computationally Efficient Diffusion Models in Medical Imaging: A Comprehensive Review](https://mlsanity.com/paper/2505.07866/) (2025)
- [Make It Up: Fake Images, Real Gains in Generalized Few-shot Semantic Segmentation](https://mlsanity.com/paper/2603.27206/) (2026)
- [Harnessing the Power of Training-Free Techniques in Text-to-2D Generation for Text-to-3D Generation via Score Distillation Sampling](https://mlsanity.com/paper/2505.19868/) (2025)

## Abstract (arXiv)

Diffusion models have demonstrated excellent performance in image generation. Although various few-shot semantic segmentation (FSS) models with different network structures have been proposed, performance improvement has reached a bottleneck. This paper presents the first work to leverage the diffusion model for FSS task, called DifFSS. DifFSS, a novel FSS paradigm, can further improve the performance of the state-of-the-art FSS models by a large margin without modifying their network structure. Specifically, we utilize the powerful generation ability of diffusion models to generate diverse auxiliary support images by using the semantic mask, scribble or soft HED boundary of the support image as control conditions. This generation process simulates the variety within the class of the query image, such as color, texture variation, lighting, etc. As a result, FSS models can refer to more diverse support images, yielding more robust representations, thereby achieving a consistent improvement in segmentation performance. Extensive experiments on three publicly available datasets based on existing advanced FSS models demonstrate the effectiveness of the diffusion model for FSS task. Furthermore, we explore in detail the impact of different input settings of the diffusion model on segmentation performance. Hopefully, this completely new paradigm will bring inspiration to the study of FSS task integrated with AI-generated content. Code is available at https://github.com/TrinitialChan/DifFSS