mlsanityData to 7 Oct 2026

DiffusionInst: Diffusion Model for Instance Segmentation

arXiv 2212.02773 ↗First posted 6 Dec 2022cs.CVcited by 49

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Diffusion frameworks have achieved comparable performance with previous state-of-the-art image generation models. Researchers are curious about its variants in discriminative tasks because of its powerful noise-to-image denoising pipeline. This paper proposes DiffusionInst, a novel framework that represents instances as instance-aware filters and formulates instance segmentation as a noise-to-filter denoising process. The model is trained to reverse the noisy groundtruth without any inductive bias from RPN. During inference, it takes a randomly generated filter as input and outputs mask in one-step or multi-step denoising. Extensive experimental results on COCO and LVIS show that DiffusionInst achieves competitive performance compared to existing instance segmentation models with various backbones, such as ResNet and Swin Transformers. We hope our work could serve as a strong baseline, which could inspire designing more efficient diffusion frameworks for challenging discriminative tasks. Our code is available in https://github.com/chenhaoxing/DiffusionInst.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 50 of its 60 references that are in the map. First posted on arXiv 6 Dec 2022.

Builds on

50 of 60 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021SegDiff: Image Segmentation with Diffusion Probabilistic Models1 Dec

Other references · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2020Denoising Diffusion Implicit Modelscited by 9,655
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  10. 2020End-to-End Object Detection with Transformerscited by 7,384
  11. 2017Mask R-CNNcited by 7,259
  12. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
37 more
  1. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  4. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  6. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  7. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  8. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  9. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  10. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
  11. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  12. 2018Path Aggregation Network for Instance Segmentationcited by 796
  13. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  14. 2022Diffusion Models for Adversarial Purificationcited by 469
  15. 2019YOLACT: Real-time Instance Segmentationcited by 441
  16. 2019PointRend: Image Segmentation as Renderingcited by 439
  17. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  18. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  19. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  20. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 341
  21. 2020Conditional Convolutions for Instance Segmentationcited by 314
  22. 2019SOLO: Segmenting Objects by Locationscited by 289
  23. 2020Equalization Loss for Long-Tailed Object Recognitioncited by 285
  24. 2019Mask Scoring R-CNNcited by 265
  25. 2016Associative Embedding: End-to-End Learning for Joint Detection and Groupingcited by 211
  26. 2019PolarMask: Single Shot Instance Segmentation with Polar Representationcited by 188
  27. 2021OTA: Optimal Transport Assignment for Object Detectioncited by 177
  28. 2021Instances as Queriescited by 157
  29. 2020BlendMask: Top-Down Meets Bottom-Up for Instance Segmentationcited by 144
  30. 2016Fully Convolutional Instance-aware Semantic Segmentationcited by 122
  31. 2020Equalization Loss v2: A New Gradient Balance Approach for Long-tailed Object Detectioncited by 99
  32. 2022A Generalist Framework for Panoptic Segmentation of Images and Videoscited by 99
  33. 2022Guided Diffusion Model for Adversarial Purificationcited by 90
  34. 2019SSAP: Single-Shot Instance Segmentation With Affinity Pyramidcited by 85
  35. 2021SOLQ: Segmenting Objects by Learning Queriescited by 68
  36. 2022Latent Diffusion Energy-Based Model for Interpretable Text Modelingcited by 67
  37. 2020Mask Encoding for Single Shot Instance Segmentationcited by 47

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 21; 2024: 16; 2025: 8; 2026: 4.

49 papers.

Sort:

By area:

  1. 2023A Survey on Video Diffusion Modelscited by 203
  2. 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
  3. 2023DiffusionTrack: Diffusion Model For Multi-Object Trackingcited by 40
  4. 2023DiffPose: SpatioTemporal Diffusion Model for Video-Based Human Pose Estimationcited by 36
  5. 2023DiffBEV: Conditional Diffusion Model for Bird's Eye View Perceptioncited by 30
  6. 2023SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete Diffusion Processcited by 28
  7. 2023Diffusion Model for Camouflaged Object Detectioncited by 27
  8. 2023Exploring Diffusion Models for Unsupervised Video Anomaly Detectioncited by 23
  9. 2023Diffusion Model for Dense Matchingcited by 21
  10. 2023CrowdDiff: Multi-hypothesis Crowd Density Estimation using Diffusion Modelscited by 20
  11. 2024SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flowcited by 20
  12. 2023DFormer: Diffusion-guided Transformer for Universal Image Segmentationcited by 19
37 more
  1. 2024A Simple Latent Diffusion Approach for Panoptic Segmentation and Mask Inpaintingcited by 18
  2. 2024DiffusionMTL: Learning Multi-Task Denoising Diffusion Model from Partially Annotated Datacited by 18
  3. 2023Denoising Diffusion Semantic Segmentation with Mask Prior Modelingcited by 15
  4. 2024Scaling Properties of Diffusion Models for Perceptual Taskscited by 15
  5. 2023Diffusion-SS3D: Diffusion Model for Semi-supervised 3D Object Detectioncited by 14
  6. 2024DiffSal: Joint Audio and Video Learning for Diffusion Saliency Predictioncited by 13
  7. 2024Explore In-Context Segmentation via Latent Diffusion Modelscited by 12
  8. 2023Diffusion for Natural Image Mattingcited by 11
  9. 2023DiffUCD:Unsupervised Hyperspectral Image Change Detection with Semantic Correlation Diffusion Modelcited by 8
  10. 2025Symmetrical Flow Matching: Unified Image Generation, Segmentation, and Classification with Score-Based Generative Modelscited by 8
  11. 2024DiffSF: Diffusion Models for Scene Flow Estimationcited by 6
  12. 2024ResVG: Enhancing Relation and Semantic Understanding in Multiple Instances for Visual Groundingcited by 6
  13. 2023Learning Structure-Guided Diffusion Model for 2D Human Pose Estimationcited by 5
  14. 2024Fractal Calibration for long-tailed object detectioncited by 5
  15. 2025SeeDiff: Off-the-Shelf Seeded Mask Generation from Diffusion Modelscited by 5
  16. 2023Backpropagation Path Search On Adversarial Transferabilitycited by 4
  17. 20233DifFusionDet: Diffusion Model for 3D Object Detection with Robust LiDAR-Camera Fusioncited by 4
  18. 2024GRIF-DM: Generation of Rich Impression Fonts using Diffusion Modelscited by 4
  19. 2024Multi-Task Label Discovery via Hierarchical Task Tokens for Partially Annotated Dense Predictionscited by 4
  20. 2026Diffusion Model as a Generalist Segmentation Learnercited by 4
  21. 2023SEMI-DiffusionInst: A Diffusion Model Based Approach for Semiconductor Defect Classification and Segmentationcited by 3
  22. 2024Diff-Reg v1: Diffusion Matching Model for Registration Problemcited by 3
  23. 2023DiffRef3D: A Diffusion-based Proposal Refinement Framework for 3D Object Detectioncited by 2
  24. 2025A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospectscited by 2
  25. 2024Answerability Fields: Answerable Location Estimation via Diffusion Modelscited by 1
  26. 2024Flow-TSVAD: Target-Speaker Voice Activity Detection via Latent Flow Matchingcited by 1
  27. 2025Computationally Efficient Diffusion Models in Medical Imaging: A Comprehensive Reviewcited by 1
  28. 2025M^3-GloDets: Multi-Region and Multi-Scale Analysis of Fine-Grained Diseased Glomerular Detectioncited by 1
  29. 2025REXO: Indoor Multi-View Radar Object Detection via 3D Bounding Box Diffusioncited by 1
  30. 2023Renderers are Good Zero-Shot Representation Learners: Exploring Diffusion Latents for Metric Learningcited by 0
  31. 2024Bring the Power of Diffusion Model to Defect Detectioncited by 0
  32. 2024Advancing SEM Based Nano-Scale Defect Analysis in Semiconductor Manufacturing for Advanced IC Nodescited by 0
  33. 2025UGD-IML: A Unified Generative Diffusion-based Framework for Constrained and Unconstrained Image Manipulation Localizationcited by 0
  34. 2025MedCondDiff: Lightweight, Robust, Semantically Guided Diffusion for Medical Image Segmentationcited by 0
  35. 2026Towards Any-Quality Image Segmentation via Generative and Adaptive Latent Space Enhancementcited by 0
  36. 2026Exclusivity-Guided Mask Learning for Semi-Supervised Crowd Instance Segmentation and Countingcited by 0
  37. 2026Modular Diffusion Models for Structured Visual Recognitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Conditional Latent Diffusion Models for Zero-Shot Instance Segmentation
  2. 2023Factorized Diffusion Architectures for Unsupervised Image Generation and Segmentation
  3. 2025TRACE: Your Diffusion Model is Secretly an Instance Edge Detector
  4. 2022Diffusion Models in Vision: A Survey
  5. 2021Label-Efficient Semantic Segmentation with Diffusion Models
  6. 2024Diffusion Models in Low-Level Vision: A Survey
  7. 2023DiffusionSeg: Adapting Diffusion Towards Unsupervised Object Discovery
  8. 2023Masked Diffusion as Self-supervised Representation Learner