mlsanityData to 14 Oct 2026

Masked Diffusion as Self-supervised Representation Learner

arXiv 2308.05695 ↗First posted 10 Aug 2023cs.CVcited by 11

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Denoising diffusion probabilistic models have recently demonstrated state-of-the-art generative performance and have been used as strong pixel-level representation learners. This paper decomposes the interrelation between the generative capability and representation learning ability inherent in diffusion models. We present the masked diffusion model (MDM), a scalable self-supervised representation learner for semantic segmentation, substituting the conventional additive Gaussian noise of traditional diffusion with a masking mechanism. Our proposed approach convincingly surpasses prior benchmarks, demonstrating remarkable advancements in both medical and natural image semantic segmentation tasks, particularly in few-shot scenarios.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 34 of its 43 references that are in the map. First posted on arXiv 10 Aug 2023.

Builds on

34 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Label-Efficient Semantic Segmentation with Diffusion Models6 Dec

Other references · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  10. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  11. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,288
21 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  2. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  3. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  5. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  6. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  7. 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,525
  8. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  9. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  10. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 830
  11. 2021Medical Transformer: Gated Axial-Attention for Medical Image Segmentationcited by 320
  12. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noisecited by 276
  13. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  14. 2016Gland Segmentation in Colon Histology Images: The GlaS Challenge Contestcited by 203
  15. 2021UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-wise Perspective with Transformercited by 197
  16. 2022Generative Modelling With Inverse Heat Dissipationcited by 167
  17. 2016Learning Features by Watching Objects Movecited by 138
  18. 2023Denoising Diffusion Autoencoders are Unified Self-supervised Learnerscited by 130
  19. 2022Soft Diffusion: Score Matching for General Corruptionscited by 102
  20. 2022Blurring Diffusion Modelscited by 91
  21. 2018Cascaded V-Net using ROI masks for brain tumor segmentationcited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 3; 2026: 2.

11 papers.

  1. 2024Diffusion Models and Representation Learning: A Surveycited by 54
  2. 2024Diffusion Feedback Helps CLIP See Bettercited by 38
  3. 2024Segment Any Cell: A SAM-based Auto-prompting Fine-tuning Framework for Nuclei Segmentationcited by 24
  4. 2024Random Walks with Tweedie: A Unified View of Score-Based Diffusion Modelscited by 6
  5. 2024SubGDiff: A Subgraph Diffusion Model to Improve Molecular Representation Learningcited by 4
  6. 2025SketchFusion: Learning Universal Sketch Features through Fusing Foundation Modelscited by 4
  7. 2026Beyond ViT Tokens: Masked-Diffusion Pretrained Convolutional Pathology Foundation Model for Cell-Level Dense Predictioncited by 1
  8. 2024Rethinking Medical Anomaly Detection in Brain MRI: An Image Quality Assessment Perspectivecited by 0
  9. 2025Less is More: Masking Elements in Image Condition Features Avoids Content Leakages in Style Transfer Diffusion Modelscited by 0
  10. 2025Automated Learning of Semantic Embedding Representations for Diffusion Modelscited by 0
  11. 2026Representation Learning in Diffusion and Flow-based Model: An Application Aspectcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Denoising Diffusion Semantic Segmentation with Mask Prior Modeling
  2. 2024Simplified and Generalized Masked Diffusion for Discrete Data
  3. 2024MaskDiffusion: Exploiting Pre-trained Diffusion Models for Semantic Segmentation
  4. 2024Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentation
  5. 2023Conditional Diffusion Models for Weakly Supervised Medical Image Segmentation
  6. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
  7. 2023Masked Diffusion Models Are Fast Distribution Learners
  8. 2023Exploring Limits of Diffusion-Synthetic Training with Weakly Supervised Semantic Segmentation