mlsanityData to 7 Oct 2026

Unsupervised Discovery of 3D Hierarchical Structure with Generative Diffusion Features

arXiv 2305.00067 ↗First posted 28 Apr 2023cs.CVcited by 4

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Inspired by recent findings that generative diffusion models learn semantically meaningful representations, we use them to discover the intrinsic hierarchical structure in biomedical 3D images using unsupervised segmentation. We show that features of diffusion models from different stages of a U-Net-based ladder-like architecture capture different hierarchy levels in 3D biomedical images. We design three losses to train a predictive unsupervised segmentation network that encourages the decomposition of 3D volumes into meaningful nested subvolumes that represent a hierarchy. First, we pretrain 3D diffusion models and use the consistency of their features across subvolumes. Second, we use the visual consistency between subvolumes. Third, we use the invariance to photometric augmentations as a regularizer. Our models achieve better performance than prior unsupervised structure discovery approaches on challenging biologically-inspired synthetic datasets and on a real-world brain tumor MRI dataset.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 26 of its 36 references that are in the map. First posted on arXiv 28 Apr 2023.

Builds on

26 of 36 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model1 Nov
  2. 2022Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Models7 Jun
  3. 2022Diffusion Models for Medical Anomaly Detection8 Mar
  4. 2022Unsupervised Denoising of Retinal OCT with Diffusion Probabilistic Model27 Jan
  5. 2021Diffusion Models for Implicit Image Segmentation Ensembles6 Dec
  6. 2021Label-Efficient Semantic Segmentation with Diffusion Models6 Dec

Other references · 20

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  4. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  5. 2018Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challengecited by 508
  6. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 454
  7. 2019Med3D: Transfer Learning for 3D Medical Image Analysiscited by 282
  8. 2019Large Scale Adversarial Representation Learningcited by 238
  9. 2021Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalizationcited by 141
  10. 2019Data augmentation using learned transformations for one-shot medical image segmentationcited by 138
  11. 2017Quicksilver: Fast Predictive Image Registration - a Deep Learning Approachcited by 113
  12. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
8 more
  1. 2018Deep Feature Factorization For Concept Discoverycited by 64
  2. 2021Unsupervised Part Discovery from Contrastive Reconstructioncited by 57
  3. 2022Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentationcited by 55
  4. 2022What is Healthy? Generative Counterfactual Diffusion for Lesion Localizationcited by 52
  5. 2020Capturing implicit hierarchical structure in 3D biomedical images with self-supervised hyperbolic representationscited by 24
  6. 2022Decoder Denoising Pretraining for Semantic Segmentationcited by 20
  7. 2019Unsupervised Segmentation of Hyperspectral Images Using 3D Convolutional Autoencoderscited by 11
  8. 2018Unsupervised Segmentation of 3D Medical Images Based on Clustering and Deep Representation Learningcited by 9

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 0; 2025: 2.

4 papers.

  1. 2023A Survey of Emerging Applications of Diffusion Probabilistic Models in MRIcited by 9
  2. 2023Conditional Generation from Unconditional Diffusion Models using Denoiser Representationscited by 7
  3. 2025Guiding Registration with Emergent Similarity from Pre-Trained Diffusion Modelscited by 6
  4. 2025TextDiffSeg: Text-guided Latent Diffusion Model for 3d Medical Images Segmentationcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Medical Semantic Segmentation with Diffusion Pretrain
  2. 2023Factorized Diffusion Architectures for Unsupervised Image Generation and Segmentation
  3. 2025Deep generative priors for 3D brain analysis
  4. 2024Diffuse-UDA: Addressing Unsupervised Domain Adaptation in Medical Image Segmentation with Appearance and Structure Aligned Diffusion Models
  5. 2023Distribution Aligned Diffusion and Prototype-guided network for Unsupervised Domain Adaptive Segmentation
  6. 2025Prior-Guided Residual Diffusion: Calibrated and Efficient Medical Image Segmentation
  7. 2024Label-Efficient 3D Brain Segmentation via Complementary 2D Diffusion Models with Orthogonal Views
  8. 2023Diff-UNet: A Diffusion Embedded Network for Volumetric Segmentation