# Unsupervised 3D out-of-distribution detection with latent diffusion models

arXiv: https://arxiv.org/abs/2307.03777 · HTML: https://mlsanity.com/paper/2307.03777/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 7 Jul 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Simple diffusion: End-to-end diffusion for high resolution images](https://mlsanity.com/paper/2301.11093/) (2023)
- [Input complexity and out-of-distribution detection with likelihood-based generative models](https://mlsanity.com/paper/1909.11480/) (2019)
- [Self-attention Does Not Need O(n2) Memory](https://mlsanity.com/paper/2112.05682/) (2021)
- [Denoising diffusion models for out-of-distribution detection](https://mlsanity.com/paper/2211.07740/) (2022)
- [Improving Reconstruction Autoencoder Out-of-distribution Detection with Mahalanobis Distance](https://mlsanity.com/paper/1812.02765/) (2018)
- [Hierarchical VAEs Know What They Don't Know](https://mlsanity.com/paper/2102.08248/) (2021)
- [Transformer-based out-of-distribution detection for clinically safe segmentation](https://mlsanity.com/paper/2205.10650/) (2022)
- [Morphology-preserving Autoregressive 3D Generative Modelling of the Brain](https://mlsanity.com/paper/2209.03177/) (2022)
- [Neuromorphologicaly-preserving Volumetric data encoding using VQ-VAE](https://mlsanity.com/paper/2002.05692/) (2020)
- [Cross Attention Transformers for Multi-modal Unsupervised Whole-Body PET Anomaly Detection](https://mlsanity.com/paper/2304.07147/) (2023)

## Built on by (most cited first, 22 of 22)

- [Out-of-distribution Detection in Medical Image Analysis: A survey](https://mlsanity.com/paper/2404.18279/) (2024)
- [A Survey on Diffusion Models for Anomaly Detection](https://mlsanity.com/paper/2501.11430/) (2025)
- [Anomaly Detection and Generation with Diffusion Models: A Survey](https://mlsanity.com/paper/2506.09368/) (2025)
- [Conditional diffusion models for guided anomaly detection in brain images using fluid-driven anomaly randomization](https://mlsanity.com/paper/2506.10233/) (2025)
- [Going beyond Compositions, DDPMs Can Produce Zero-Shot Interpolations](https://mlsanity.com/paper/2405.19201/) (2024)
- [Enhancing Reconstruction-Based Out-of-Distribution Detection in Brain MRI with Model and Metric Ensembles](https://mlsanity.com/paper/2412.17586/) (2024)
- [Unraveling Normal Anatomy via Fluid-Driven Anomaly Randomization](https://mlsanity.com/paper/2501.13370/) (2025)
- [Denoising Diffusion Models for Anomaly Localization in Medical Images](https://mlsanity.com/paper/2410.23834/) (2024)
- [MedSymmFlow: Bridging Generative Modeling and Classification in Medical Imaging through Symmetrical Flow Matching](https://mlsanity.com/paper/2507.19098/) (2025)
- [Differentiable Score-Based Likelihoods: Learning CT Motion Compensation From Clean Images](https://mlsanity.com/paper/2404.14747/) (2024)
- [Tumor-anchored deep feature random forests for out-of-distribution detection in lung cancer segmentation](https://mlsanity.com/paper/2512.08216/) (2025)
- [Uncertainty-Aware Distribution-to-Distribution Flow Matching for Scientific Imaging](https://mlsanity.com/paper/2603.21717/) (2026)
- [Diffuse to Detect: Generative Diffusion Models for Unsupervised IC Anomaly Detection](https://mlsanity.com/paper/2605.26468/) (2026)
- [LoGex: Improved tail detection of extremely rare histopathology classes via guided diffusion](https://mlsanity.com/paper/2409.01317/) (2024)
- [Exploring Out-of-distribution Detection for Sparse-view Computed Tomography with Diffusion Models](https://mlsanity.com/paper/2411.06308/) (2024)
- [Investigating the Role of Bilateral Symmetry for Inpainting Brain MRI](https://mlsanity.com/paper/2504.10039/) (2025)
- [Unsupervised Out-of-Distribution Detection in Medical Imaging Using Multi-Exit Class Activation Maps and Feature Masking](https://mlsanity.com/paper/2505.08604/) (2025)
- [CADD: Context aware disease deviations via restoration of brain images using normative conditional diffusion models](https://mlsanity.com/paper/2508.03594/) (2025)
- [Generating healthy counterfactuals with denoising diffusion bridge models](https://mlsanity.com/paper/2510.13684/) (2025)
- [Deep generative priors for 3D brain analysis](https://mlsanity.com/paper/2510.15119/) (2025)
- [Kidney Cancer Detection Using 3D-Based Latent Diffusion Models](https://mlsanity.com/paper/2601.05852/) (2026)
- [Unsupervised Anomaly Detection for Image Dataset Quality Assurance in Multi-Center Breast MRI](https://mlsanity.com/paper/2608.16725/) (2026)

## Abstract (arXiv)

Methods for out-of-distribution (OOD) detection that scale to 3D data are crucial components of any real-world clinical deep learning system. Classic denoising diffusion probabilistic models (DDPMs) have been recently proposed as a robust way to perform reconstruction-based OOD detection on 2D datasets, but do not trivially scale to 3D data. In this work, we propose to use Latent Diffusion Models (LDMs), which enable the scaling of DDPMs to high-resolution 3D medical data. We validate the proposed approach on near- and far-OOD datasets and compare it to a recently proposed, 3D-enabled approach using Latent Transformer Models (LTMs). Not only does the proposed LDM-based approach achieve statistically significant better performance, it also shows less sensitivity to the underlying latent representation, more favourable memory scaling, and produces better spatial anomaly maps. Code is available at https://github.com/marksgraham/ddpm-ood