# Patched Diffusion Models for Unsupervised Anomaly Detection in Brain MRI

arXiv: https://arxiv.org/abs/2303.03758 · HTML: https://mlsanity.com/paper/2303.03758/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 7 Mar 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 25 of 25)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [Deep Learning for Medical Image Analysis](https://mlsanity.com/paper/1708.08987/) (2017)
- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models](https://mlsanity.com/paper/2207.14626/) (2022)
- [Deep learning with noisy labels: exploring techniques and remedies in medical image analysis](https://mlsanity.com/paper/1912.02911/) (2019)
- [Autoencoders for Unsupervised Anomaly Segmentation in Brain MR Images: A Comparative Study](https://mlsanity.com/paper/2004.03271/) (2020)
- [Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images](https://mlsanity.com/paper/1804.04488/) (2018)
- [Automated brain extraction of multi-sequence MRI using artificial neural networks](https://mlsanity.com/paper/1901.11341/) (2019)
- [Denoising diffusion models for out-of-distribution detection](https://mlsanity.com/paper/2211.07740/) (2022)
- [Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Models](https://mlsanity.com/paper/2206.03461/) (2022)
- [Unsupervised Detection of Lesions in Brain MRI using constrained adversarial auto-encoders](https://mlsanity.com/paper/1806.04972/) (2018)
- [Unsupervised Lesion Detection via Image Restoration with a Normative Prior](https://mlsanity.com/paper/2005.00031/) (2020)
- [Context-encoding Variational Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1812.05941/) (2018)
- [What is Healthy? Generative Counterfactual Diffusion for Lesion Localization](https://mlsanity.com/paper/2207.12268/) (2022)
- [Challenging Current Semi-Supervised Anomaly Segmentation Methods for Brain MRI](https://mlsanity.com/paper/2109.06023/) (2021)
- [Scale-Space Autoencoders for Unsupervised Anomaly Segmentation in Brain MRI](https://mlsanity.com/paper/2006.12852/) (2020)
- [Constrained unsupervised anomaly segmentation](https://mlsanity.com/paper/2203.01671/) (2022)
- [Unsupervised Region-based Anomaly Detection in Brain MRI with Adversarial Image Inpainting](https://mlsanity.com/paper/2010.01942/) (2020)

## Built on by (most cited first, 30 of 45)

- [Mask, Stitch, and Re-Sample: Enhancing Robustness and Generalizability in Anomaly Detection through Automatic Diffusion Models](https://mlsanity.com/paper/2305.19643/) (2023)
- [Patched Denoising Diffusion Models For High-Resolution Image Synthesis](https://mlsanity.com/paper/2308.01316/) (2023)
- [Diffusion Models with Implicit Guidance for Medical Anomaly Detection](https://mlsanity.com/paper/2403.08464/) (2024)
- [MedIAnomaly: A comparative study of anomaly detection in medical images](https://mlsanity.com/paper/2404.04518/) (2024)
- [Unsupervised Anomaly Detection in Medical Images Using Masked Diffusion Model](https://mlsanity.com/paper/2305.19867/) (2023)
- [Guided Reconstruction with Conditioned Diffusion Models for Unsupervised Anomaly Detection in Brain MRIs](https://mlsanity.com/paper/2312.04215/) (2023)
- [Leveraging the Mahalanobis Distance to enhance Unsupervised Brain MRI Anomaly Detection](https://mlsanity.com/paper/2407.12474/) (2024)
- [MAD-AD: Masked Diffusion for Unsupervised Brain Anomaly Detection](https://mlsanity.com/paper/2502.16943/) (2025)
- [A Survey of Emerging Applications of Diffusion Probabilistic Models in MRI](https://mlsanity.com/paper/2311.11383/) (2023)
- [Binary Noise for Binary Tasks: Masked Bernoulli Diffusion for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2403.11667/) (2024)
- [MAEDiff: Masked Autoencoder-enhanced Diffusion Models for Unsupervised Anomaly Detection in Brain Images](https://mlsanity.com/paper/2401.10561/) (2024)
- [REFLECT: Rectified Flows for Efficient Brain Anomaly Correction Transport](https://mlsanity.com/paper/2508.02889/) (2025)
- [Diffusion Models with Ensembled Structure-Based Anomaly Scoring for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2403.14262/) (2024)
- [Enhancing Reconstruction-Based Out-of-Distribution Detection in Brain MRI with Model and Metric Ensembles](https://mlsanity.com/paper/2412.17586/) (2024)
- [Generating Realistic Counterfactuals for Retinal Fundus and OCT Images using Diffusion Models](https://mlsanity.com/paper/2311.11629/) (2023)
- [Language Models Meet Anomaly Detection for Better Interpretability and Generalizability](https://mlsanity.com/paper/2404.07622/) (2024)
- [Diffusion Models for Unsupervised Anomaly Detection in Fetal Brain Ultrasound](https://mlsanity.com/paper/2407.15119/) (2024)
- [Denoising Diffusion Models for Anomaly Localization in Medical Images](https://mlsanity.com/paper/2410.23834/) (2024)
- [Synomaly Noise and Multi-Stage Diffusion: A Novel Approach for Unsupervised Anomaly Detection in Medical Images](https://mlsanity.com/paper/2411.04004/) (2024)
- [Histogram- and Diffusion-Based Medical Out-of-Distribution Detection](https://mlsanity.com/paper/2310.08654/) (2023)
- [AnoFPDM: Anomaly Segmentation with Forward Process of Diffusion Models for Brain MRI](https://mlsanity.com/paper/2404.15683/) (2024)
- [MCDDPM: Multichannel Conditional Denoising Diffusion Model for Unsupervised Anomaly Detection in Brain MRI](https://mlsanity.com/paper/2409.19623/) (2024)
- [Bigger Isn't Always Better: Towards a General Prior for Medical Image Reconstruction](https://mlsanity.com/paper/2501.07376/) (2025)
- [Discrepancy-based Diffusion Models for Lesion Detection in Brain MRI](https://mlsanity.com/paper/2405.04974/) (2024)
- [Unsupervised contrastive analysis for anomaly detection in brain MRIs via conditional diffusion models](https://mlsanity.com/paper/2406.00772/) (2024)
- [Pancreatic Tumor Segmentation as Anomaly Detection in CT Images Using Denoising Diffusion Models](https://mlsanity.com/paper/2406.02653/) (2024)
- [Normative Diffusion Autoencoders: Application to Amyotrophic Lateral Sclerosis](https://mlsanity.com/paper/2407.14191/) (2024)
- [Selective Test-Time Adaptation for Unsupervised Anomaly Detection using Neural Implicit Representations](https://mlsanity.com/paper/2410.03306/) (2024)
- [Patch2Loc: Learning to Localize Patches for Unsupervised Brain Lesion Detection](https://mlsanity.com/paper/2506.22504/) (2025)
- [Unsupervised Deep Generative Models for Anomaly Detection in Neuroimaging: A Systematic Scoping Review](https://mlsanity.com/paper/2510.14462/) (2025)

## Abstract (arXiv)

The use of supervised deep learning techniques to detect pathologies in brain MRI scans can be challenging due to the diversity of brain anatomy and the need for annotated data sets. An alternative approach is to use unsupervised anomaly detection, which only requires sample-level labels of healthy brains to create a reference representation. This reference representation can then be compared to unhealthy brain anatomy in a pixel-wise manner to identify abnormalities. To accomplish this, generative models are needed to create anatomically consistent MRI scans of healthy brains. While recent diffusion models have shown promise in this task, accurately generating the complex structure of the human brain remains a challenge. In this paper, we propose a method that reformulates the generation task of diffusion models as a patch-based estimation of healthy brain anatomy, using spatial context to guide and improve reconstruction. We evaluate our approach on data of tumors and multiple sclerosis lesions and demonstrate a relative improvement of 25.1% compared to existing baselines.