# Masked Autoencoders for Unsupervised Anomaly Detection in Medical Images

arXiv: https://arxiv.org/abs/2307.07534 · HTML: https://mlsanity.com/paper/2307.07534/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 14 Jul 2023.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [Masked Autoencoders that Listen](https://mlsanity.com/paper/2207.06405/) (2022)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge](https://mlsanity.com/paper/1612.08012/) (2016)
- [Autoencoders for Unsupervised Anomaly Segmentation in Brain MR Images: A Comparative Study](https://mlsanity.com/paper/2004.03271/) (2020)
- [Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images](https://mlsanity.com/paper/1804.04488/) (2018)
- [Asymmetric Student-Teacher Networks for Industrial Anomaly Detection](https://mlsanity.com/paper/2210.07829/) (2022)
- [Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Models](https://mlsanity.com/paper/2206.03461/) (2022)
- [Unsupervised Detection of Lesions in Brain MRI using constrained adversarial auto-encoders](https://mlsanity.com/paper/1806.04972/) (2018)
- [Unsupervised Lesion Detection via Image Restoration with a Normative Prior](https://mlsanity.com/paper/2005.00031/) (2020)
- [Context-encoding Variational Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1812.05941/) (2018)
- [MADGAN: unsupervised Medical Anomaly Detection GAN using multiple adjacent brain MRI slice reconstruction](https://mlsanity.com/paper/2007.13559/) (2020)
- [Anomaly Detection in Medical Imaging with Deep Perceptual Autoencoders](https://mlsanity.com/paper/2006.13265/) (2020)
- [Scale-Space Autoencoders for Unsupervised Anomaly Segmentation in Brain MRI](https://mlsanity.com/paper/2006.12852/) (2020)
- [Unsupervised Anomaly Detection in 3D Brain MRI using Deep Learning with Multi-Task Brain Age Prediction](https://mlsanity.com/paper/2201.13081/) (2022)

## Built on by (most cited first, 9 of 9)

- [Variational Autoencoder for Anomaly Detection: A Comparative Study](https://mlsanity.com/paper/2408.13561/) (2024)
- [Back-in-Time Diffusion: Unsupervised Detection of Medical Deepfakes](https://mlsanity.com/paper/2407.15169/) (2024)
- [Foundation Model for Composite Microstructures: Reconstruction, Stiffness, and Nonlinear Behavior Prediction](https://mlsanity.com/paper/2411.06565/) (2024)
- [Q-Former Autoencoder: A Modern Framework for Medical Anomaly Detection](https://mlsanity.com/paper/2507.18481/) (2025)
- [Self-Supervised Learning of Graph Representations for Network Intrusion Detection](https://mlsanity.com/paper/2509.16625/) (2025)
- [Toward Faithful Explanations in Acoustic Anomaly Detection](https://mlsanity.com/paper/2601.12660/) (2026)
- [Drift-Aware Variational Autoencoder-based Anomaly Detection with Two-level Ensembling](https://mlsanity.com/paper/2602.12976/) (2026)
- [AUCp: Pseudo-AUC for Inference Model Selection with Unlabeled Validation Data in Abnormality Detection](https://mlsanity.com/paper/2606.08742/) (2026)
- [When Does Unsupervised Learning Succeed or Fail? A PoS Perspective on Reconstruction-Based Anomaly Detection](https://mlsanity.com/paper/2609.28832/) (2026)

## Abstract (arXiv)

Pathological anomalies exhibit diverse appearances in medical imaging, making it difficult to collect and annotate a representative amount of data required to train deep learning models in a supervised setting. Therefore, in this work, we tackle anomaly detection in medical images training our framework using only healthy samples. We propose to use the Masked Autoencoder model to learn the structure of the normal samples, then train an anomaly classifier on top of the difference between the original image and the reconstruction provided by the masked autoencoder. We train the anomaly classifier in a supervised manner using as negative samples the reconstruction of the healthy scans, while as positive samples, we use pseudo-abnormal scans obtained via our novel pseudo-abnormal module. The pseudo-abnormal module alters the reconstruction of the normal samples by changing the intensity of several regions. We conduct experiments on two medical image data sets, namely BRATS2020 and LUNA16 and compare our method with four state-of-the-art anomaly detection frameworks, namely AST, RD4AD, AnoVAEGAN and f-AnoGAN.