# Rethinking Autoencoders for Medical Anomaly Detection from A Theoretical Perspective

arXiv: https://arxiv.org/abs/2403.09303 · HTML: https://mlsanity.com/paper/2403.09303/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Builds on (most cited first, 15 of 15)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)
- [A Unified Model for Multi-class Anomaly Detection](https://mlsanity.com/paper/2206.03687/) (2022)
- [VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations](https://mlsanity.com/paper/2012.15029/) (2020)
- [Autoencoders for Unsupervised Anomaly Segmentation in Brain MR Images: A Comparative Study](https://mlsanity.com/paper/2004.03271/) (2020)
- [Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images](https://mlsanity.com/paper/1804.04488/) (2018)
- [Unsupervised Detection of Lesions in Brain MRI using constrained adversarial auto-encoders](https://mlsanity.com/paper/1806.04972/) (2018)
- [Hierarchical Vector Quantized Transformer for Multi-class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2310.14228/) (2023)
- [Unsupervised Lesion Detection via Image Restoration with a Normative Prior](https://mlsanity.com/paper/2005.00031/) (2020)
- [Context-encoding Variational Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1812.05941/) (2018)
- [Dual-Distribution Discrepancy for Anomaly Detection in Chest X-Rays](https://mlsanity.com/paper/2206.03935/) (2022)

## Built on by (most cited first, 23 of 23)

- [UniVAD: A Training-free Unified Model for Few-shot Visual Anomaly Detection](https://mlsanity.com/paper/2412.03342/) (2024)
- [MedIAnomaly: A comparative study of anomaly detection in medical images](https://mlsanity.com/paper/2404.04518/) (2024)
- [Autoencoders for Anomaly Detection are Unreliable](https://mlsanity.com/paper/2501.13864/) (2025)
- [Revisiting Deep Ensemble Uncertainty for Enhanced Medical Anomaly Detection](https://mlsanity.com/paper/2409.17485/) (2024)
- [Enhancing Reconstruction-Based Out-of-Distribution Detection in Brain MRI with Model and Metric Ensembles](https://mlsanity.com/paper/2412.17586/) (2024)
- [Constitution or Collapse? Exploring Constitutional AI with Llama 3-8B](https://mlsanity.com/paper/2504.04918/) (2025)
- [Q-Former Autoencoder: A Modern Framework for Medical Anomaly Detection](https://mlsanity.com/paper/2507.18481/) (2025)
- [WDT-MD: Wavelet Diffusion Transformers for Microaneurysm Detection in Fundus Images](https://mlsanity.com/paper/2511.08987/) (2025)
- [Training-Free Zero-Shot Anomaly Detection in 3D Brain MRI with 2D Foundation Models](https://mlsanity.com/paper/2602.15315/) (2026)
- [ACMamba: Fast Unsupervised Anomaly Detection via An Asymmetrical Consensus State Space Model](https://mlsanity.com/paper/2504.11781/) (2025)
- [Unsupervised anomaly detection using Bayesian flow networks: application to brain FDG PET in the context of Alzheimer's disease](https://mlsanity.com/paper/2507.17486/) (2025)
- [Mitigating the reconstruction-detection trade-off in VAE-based unsupervised anomaly detection](https://mlsanity.com/paper/2605.02918/) (2026)
- [U2AD: Uncertainty-based Unsupervised Anomaly Detection Framework for Detecting T2 Hyperintensity in MRI Spinal Cord](https://mlsanity.com/paper/2503.13400/) (2025)
- [CADD: Context aware disease deviations via restoration of brain images using normative conditional diffusion models](https://mlsanity.com/paper/2508.03594/) (2025)
- [TMUAD: Enhancing Logical Capabilities in Unified Anomaly Detection Models with a Text Memory Bank](https://mlsanity.com/paper/2508.21795/) (2025)
- [ShortcutBreaker: Low-Rank Noisy Bottleneck and Frequency Filtering Block for Multi-Class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2510.18342/) (2025)
- [RcAE: Recursive Reconstruction Framework for Unsupervised Industrial Anomaly Detection](https://mlsanity.com/paper/2512.11284/) (2025)
- [MedAD-R1: Eliciting Consistent Reasoning in Interpretible Medical Anomaly Detection via Consistency-Reinforced Policy Optimization](https://mlsanity.com/paper/2602.01081/) (2026)
- [MTL-MAD: Multi-Task Learners are Effective Medical Anomaly Detectors](https://mlsanity.com/paper/2605.05891/) (2026)
- [Toward Training-Free Zero-Shot Anomaly Detection in 3D Medical Images: A Batch-Based Approach Using 2D Foundation Models](https://mlsanity.com/paper/2606.18749/) (2026)
- [Robust Zero-shot Anomaly Detection under Limited Auxiliary Anomaly Priors](https://mlsanity.com/paper/2606.29428/) (2026)
- [GeoMAD: Geometry-Aware Multi-View Anomaly Detection via Deformable Fusion and Distributional Alignment](https://mlsanity.com/paper/2608.26724/) (2026)
- [Quasi-Binarized Autoencoders: An Architecture-Independent Information Bottleneck for Medical Image Anomaly Detection](https://mlsanity.com/paper/2610.09670/) (2026)

## Abstract (arXiv)

Medical anomaly detection aims to identify abnormal findings using only normal training data, playing a crucial role in health screening and recognizing rare diseases. Reconstruction-based methods, particularly those utilizing autoencoders (AEs), are dominant in this field. They work under the assumption that AEs trained on only normal data cannot reconstruct unseen abnormal regions well, thereby enabling the anomaly detection based on reconstruction errors. However, this assumption does not always hold due to the mismatch between the reconstruction training objective and the anomaly detection task objective, rendering these methods theoretically unsound. This study focuses on providing a theoretical foundation for AE-based reconstruction methods in anomaly detection. By leveraging information theory, we elucidate the principles of these methods and reveal that the key to improving AE in anomaly detection lies in minimizing the information entropy of latent vectors. Experiments on four datasets with two image modalities validate the effectiveness of our theory. To the best of our knowledge, this is the first effort to theoretically clarify the principles and design philosophy of AE for anomaly detection. The code is available at urlhttps://github.com/caiyu6666/AE4AD.