# Unsupervised anomaly localization using VAE and beta-VAE

arXiv: https://arxiv.org/abs/2005.10686 · HTML: https://mlsanity.com/paper/2005.10686/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2020.

## Built on by (most cited first, 8 of 8)

- [What is Healthy? Generative Counterfactual Diffusion for Lesion Localization](https://mlsanity.com/paper/2207.12268/) (2022)
- [Unsupervised anomaly detection algorithms on real-world data: how many do we need?](https://mlsanity.com/paper/2305.00735/) (2023)
- [Efficient Out-of-Distribution Detection Using Latent Space of β-VAE for Cyber-Physical Systems](https://mlsanity.com/paper/2108.11800/) (2021)
- [Scalable, Explainable and Provably Robust Anomaly Detection with One-Step Flow Matching](https://mlsanity.com/paper/2510.18328/) (2025)
- [Blind Localization and Clustering of Anomalies in Textures](https://mlsanity.com/paper/2404.12246/) (2024)
- [Confidence-Aware and Self-Supervised Image Anomaly Localisation](https://mlsanity.com/paper/2303.13227/) (2023)
- [Learning Cortical Anomaly through Masked Encoding for Unsupervised Heterogeneity Mapping](https://mlsanity.com/paper/2312.02762/) (2023)
- [Applying Deep Learning to Anomaly Detection of Russian Satellite Activity for Indications Prior to Military Activity](https://mlsanity.com/paper/2509.00050/) (2025)

## Abstract (arXiv)

Variational Auto-Encoders (VAEs) have shown great potential in the unsupervised learning of data distributions. An VAE trained on normal images is expected to only be able to reconstruct normal images, allowing the localization of anomalous pixels in an image via manipulating information within the VAE ELBO loss. The ELBO consists of KL divergence loss (image-wise) and reconstruction loss (pixel-wise). It is natural and straightforward to use the later as the predictor. However, usually local anomaly added to a normal image can deteriorate the whole reconstructed image, causing segmentation using only naive pixel errors not accurate. Energy based projection was proposed to increase the reconstruction accuracy of normal regions/pixels, which achieved the state-of-the-art localization accuracy on simple natural images. Another possible predictors are ELBO and its components gradients with respect to each pixels. Previous work claimed that KL gradient is a robust predictor. In this paper, we argue that the energy based projection in medical imaging is not as useful as on natural images. Moreover, we observe that the robustness of KL gradient predictor totally depends on the setting of the VAE and dataset. We also explored the effect of the weight of KL loss within beta-VAE and predictor ensemble in anomaly localization.