# Anomaly detection through latent space restoration using vector-quantized variational autoencoders

arXiv: https://arxiv.org/abs/2012.06765 · HTML: https://mlsanity.com/paper/2012.06765/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2020.

## Built on by (most cited first, 30 of 45)

- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [Natural Synthetic Anomalies for Self-Supervised Anomaly Detection and Localization](https://mlsanity.com/paper/2109.15222/) (2021)
- [Vector Quantized Time Series Generation with a Bidirectional Prior Model](https://mlsanity.com/paper/2303.04743/) (2023)
- [Detecting Outliers with Poisson Image Interpolation](https://mlsanity.com/paper/2107.02622/) (2021)
- [Disentanglement via Latent Quantization](https://mlsanity.com/paper/2305.18378/) (2023)
- [MedIAnomaly: A comparative study of anomaly detection in medical images](https://mlsanity.com/paper/2404.04518/) (2024)
- [MSFlow: Multi-Scale Flow-based Framework for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2308.15300/) (2023)
- [Segment Every Out-of-Distribution Object](https://mlsanity.com/paper/2311.16516/) (2023)
- [Implicit field learning for unsupervised anomaly detection in medical images](https://mlsanity.com/paper/2106.05214/) (2021)
- [Dual-Distribution Discrepancy for Anomaly Detection in Chest X-Rays](https://mlsanity.com/paper/2206.03935/) (2022)
- [Explainable Time Series Anomaly Detection using Masked Latent Generative Modeling](https://mlsanity.com/paper/2311.12550/) (2023)
- [Evaluation of pseudo-healthy image reconstruction for anomaly detection with deep generative models: Application to brain FDG PET](https://mlsanity.com/paper/2401.16363/) (2024)
- [Unmasking the Chameleons: A Benchmark for Out-of-Distribution Detection in Medical Tabular Data](https://mlsanity.com/paper/2309.16220/) (2023)
- [DISYRE: Diffusion-Inspired SYnthetic REstoration for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2311.15453/) (2023)
- [Cross Attention Transformers for Multi-modal Unsupervised Whole-Body PET Anomaly Detection](https://mlsanity.com/paper/2304.07147/) (2023)
- [Binary Noise for Binary Tasks: Masked Bernoulli Diffusion for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2403.11667/) (2024)
- [StRegA: Unsupervised Anomaly Detection in Brain MRIs using a Compact Context-encoding Variational Autoencoder](https://mlsanity.com/paper/2201.13271/) (2022)
- [Scale-Aware Contrastive Reverse Distillation for Unsupervised Medical Anomaly Detection](https://mlsanity.com/paper/2503.13828/) (2025)
- [CRADL: Contrastive Representations for Unsupervised Anomaly Detection and Localization](https://mlsanity.com/paper/2301.02126/) (2023)
- [cOOpD: Reformulating COPD classification on chest CT scans as anomaly detection using contrastive representations](https://mlsanity.com/paper/2307.07254/) (2023)
- [Out-of-Distribution Radar Detection in Compound Clutter and Thermal Noise through Variational Autoencoders](https://mlsanity.com/paper/2503.04861/) (2025)
- [Dual Distillation for Few-Shot Anomaly Detection](https://mlsanity.com/paper/2603.01713/) (2026)
- [nnOOD: A Framework for Benchmarking Self-supervised Anomaly Localisation Methods](https://mlsanity.com/paper/2209.01124/) (2022)
- [Anomaly Detection using Generative Models and Sum-Product Networks in Mammography Scans](https://mlsanity.com/paper/2210.06188/) (2022)
- [Burnt area extraction from high-resolution satellite images based on anomaly detection](https://mlsanity.com/paper/2308.13367/) (2023)
- [Histogram- and Diffusion-Based Medical Out-of-Distribution Detection](https://mlsanity.com/paper/2310.08654/) (2023)
- [Q-Former Autoencoder: A Modern Framework for Medical Anomaly Detection](https://mlsanity.com/paper/2507.18481/) (2025)
- [A Principled Approach to Unsupervised Anomaly Detection](https://mlsanity.com/paper/2609.21800/) (2026)
- [Unsupervised Anomaly Detection with Local-Sensitive VQVAE and Global-Sensitive Transformers](https://mlsanity.com/paper/2303.17505/) (2023)
- [Unsupervised Brain Tumor Segmentation with Image-based Prompts](https://mlsanity.com/paper/2304.01472/) (2023)

## Abstract (arXiv)

We propose an out-of-distribution detection method that combines density and restoration-based approaches using Vector-Quantized Variational Auto-Encoders (VQ-VAEs). The VQ-VAE model learns to encode images in a categorical latent space. The prior distribution of latent codes is then modelled using an Auto-Regressive (AR) model. We found that the prior probability estimated by the AR model can be useful for unsupervised anomaly detection and enables the estimation of both sample and pixel-wise anomaly scores. The sample-wise score is defined as the negative log-likelihood of the latent variables above a threshold selecting highly unlikely codes. Additionally, out-of-distribution images are restored into in-distribution images by replacing unlikely latent codes with samples from the prior model and decoding to pixel space. The average L1 distance between generated restorations and original image is used as pixel-wise anomaly score. We tested our approach on the MOOD challenge datasets, and report higher accuracies compared to a standard reconstruction-based approach with VAEs.