# Unsupervised Anomaly Detection with Local-Sensitive VQVAE and Global-Sensitive Transformers

arXiv: https://arxiv.org/abs/2303.17505 · HTML: https://mlsanity.com/paper/2303.17505/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 18 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Builds on (most cited first, 16 of 16)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detection](https://mlsanity.com/paper/2108.07610/) (2021)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings](https://mlsanity.com/paper/1911.02357/) (2019)
- [Autoencoders for Unsupervised Anomaly Segmentation in Brain MR Images: A Comparative Study](https://mlsanity.com/paper/2004.03271/) (2020)
- [Divide-and-Assemble: Learning Block-wise Memory for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2107.13118/) (2021)
- [Unsupervised Lesion Detection via Image Restoration with a Normative Prior](https://mlsanity.com/paper/2005.00031/) (2020)
- [Anomaly detection through latent space restoration using vector-quantized variational autoencoders](https://mlsanity.com/paper/2012.06765/) (2020)
- [CaraNet: Context Axial Reverse Attention Network for Segmentation of Small Medical Objects](https://mlsanity.com/paper/2301.13366/) (2023)
- [Unsupervised Brain Anomaly Detection and Segmentation with Transformers](https://mlsanity.com/paper/2102.11650/) (2021)
- [Unsupervised brain lesion segmentation from MRI using a convolutional autoencoder](https://mlsanity.com/paper/1811.09655/) (2018)
- [Unsupervised Anomaly Segmentation using Image-Semantic Cycle Translation](https://mlsanity.com/paper/2103.09094/) (2021)

## Built on by (most cited first, 1 of 1)

- [HarmoniAD: Harmonizing Local Structures and Global Semantics for Anomaly Detection](https://mlsanity.com/paper/2601.00327/) (2026)

## Abstract (arXiv)

Unsupervised anomaly detection (UAD) has been widely implemented in industrial and medical applications, which reduces the cost of manual annotation and improves efficiency in disease diagnosis. Recently, deep auto-encoder with its variants has demonstrated its advantages in many UAD scenarios. Training on the normal data, these models are expected to locate anomalies by producing higher reconstruction error for the abnormal areas than the normal ones. However, this assumption does not always hold because of the uncontrollable generalization capability. To solve this problem, we present LSGS, a method that builds on Vector Quantised-Variational Autoencoder (VQVAE) with a novel aggregated codebook and transformers with global attention. In this work, the VQVAE focus on feature extraction and reconstruction of images, and the transformers fit the manifold and locate anomalies in the latent space. Then, leveraging the generated encoding sequences that conform to a normal distribution, we can reconstruct a more accurate image for locating the anomalies. Experiments on various datasets demonstrate the effectiveness of the proposed method.