# AnoViT: Unsupervised Anomaly Detection and Localization with Vision Transformer-based Encoder-Decoder

arXiv: https://arxiv.org/abs/2203.10808 · HTML: https://mlsanity.com/paper/2203.10808/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks](https://mlsanity.com/paper/1406.6909/) (2014)
- [Sub-Image Anomaly Detection with Deep Pyramid Correspondences](https://mlsanity.com/paper/2005.02357/) (2020)
- [Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders](https://mlsanity.com/paper/1807.02011/) (2018)
- [Patch SVDD: Patch-level SVDD for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2006.16067/) (2020)
- [Diagnosing and Enhancing VAE Models](https://mlsanity.com/paper/1903.05789/) (2019)
- [Deep-Anomaly: Fully Convolutional Neural Network for Fast Anomaly Detection in Crowded Scenes](https://mlsanity.com/paper/1609.00866/) (2016)
- [Visual Anomaly Detection for Images: A Survey](https://mlsanity.com/paper/2109.13157/) (2021)
- [Image/Video Deep Anomaly Detection: A Survey](https://mlsanity.com/paper/2103.01739/) (2021)
- [Medical Image Segmentation Based on Multi-Modal Convolutional Neural Network: Study on Image Fusion Schemes](https://mlsanity.com/paper/1711.00049/) (2017)

## Built on by (most cited first, 29 of 29)

- [A Unified Model for Multi-class Anomaly Detection](https://mlsanity.com/paper/2206.03687/) (2022)
- [Deep Industrial Image Anomaly Detection: A Survey](https://mlsanity.com/paper/2301.11514/) (2023)
- [BMAD: Benchmarks for Medical Anomaly Detection](https://mlsanity.com/paper/2306.11876/) (2023)
- [Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2312.07495/) (2023)
- [Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection](https://mlsanity.com/paper/2209.12148/) (2022)
- [ADTR: Anomaly Detection Transformer with Feature Reconstruction](https://mlsanity.com/paper/2209.01816/) (2022)
- [Learning Unified Reference Representation for Unsupervised Multi-class Anomaly Detection](https://mlsanity.com/paper/2403.11561/) (2024)
- [Learning to Detect Multi-class Anomalies with Just One Normal Image Prompt](https://mlsanity.com/paper/2505.09264/) (2025)
- [A Comprehensive Survey of Transformers for Computer Vision](https://mlsanity.com/paper/2211.06004/) (2022)
- [Long-Tailed Anomaly Detection with Learnable Class Names](https://mlsanity.com/paper/2403.20236/) (2024)
- [Foundation Models and Transformers for Anomaly Detection: A Survey](https://mlsanity.com/paper/2507.15905/) (2025)
- [Variational Autoencoder for Anomaly Detection: A Comparative Study](https://mlsanity.com/paper/2408.13561/) (2024)
- [CableInspect-AD: An Expert-Annotated Anomaly Detection Dataset](https://mlsanity.com/paper/2409.20353/) (2024)
- [Generalizable Industrial Visual Anomaly Detection with Self-Induction Vision Transformer](https://mlsanity.com/paper/2211.12311/) (2022)
- [ISSTAD: Incremental Self-Supervised Learning Based on Transformer for Anomaly Detection and Localization](https://mlsanity.com/paper/2303.17354/) (2023)
- [Creating an Atlas of Normal Tissue for Pruning WSI Patching Through Anomaly Detection](https://mlsanity.com/paper/2310.03106/) (2023)
- [Self-Supervised Training with Autoencoders for Visual Anomaly Detection](https://mlsanity.com/paper/2206.11723/) (2022)
- [Multimodal Sleep Apnea Detection with Missing or Noisy Modalities](https://mlsanity.com/paper/2402.17788/) (2024)
- [Critical Review for One-class Classification: recent advances and the reality behind them](https://mlsanity.com/paper/2404.17931/) (2024)
- [Identifying Spatio-Temporal Drivers of Extreme Events](https://mlsanity.com/paper/2410.24075/) (2024)
- [User-Based Sequential Modeling with Transformer Encoders for Insider Threat Detection](https://mlsanity.com/paper/2506.23446/) (2025)
- [Feature Purified Transformer With Cross-level Feature Guiding Decoder For Multi-class OOD and Anomaly Deteciton](https://mlsanity.com/paper/2406.15396/) (2024)
- [Feature Attenuation of Defective Representation Can Resolve Incomplete Masking on Anomaly Detection](https://mlsanity.com/paper/2407.04597/) (2024)
- [Unsupervised Hybrid framework for ANomaly Detection (HAND) -- applied to Screening Mammogram](https://mlsanity.com/paper/2409.11534/) (2024)
- [CONSULT: Contrastive Self-Supervised Learning for Few-shot Tumor Detection](https://mlsanity.com/paper/2410.11307/) (2024)
- [Surface Defect Detection with Gabor Filter Using Reconstruction-Based Blurring U-Net-ViT](https://mlsanity.com/paper/2509.00827/) (2025)
- [Novel Anomaly Detection Scenarios and Evaluation Metrics to Address the Ambiguity in the Definition of Normal Samples](https://mlsanity.com/paper/2604.07097/) (2026)
- [Unsupervised Memory-Enhanced Video Transformers: Obstacle Detection for Autonomous Agricultural Rover](https://mlsanity.com/paper/2606.26151/) (2026)
- [Identifying the Unknown: Prompt-Free Open Vocabulary Anomaly Recognition for Robot-Object Interaction](https://mlsanity.com/paper/2606.26829/) (2026)

## Abstract (arXiv)

Image anomaly detection problems aim to determine whether an image is abnormal, and to detect anomalous areas. These methods are actively used in various fields such as manufacturing, medical care, and intelligent information. Encoder-decoder structures have been widely used in the field of anomaly detection because they can easily learn normal patterns in an unsupervised learning environment and calculate a score to identify abnormalities through a reconstruction error indicating the difference between input and reconstructed images. Therefore, current image anomaly detection methods have commonly used convolutional encoder-decoders to extract normal information through the local features of images. However, they are limited in that only local features of the image can be utilized when constructing a normal representation owing to the characteristics of convolution operations using a filter of fixed size. Therefore, we propose a vision transformer-based encoder-decoder model, named AnoViT, designed to reflect normal information by additionally learning the global relationship between image patches, which is capable of both image anomaly detection and localization. The proposed approach constructs a feature map that maintains the existing location information of individual patches by using the embeddings of all patches passed through multiple self-attention layers. The proposed AnoViT model performed better than the convolution-based model on three benchmark datasets. In MVTecAD, which is a representative benchmark dataset for anomaly localization, it showed improved results on 10 out of 15 classes compared with the baseline. Furthermore, the proposed method showed good performance regardless of the class and type of the anomalous area when localization results were evaluated qualitatively.