# Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detection

arXiv: https://arxiv.org/abs/2312.07495 · HTML: https://mlsanity.com/paper/2312.07495/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 96 of its 118 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 30 of 96)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detection](https://mlsanity.com/paper/2108.07610/) (2021)
- [ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation](https://mlsanity.com/paper/2204.12484/) (2022)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)

## Built on by (most cited first, 30 of 47)

- [Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection](https://mlsanity.com/paper/2403.12580/) (2024)
- [AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2407.15795/) (2024)
- [MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2404.06564/) (2024)
- [Dinomaly: The Less Is More Philosophy in Multi-Class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2405.14325/) (2024)
- [A Survey on Visual Anomaly Detection: Challenge, Approach, and Prospect](https://mlsanity.com/paper/2401.16402/) (2024)
- [M3DM-NR: RGB-3D Noisy-Resistant Industrial Anomaly Detection via Multimodal Denoising](https://mlsanity.com/paper/2406.02263/) (2024)
- [Learning Feature Inversion for Multi-class Anomaly Detection under General-purpose COCO-AD Benchmark](https://mlsanity.com/paper/2404.10760/) (2024)
- [Salvaging the Overlooked: Leveraging Class-Aware Contrastive Learning for Multi-Class Anomaly Detection](https://mlsanity.com/paper/2412.04769/) (2024)
- [Correcting Deviations from Normality: A Reformulated Diffusion Model for Multi-Class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2503.19357/) (2025)
- [Learning Multi-view Anomaly Detection with Efficient Adaptive Selection](https://mlsanity.com/paper/2407.11935/) (2024)
- [RAD: A Comprehensive Dataset for Benchmarking the Robustness of Image Anomaly Detection](https://mlsanity.com/paper/2406.07176/) (2024)
- [Foundation Models and Transformers for Anomaly Detection: A Survey](https://mlsanity.com/paper/2507.15905/) (2025)
- [Dual-path Frequency Discriminators for Few-shot Anomaly Detection](https://mlsanity.com/paper/2403.04151/) (2024)
- [A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection](https://mlsanity.com/paper/2406.03262/) (2024)
- [Towards Accurate Unified Anomaly Segmentation](https://mlsanity.com/paper/2501.12295/) (2025)
- [INP-Former++: Advancing Universal Anomaly Detection via Intrinsic Normal Prototypes and Residual Learning](https://mlsanity.com/paper/2506.03660/) (2025)
- [Variational Autoencoder for Anomaly Detection: A Comparative Study](https://mlsanity.com/paper/2408.13561/) (2024)
- [DMAD: Dual Memory Bank for Real-World Anomaly Detection](https://mlsanity.com/paper/2403.12362/) (2024)
- [Small Object Few-shot Segmentation for Vision-based Industrial Inspection](https://mlsanity.com/paper/2407.21351/) (2024)
- [VQ-Flow: Taming Normalizing Flows for Multi-Class Anomaly Detection via Hierarchical Vector Quantization](https://mlsanity.com/paper/2409.00942/) (2024)
- [ROADS: Robust Prompt-driven Multi-Class Anomaly Detection under Domain Shift](https://mlsanity.com/paper/2411.16049/) (2024)
- [Robust Distribution Alignment for Industrial Anomaly Detection under Distribution Shift](https://mlsanity.com/paper/2503.14910/) (2025)
- [Towards an Incremental Unified Multimodal Anomaly Detection: Augmenting Multimodal Denoising From an Information Bottleneck Perspective](https://mlsanity.com/paper/2603.02629/) (2026)
- [MiniMaxAD: A Lightweight Autoencoder for Feature-Rich Anomaly Detection](https://mlsanity.com/paper/2405.09933/) (2024)
- [VLMDiff: Leveraging Vision-Language Models for Multi-Class Anomaly Detection with Diffusion](https://mlsanity.com/paper/2511.08173/) (2025)
- [Is Task-Specific Training Necessary for Anomaly Detection?](https://mlsanity.com/paper/2601.22763/) (2026)
- [RAID: Retrieval-Augmented Anomaly Detection](https://mlsanity.com/paper/2602.19611/) (2026)
- [Flow Mismatching: Unsupervised Anomaly Detection via Velocity Discrepancies in Flow Matching Models](https://mlsanity.com/paper/2605.23070/) (2026)
- [Search is All You Need for Few-shot Anomaly Detection](https://mlsanity.com/paper/2504.11895/) (2025)
- [HSS-IAD: A Heterogeneous Same-Sort Industrial Anomaly Detection Dataset](https://mlsanity.com/paper/2504.12689/) (2025)

## Abstract (arXiv)

This work studies a challenging and practical issue known as multi-class unsupervised anomaly detection (MUAD). This problem requires only normal images for training while simultaneously testing both normal and anomaly images across multiple classes. Existing reconstruction-based methods typically adopt pyramidal networks as encoders and decoders to obtain multi-resolution features, often involving complex sub-modules with extensive handcraft engineering. In contrast, a plain Vision Transformer (ViT) showcasing a more straightforward architecture has proven effective in multiple domains, including detection and segmentation tasks. It is simpler, more effective, and elegant. Following this spirit, we explore the use of only plain ViT features for MUAD. We first abstract a Meta-AD concept by synthesizing current reconstruction-based methods. Subsequently, we instantiate a novel ViT-based ViTAD structure, designed incrementally from both global and local perspectives. This model provide a strong baseline to facilitate future research. Additionally, this paper uncovers several intriguing findings for further investigation. Finally, we comprehensively and fairly benchmark various approaches using eight metrics. Utilizing a basic training regimen with only an MSE loss, ViTAD achieves state-of-the-art results and efficiency on MVTec AD, VisA, and Uni-Medical datasets. Eg, achieving 85.4 mAD that surpasses UniAD by +3.0 for the MVTec AD dataset, and it requires only 1.1 hours and 2.3G GPU memory to complete model training on a single V100 that can serve as a strong baseline to facilitate the development of future research. Full code is available at https://zhangzjn.github.io/projects/ViTAD/.