# ADTR: Anomaly Detection Transformer with Feature Reconstruction

arXiv: https://arxiv.org/abs/2209.01816 · HTML: https://mlsanity.com/paper/2209.01816/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 5 Sep 2022.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)
- [Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings](https://mlsanity.com/paper/1911.02357/) (2019)
- [Sub-Image Anomaly Detection with Deep Pyramid Correspondences](https://mlsanity.com/paper/2005.02357/) (2020)
- [Multiresolution Knowledge Distillation for Anomaly Detection](https://mlsanity.com/paper/2011.11108/) (2020)
- [Learning Memory-guided Normality for Anomaly Detection](https://mlsanity.com/paper/2003.13228/) (2020)
- [Deep Anomaly Detection Using Geometric Transformations](https://mlsanity.com/paper/1805.10917/) (2018)
- [Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders](https://mlsanity.com/paper/1807.02011/) (2018)
- [VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2104.10036/) (2021)
- [Patch SVDD: Patch-level SVDD for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2006.16067/) (2020)
- [Adversarially Learned One-Class Classifier for Novelty Detection](https://mlsanity.com/paper/1802.09088/) (2018)
- [OCGAN: One-class Novelty Detection Using GANs with Constrained Latent Representations](https://mlsanity.com/paper/1903.08550/) (2019)
- [Latent Space Autoregression for Novelty Detection](https://mlsanity.com/paper/1807.01653/) (2018)
- [Explainable Deep One-Class Classification](https://mlsanity.com/paper/2007.01760/) (2020)
- [Old is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigm](https://mlsanity.com/paper/2004.07657/) (2020)
- [Towards Visually Explaining Variational Autoencoders](https://mlsanity.com/paper/1911.07389/) (2019)
- [Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation](https://mlsanity.com/paper/2003.08440/) (2020)
- [Inpainting Transformer for Anomaly Detection](https://mlsanity.com/paper/2104.13897/) (2021)
- [Backpropagated Gradient Representations for Anomaly Detection](https://mlsanity.com/paper/2007.09507/) (2020)
- [Encoding Structure-Texture Relation with P-Net for Anomaly Detection in Retinal Images](https://mlsanity.com/paper/2008.03632/) (2020)
- [Iterative energy-based projection on a normal data manifold for anomaly localization](https://mlsanity.com/paper/2002.03734/) (2020)
- [AnoViT: Unsupervised Anomaly Detection and Localization with Vision Transformer-based Encoder-Decoder](https://mlsanity.com/paper/2203.10808/) (2022)
- [Anomaly Detection using Autoencoders in High Performance Computing Systems](https://mlsanity.com/paper/1811.05269/) (2018)

## Built on by (most cited first, 30 of 31)

- [Deep Industrial Image Anomaly Detection: A Survey](https://mlsanity.com/paper/2301.11514/) (2023)
- [BMAD: Benchmarks for Medical Anomaly Detection](https://mlsanity.com/paper/2306.11876/) (2023)
- [AA-CLIP: Enhancing Zero-shot Anomaly Detection via Anomaly-Aware CLIP](https://mlsanity.com/paper/2503.06661/) (2025)
- [ReContrast: Domain-Specific Anomaly Detection via Contrastive Reconstruction](https://mlsanity.com/paper/2306.02602/) (2023)
- [Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2312.07495/) (2023)
- [PAD: A Dataset and Benchmark for Pose-agnostic Anomaly Detection](https://mlsanity.com/paper/2310.07716/) (2023)
- [MedIAnomaly: A comparative study of anomaly detection in medical images](https://mlsanity.com/paper/2404.04518/) (2024)
- [Learning Unified Reference Representation for Unsupervised Multi-class Anomaly Detection](https://mlsanity.com/paper/2403.11561/) (2024)
- [Learning Feature Inversion for Multi-class Anomaly Detection under General-purpose COCO-AD Benchmark](https://mlsanity.com/paper/2404.10760/) (2024)
- [Towards Accurate Unified Anomaly Segmentation](https://mlsanity.com/paper/2501.12295/) (2025)
- [Unsupervised Anomaly Detection via Masked Diffusion Posterior Sampling](https://mlsanity.com/paper/2404.17900/) (2024)
- [CableInspect-AD: An Expert-Annotated Anomaly Detection Dataset](https://mlsanity.com/paper/2409.20353/) (2024)
- [Absolute-Unified Multi-Class Anomaly Detection via Class-Agnostic Distribution Alignment](https://mlsanity.com/paper/2404.00724/) (2024)
- [Enhancing Multi-Class Anomaly Detection via Diffusion Refinement with Dual Conditioning](https://mlsanity.com/paper/2407.01905/) (2024)
- [ALMRR: Anomaly Localization Mamba on Industrial Textured Surface with Feature Reconstruction and Refinement](https://mlsanity.com/paper/2407.17705/) (2024)
- [Few-shot Human Action Anomaly Detection via a Unified Contrastive Learning Framework](https://mlsanity.com/paper/2508.17726/) (2025)
- [ART-ASyn: Anatomy-aware Realistic Texture-based Anomaly Synthesis Framework for Chest X-Rays](https://mlsanity.com/paper/2512.00310/) (2025)
- [Multimodal Industrial Anomaly Detection via Geometric Prior](https://mlsanity.com/paper/2603.22757/) (2026)
- [Feature Attenuation of Defective Representation Can Resolve Incomplete Masking on Anomaly Detection](https://mlsanity.com/paper/2407.04597/) (2024)
- [Attention-Guided Perturbation Network for Industrial Anomaly Detection](https://mlsanity.com/paper/2408.07490/) (2024)
- [Unsupervised Hybrid framework for ANomaly Detection (HAND) -- applied to Screening Mammogram](https://mlsanity.com/paper/2409.11534/) (2024)
- [Evaluating Vision Transformer Models for Visual Quality Control in Industrial Manufacturing](https://mlsanity.com/paper/2411.14953/) (2024)
- [Out-of-Distribution Detection with Overlap Index](https://mlsanity.com/paper/2412.06168/) (2024)
- [Noise Fusion-based Distillation Learning for Anomaly Detection in Complex Industrial Environments](https://mlsanity.com/paper/2506.16050/) (2025)
- [GATE-AD: Graph Attention Network Encoding For Few-Shot Industrial Visual Anomaly Detection](https://mlsanity.com/paper/2603.15300/) (2026)
- [Semantic Iterative Reconstruction: One-Shot Universal Anomaly Detection](https://mlsanity.com/paper/2603.23766/) (2026)
- [Dual Prototype-Conditioned Diffusion Model for Scalable Multi-Class Unsupervised Anomaly Detection in Large Category Spaces](https://mlsanity.com/paper/2605.24402/) (2026)
- [Unsupervised Memory-Enhanced Video Transformers: Obstacle Detection for Autonomous Agricultural Rover](https://mlsanity.com/paper/2606.26151/) (2026)
- [Statistical Non-linear Reconstruction Loss for Image Anomaly Detection](https://mlsanity.com/paper/2607.12866/) (2026)
- [XMatchAD: A Cross-Modal Matching Perspective on Reconstruction-based Anomaly Detection](https://mlsanity.com/paper/2607.23658/) (2026)

## Abstract (arXiv)

Anomaly detection with only prior knowledge from normal samples attracts more attention because of the lack of anomaly samples. Existing CNN-based pixel reconstruction approaches suffer from two concerns. First, the reconstruction source and target are raw pixel values that contain indistinguishable semantic information. Second, CNN tends to reconstruct both normal samples and anomalies well, making them still hard to distinguish. In this paper, we propose Anomaly Detection TRansformer (ADTR) to apply a transformer to reconstruct pre-trained features. The pre-trained features contain distinguishable semantic information. Also, the adoption of transformer limits to reconstruct anomalies well such that anomalies could be detected easily once the reconstruction fails. Moreover, we propose novel loss functions to make our approach compatible with the normal-sample-only case and the anomaly-available case with both image-level and pixel-level labeled anomalies. The performance could be further improved by adding simple synthetic or external irrelevant anomalies. Extensive experiments are conducted on anomaly detection datasets including MVTec-AD and CIFAR-10. Our method achieves superior performance compared with all baselines.