# Inpainting Transformer for Anomaly Detection

arXiv: https://arxiv.org/abs/2104.13897 · HTML: https://mlsanity.com/paper/2104.13897/

## Key facts

- Cited by 91 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 45 references that are in the map.
- First posted on arXiv 28 Apr 2021.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)
- [Axial Attention in Multidimensional Transformers](https://mlsanity.com/paper/1912.12180/) (2019)
- [Learning Deep Transformer Models for Machine Translation](https://mlsanity.com/paper/1906.01787/) (2019)
- [Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders](https://mlsanity.com/paper/1807.02011/) (2018)
- [Patch SVDD: Patch-level SVDD for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2006.16067/) (2020)
- [Same Same But DifferNet: Semi-Supervised Defect Detection with Normalizing Flows](https://mlsanity.com/paper/2008.12577/) (2020)
- [Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index](https://mlsanity.com/paper/1308.3052/) (2013)
- [Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images](https://mlsanity.com/paper/1804.04488/) (2018)
- [Deep Learning for Medical Anomaly Detection -- A Survey](https://mlsanity.com/paper/2012.02364/) (2020)
- [Skip-GANomaly: Skip Connected and Adversarially Trained Encoder-Decoder Anomaly Detection](https://mlsanity.com/paper/1901.08954/) (2019)
- [Anomaly Detection using Deep Learning based Image Completion](https://mlsanity.com/paper/1811.06861/) (2018)
- [q-Space Novelty Detection with Variational Autoencoders](https://mlsanity.com/paper/1806.02997/) (2018)
- [Anomaly Detection for Skin Disease Images Using Variational Autoencoder](https://mlsanity.com/paper/1807.01349/) (2018)
- [Unsupervised Region-based Anomaly Detection in Brain MRI with Adversarial Image Inpainting](https://mlsanity.com/paper/2010.01942/) (2020)

## Built on by (most cited first, 30 of 91)

- [A Unified Model for Multi-class Anomaly Detection](https://mlsanity.com/paper/2206.03687/) (2022)
- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models](https://mlsanity.com/paper/2308.15366/) (2023)
- [DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection](https://mlsanity.com/paper/2211.11317/) (2022)
- [Deep Industrial Image Anomaly Detection: A Survey](https://mlsanity.com/paper/2301.11514/) (2023)
- [RealNet: A Feature Selection Network with Realistic Synthetic Anomaly for Anomaly Detection](https://mlsanity.com/paper/2403.05897/) (2024)
- [MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2404.06564/) (2024)
- [Omni-frequency Channel-selection Representations for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2203.00259/) (2022)
- [PNI : Industrial Anomaly Detection using Position and Neighborhood Information](https://mlsanity.com/paper/2211.12634/) (2022)
- [MemSeg: A semi-supervised method for image surface defect detection using differences and commonalities](https://mlsanity.com/paper/2205.00908/) (2022)
- [Deep Learning for Unsupervised Anomaly Localization in Industrial Images: A Survey](https://mlsanity.com/paper/2207.10298/) (2022)
- [IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturing](https://mlsanity.com/paper/2301.13359/) (2023)
- [Prototypical Residual Networks for Anomaly Detection and Localization](https://mlsanity.com/paper/2212.02031/) (2022)
- [Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2312.07495/) (2023)
- [PAD: A Dataset and Benchmark for Pose-agnostic Anomaly Detection](https://mlsanity.com/paper/2310.07716/) (2023)
- [Multimodal Industrial Anomaly Detection by Crossmodal Feature Mapping](https://mlsanity.com/paper/2312.04521/) (2023)
- [Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection](https://mlsanity.com/paper/2209.12148/) (2022)
- [Focus the Discrepancy: Intra- and Inter-Correlation Learning for Image Anomaly Detection](https://mlsanity.com/paper/2308.02983/) (2023)
- [ADTR: Anomaly Detection Transformer with Feature Reconstruction](https://mlsanity.com/paper/2209.01816/) (2022)
- [PO3AD: Predicting Point Offsets toward Better 3D Point Cloud Anomaly Detection](https://mlsanity.com/paper/2412.12617/) (2024)
- [Transformaly -- Two (Feature Spaces) Are Better Than One](https://mlsanity.com/paper/2112.04185/) (2021)
- [An Incremental Unified Framework for Small Defect Inspection](https://mlsanity.com/paper/2312.08917/) (2023)
- [Industrial Anomaly Detection with Domain Shift: A Real-world Dataset and Masked Multi-scale Reconstruction](https://mlsanity.com/paper/2304.02216/) (2023)
- [A Survey on Unsupervised Anomaly Detection Algorithms for Industrial Images](https://mlsanity.com/paper/2204.11161/) (2022)
- [Learning Unified Reference Representation for Unsupervised Multi-class Anomaly Detection](https://mlsanity.com/paper/2403.11561/) (2024)
- [Reconstruction from edge image combined with color and gradient difference for industrial surface anomaly detection](https://mlsanity.com/paper/2210.14485/) (2022)
- [Progressive Boundary Guided Anomaly Synthesis for Industrial Anomaly Detection](https://mlsanity.com/paper/2412.17458/) (2024)
- [FAPM: Fast Adaptive Patch Memory for Real-time Industrial Anomaly Detection](https://mlsanity.com/paper/2211.07381/) (2022)
- [Long-Tailed Anomaly Detection with Learnable Class Names](https://mlsanity.com/paper/2403.20236/) (2024)
- [LafitE: Latent Diffusion Model with Feature Editing for Unsupervised Multi-class Anomaly Detection](https://mlsanity.com/paper/2307.08059/) (2023)

## Abstract (arXiv)

Anomaly detection in computer vision is the task of identifying images which deviate from a set of normal images. A common approach is to train deep convolutional autoencoders to inpaint covered parts of an image and compare the output with the original image. By training on anomaly-free samples only, the model is assumed to not being able to reconstruct anomalous regions properly. For anomaly detection by inpainting we suggest it to be beneficial to incorporate information from potentially distant regions. In particular we pose anomaly detection as a patch-inpainting problem and propose to solve it with a purely self-attention based approach discarding convolutions. The proposed Inpainting Transformer (InTra) is trained to inpaint covered patches in a large sequence of image patches, thereby integrating information across large regions of the input image. When training from scratch, in comparison to other methods not using extra training data, InTra achieves results on par with the current state-of-the-art on the MVTec AD dataset for detection and surpassing them on segmentation.