# Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection

arXiv: https://arxiv.org/abs/2209.12148 · HTML: https://mlsanity.com/paper/2209.12148/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 68 of its 117 references that are in the map.
- First posted on arXiv 25 Sep 2022.

## Builds on (most cited first, 30 of 68)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Masked Feature Prediction for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2112.09133/) (2021)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detection](https://mlsanity.com/paper/2108.07610/) (2021)
- [Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy](https://mlsanity.com/paper/2110.02642/) (2021)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings](https://mlsanity.com/paper/1911.02357/) (2019)
- [Multiresolution Knowledge Distillation for Anomaly Detection](https://mlsanity.com/paper/2011.11108/) (2020)
- [Learning Memory-guided Normality for Anomaly Detection](https://mlsanity.com/paper/2003.13228/) (2020)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)

## Built on by (most cited first, 30 of 34)

- [Deep Industrial Image Anomaly Detection: A Survey](https://mlsanity.com/paper/2301.11514/) (2023)
- [CaptainCook4D: A Dataset for Understanding Errors in Procedural Activities](https://mlsanity.com/paper/2312.14556/) (2023)
- [Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2312.07495/) (2023)
- [Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors](https://mlsanity.com/paper/2306.12041/) (2023)
- [Masked Image Modeling: A Survey](https://mlsanity.com/paper/2408.06687/) (2024)
- [Deep Learning for Video Anomaly Detection: A Review](https://mlsanity.com/paper/2409.05383/) (2024)
- [MedIAnomaly: A comparative study of anomaly detection in medical images](https://mlsanity.com/paper/2404.04518/) (2024)
- [Prior Normality Prompt Transformer for Multi-class Industrial Image Anomaly Detection](https://mlsanity.com/paper/2406.11507/) (2024)
- [InfMAE: A Foundation Model in the Infrared Modality](https://mlsanity.com/paper/2402.00407/) (2024)
- [SLSG: Industrial Image Anomaly Detection by Learning Better Feature Embeddings and One-Class Classification](https://mlsanity.com/paper/2305.00398/) (2023)
- [Foundation Models and Transformers for Anomaly Detection: A Survey](https://mlsanity.com/paper/2507.15905/) (2025)
- [Towards Real Unsupervised Anomaly Detection Via Confident Meta-Learning](https://mlsanity.com/paper/2508.02293/) (2025)
- [Dual-path Frequency Discriminators for Few-shot Anomaly Detection](https://mlsanity.com/paper/2403.04151/) (2024)
- [Human-Centric Video Anomaly Detection Through Spatio-Temporal Pose Tokenization and Transformer](https://mlsanity.com/paper/2408.15185/) (2024)
- [Vision-Language Models Assisted Unsupervised Video Anomaly Detection](https://mlsanity.com/paper/2409.14109/) (2024)
- [SlowFastVAD: Video Anomaly Detection via Integrating Simple Detector and RAG-Enhanced Vision-Language Model](https://mlsanity.com/paper/2504.10320/) (2025)
- [Lightning Fast Video Anomaly Detection via Adversarial Knowledge Distillation](https://mlsanity.com/paper/2211.15597/) (2022)
- [Aerial Image Object Detection With Vision Transformer Detector (ViTDet)](https://mlsanity.com/paper/2301.12058/) (2023)
- [Appearance Blur-driven AutoEncoder and Motion-guided Memory Module for Video Anomaly Detection](https://mlsanity.com/paper/2409.17608/) (2024)
- [MuSc-V2: Zero-Shot Multimodal Industrial Anomaly Classification and Segmentation with Mutual Scoring of Unlabeled Samples](https://mlsanity.com/paper/2511.10047/) (2025)
- [Improved Anomaly Detection by Using the Attention-Based Isolation Forest](https://mlsanity.com/paper/2210.02558/) (2022)
- [A Comprehensive Survey for Real-World Industrial Defect Detection: Challenges, Approaches, and Prospects](https://mlsanity.com/paper/2507.13378/) (2025)
- [Cluster-aware Contrastive Learning for Unsupervised Out-of-distribution Detection](https://mlsanity.com/paper/2302.02598/) (2023)
- [Multilevel Saliency-Guided Self-Supervised Learning for Image Anomaly Detection](https://mlsanity.com/paper/2311.18332/) (2023)
- [Sparse Anatomical Prompt Semi-Supervised Learning with Masked Image Modeling for CBCT Tooth Segmentation](https://mlsanity.com/paper/2402.04587/) (2024)
- [Identifying Spatio-Temporal Drivers of Extreme Events](https://mlsanity.com/paper/2410.24075/) (2024)
- [Multi-AD: Cross-Domain Unsupervised Anomaly Detection for Medical and Industrial Applications](https://mlsanity.com/paper/2602.05426/) (2026)
- [VADMamba++: Efficient Video Anomaly Detection via Hybrid Modeling in Grayscale Space](https://mlsanity.com/paper/2604.00360/) (2026)
- [Depth-Weighted Detection of Behaviours of Risk in People with Dementia using Cameras](https://mlsanity.com/paper/2408.15519/) (2024)
- [An aerial color image anomaly dataset for search missions in complex forested terrain](https://mlsanity.com/paper/2507.15492/) (2025)

## Abstract (arXiv)

Anomaly detection has recently gained increasing attention in the field of computer vision, likely due to its broad set of applications ranging from product fault detection on industrial production lines and impending event detection in video surveillance to finding lesions in medical scans. Regardless of the domain, anomaly detection is typically framed as a one-class classification task, where the learning is conducted on normal examples only. An entire family of successful anomaly detection methods is based on learning to reconstruct masked normal inputs (e.g. patches, future frames, etc.) and exerting the magnitude of the reconstruction error as an indicator for the abnormality level. Unlike other reconstruction-based methods, we present a novel self-supervised masked convolutional transformer block (SSMCTB) that comprises the reconstruction-based functionality at a core architectural level. The proposed self-supervised block is extremely flexible, enabling information masking at any layer of a neural network and being compatible with a wide range of neural architectures. In this work, we extend our previous self-supervised predictive convolutional attentive block (SSPCAB) with a 3D masked convolutional layer, a transformer for channel-wise attention, as well as a novel self-supervised objective based on Huber loss. Furthermore, we show that our block is applicable to a wider variety of tasks, adding anomaly detection in medical images and thermal videos to the previously considered tasks based on RGB images and surveillance videos. We exhibit the generality and flexibility of SSMCTB by integrating it into multiple state-of-the-art neural models for anomaly detection, bringing forth empirical results that confirm considerable performance improvements on five benchmarks. We release our code and data as open source at: https://github.com/ristea/ssmctb.