# Siamese Transition Masked Autoencoders as Uniform Unsupervised Visual Anomaly Detector

arXiv: https://arxiv.org/abs/2211.00349 · HTML: https://mlsanity.com/paper/2211.00349/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 57 references that are in the map.
- First posted on arXiv 1 Nov 2022.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)
- [Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings](https://mlsanity.com/paper/1911.02357/) (2019)
- [Recent Advances in Convolutional Neural Networks](https://mlsanity.com/paper/1512.07108/) (2015)
- [Sub-Image Anomaly Detection with Deep Pyramid Correspondences](https://mlsanity.com/paper/2005.02357/) (2020)
- [Multiresolution Knowledge Distillation for Anomaly Detection](https://mlsanity.com/paper/2011.11108/) (2020)
- [Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders](https://mlsanity.com/paper/1807.02011/) (2018)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction](https://mlsanity.com/paper/1611.09842/) (2016)
- [Patch SVDD: Patch-level SVDD for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2006.16067/) (2020)
- [Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images](https://mlsanity.com/paper/1804.04488/) (2018)
- [Skip-GANomaly: Skip Connected and Adversarially Trained Encoder-Decoder Anomaly Detection](https://mlsanity.com/paper/1901.08954/) (2019)
- [Inpainting Transformer for Anomaly Detection](https://mlsanity.com/paper/2104.13897/) (2021)
- [Learning Deep Features for One-Class Classification](https://mlsanity.com/paper/1801.05365/) (2018)
- [Iterative energy-based projection on a normal data manifold for anomaly localization](https://mlsanity.com/paper/2002.03734/) (2020)
- [Robust Unsupervised Video Anomaly Detection by Multi-Path Frame Prediction](https://mlsanity.com/paper/2011.02763/) (2020)
- [Focus Your Distribution: Coarse-to-Fine Non-Contrastive Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2110.04538/) (2021)
- [Proxy-bridged Image Reconstruction Network for Anomaly Detection in Medical Images](https://mlsanity.com/paper/2110.01761/) (2021)
- [A Feature Memory Rearrangement Network for Visual Inspection of Textured Surface Defects Toward Edge Intelligent Manufacturing](https://mlsanity.com/paper/2206.10830/) (2022)

## Built on by (most cited first, 9 of 9)

- [Learning Global-Local Correspondence with Semantic Bottleneck for Logical Anomaly Detection](https://mlsanity.com/paper/2303.05768/) (2023)
- [AMI-Net: Adaptive Mask Inpainting Network for Industrial Anomaly Detection and Localization](https://mlsanity.com/paper/2412.11802/) (2024)
- [Normal Reference Attention and Defective Feature Perception Network for Surface Defect Detection](https://mlsanity.com/paper/2211.10060/) (2022)
- [Foundation Models and Transformers for Anomaly Detection: A Survey](https://mlsanity.com/paper/2507.15905/) (2025)
- [Generalizable Industrial Visual Anomaly Detection with Self-Induction Vision Transformer](https://mlsanity.com/paper/2211.12311/) (2022)
- [Aerial Image Object Detection With Vision Transformer Detector (ViTDet)](https://mlsanity.com/paper/2301.12058/) (2023)
- [LADMIM: Logical Anomaly Detection with Masked Image Modeling in Discrete Latent Space](https://mlsanity.com/paper/2410.10234/) (2024)
- [A Feature Shuffling and Restoration Strategy for Universal Unsupervised Anomaly Detection](https://mlsanity.com/paper/2603.22861/) (2026)
- [Template-Based Feature Aggregation Network for Industrial Anomaly Detection](https://mlsanity.com/paper/2603.22874/) (2026)

## Abstract (arXiv)

Unsupervised visual anomaly detection conveys practical significance in many scenarios and is a challenging task due to the unbounded definition of anomalies. Moreover, most previous methods are application-specific, and establishing a unified model for anomalies across application scenarios remains unsolved. This paper proposes a novel hybrid framework termed Siamese Transition Masked Autoencoders(ST-MAE) to handle various visual anomaly detection tasks uniformly via deep feature transition. Concretely, the proposed method first extracts hierarchical semantics features from a pre-trained deep convolutional neural network and then develops a feature decoupling strategy to split the deep features into two disjoint feature patch subsets. Leveraging the decoupled features, the ST-MAE is developed with the Siamese encoders that operate on each subset of feature patches and perform the latent representations transition of two subsets, along with a lightweight decoder that reconstructs the original feature from the transitioned latent representation. Finally, the anomalous attributes can be detected using the semantic deep feature residual. Our deep feature transition scheme yields a nontrivial and semantic self-supervisory task to extract prototypical normal patterns, which allows for learning uniform models that generalize well for different visual anomaly detection tasks. The extensive experiments conducted demonstrate that the proposed ST-MAE method can advance state-of-the-art performance on multiple benchmarks across application scenarios with a superior inference efficiency, which exhibits great potential to be the uniform model for unsupervised visual anomaly detection.