# Reconstruction Student with Attention for Student-Teacher Pyramid Matching

arXiv: https://arxiv.org/abs/2111.15376 · HTML: https://mlsanity.com/paper/2111.15376/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 33 references that are in the map.
- First posted on arXiv 30 Nov 2021.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)
- [Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings](https://mlsanity.com/paper/1911.02357/) (2019)
- [Sub-Image Anomaly Detection with Deep Pyramid Correspondences](https://mlsanity.com/paper/2005.02357/) (2020)
- [Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images](https://mlsanity.com/paper/1804.04488/) (2018)
- [Attention Branch Network: Learning of Attention Mechanism for Visual Explanation](https://mlsanity.com/paper/1812.10025/) (2018)
- [Anomaly Detection using One-Class Neural Networks](https://mlsanity.com/paper/1802.06360/) (2018)
- [An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos](https://mlsanity.com/paper/1801.03149/) (2018)
- [Training Adversarial Discriminators for Cross-channel Abnormal Event Detection in Crowds](https://mlsanity.com/paper/1706.07680/) (2017)
- [Metric Learning for Novelty and Anomaly Detection](https://mlsanity.com/paper/1808.05492/) (2018)
- [Accuracy Booster: Performance Boosting using Feature Map Re-calibration](https://mlsanity.com/paper/1903.04407/) (2019)

## Built on by (most cited first, 20 of 20)

- [DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection](https://mlsanity.com/paper/2211.11317/) (2022)
- [Deep Industrial Image Anomaly Detection: A Survey](https://mlsanity.com/paper/2301.11514/) (2023)
- [BMAD: Benchmarks for Medical Anomaly Detection](https://mlsanity.com/paper/2306.11876/) (2023)
- [IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturing](https://mlsanity.com/paper/2301.13359/) (2023)
- [PAD: A Dataset and Benchmark for Pose-agnostic Anomaly Detection](https://mlsanity.com/paper/2310.07716/) (2023)
- [DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection](https://mlsanity.com/paper/2312.06607/) (2023)
- [Look Inside for More: Internal Spatial Modality Perception for 3D Anomaly Detection](https://mlsanity.com/paper/2412.13461/) (2024)
- [MIAD: A Maintenance Inspection Dataset for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2211.13968/) (2022)
- [What makes a good data augmentation for few-shot unsupervised image anomaly detection?](https://mlsanity.com/paper/2304.03294/) (2023)
- [Foundation Models and Transformers for Anomaly Detection: A Survey](https://mlsanity.com/paper/2507.15905/) (2025)
- [Unsupervised Visual Defect Detection with Score-Based Generative Model](https://mlsanity.com/paper/2211.16092/) (2022)
- [MedDiff-FM: A Diffusion-based Foundation Model for Versatile Medical Image Applications](https://mlsanity.com/paper/2410.15432/) (2024)
- [Is Hyperbolic Space All You Need for Medical Anomaly Detection?](https://mlsanity.com/paper/2505.21228/) (2025)
- [MixedTeacher : Knowledge Distillation for fast inference textural anomaly detection](https://mlsanity.com/paper/2306.09859/) (2023)
- [A Recover-then-Discriminate Framework for Robust Anomaly Detection](https://mlsanity.com/paper/2406.04608/) (2024)
- [Q-Former Autoencoder: A Modern Framework for Medical Anomaly Detection](https://mlsanity.com/paper/2507.18481/) (2025)
- [Unsupervised Learning for Industrial Defect Detection: A Case Study on Shearographic Data](https://mlsanity.com/paper/2511.02541/) (2025)
- [Teacher Encoder-Student Decoder Denoising Guided Segmentation Network for Anomaly Detection](https://mlsanity.com/paper/2501.12104/) (2025)
- [Novel Anomaly Detection Scenarios and Evaluation Metrics to Address the Ambiguity in the Definition of Normal Samples](https://mlsanity.com/paper/2604.07097/) (2026)
- [MTL-MAD: Multi-Task Learners are Effective Medical Anomaly Detectors](https://mlsanity.com/paper/2605.05891/) (2026)

## Abstract (arXiv)

Anomaly detection and localization are important problems in computer vision. Recently, Convolutional Neural Network (CNN) has been used for visual inspection. In particular, the scarcity of anomalous samples increases the difficulty of this task, and unsupervised leaning based methods are attracting attention. We focus on Student-Teacher Feature Pyramid Matching (STPM) which can be trained from only normal images with small number of epochs. Here we proposed a powerful method which compensates for the shortcomings of STPM. Proposed method consists of two students and two teachers that a pair of student-teacher network is the same as STPM. The other student-teacher network has a role to reconstruct the features of normal products. By reconstructing the features of normal products from an abnormal image, it is possible to detect abnormalities with higher accuracy by taking the difference between them. The new student-teacher network uses attention modules and different teacher network from the original STPM. Attention mechanism acts to successfully reconstruct the normal regions in an input image. Different teacher network prevents looking at the same regions as the original STPM. Six anomaly maps obtained from the two student-teacher networks are used to calculate the final anomaly map. Student-teacher network for reconstructing features improved AUC scores for pixel level and image level in comparison with the original STPM.