mlsanityData to 7 Oct 2026

Advancing Pre-trained Teacher: Towards Robust Feature Discrepancy for Anomaly Detection

arXiv 2405.02068 ↗First posted 3 May 2024cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

With the wide application of knowledge distillation between an ImageNet pre-trained teacher model and a learnable student model, unsupervised anomaly detection has witnessed a significant achievement in the past few years. The success of this framework mainly relies on how to keep the feature discrepancy between the teacher and student model, in which it has two underlying sub-assumptions: (1) The teacher model can represent two separable distributions for the normal and abnormal patterns, while (2) the student model can only reconstruct the normal distribution. However, it still remains a challenging issue to maintain these ideal assumptions in practice. In this paper, we propose a simple yet effective two-stage industrial anomaly detection framework, termed AAND, which sequentially performs Anomaly Amplification and Normality Distillation to enhance the two assumptions. In the first anomaly amplification stage, we propose a novel Residual Anomaly Amplification (RAA) module to advance the pre-trained teacher encoder with synthetic anomalies. It generates adaptive residuals to amplify anomalies while maintaining the feature integrity of pre-trained model. It mainly comprises a Matching-guided Residual Gate and an Attribute-scaling Residual Generator, which can determine the residuals' proportion and characteristic, respectively. In the second normality distillation stage, we further employ a reverse distillation paradigm to train a student decoder, in which a novel Hard Knowledge Distillation (HKD) loss is built to better facilitate the reconstruction of normal patterns. Comprehensive experiments on the MvTecAD, VisA, and MvTec3D-RGB datasets show that our method achieves state-of-the-art performance.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 35 of its 52 references that are in the map. First posted on arXiv 3 May 2024.

Builds on

35 of 52 references.

References · 35

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2016Wide Residual Networkscited by 3,057
  5. 2015Variational Inference with Normalizing Flowscited by 2,161
  6. 2013Describing Textures in the Wildcited by 2,110
  7. 2021Towards Total Recall in Industrial Anomaly Detectioncited by 772
  8. 2020PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localizationcited by 591
  9. 2017Generative Adversarial Networks: An Overviewcited by 588
  10. 2019Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detectioncited by 560
  11. 2021CutPaste: Self-Supervised Learning for Anomaly Detection and Localizationcited by 529
  12. 2021DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detectioncited by 510
23 more
  1. 2022Anomaly Detection via Reverse Distillation from One-Class Embeddingcited by 419
  2. 2022SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentationcited by 397
  3. 2019Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddingscited by 386
  4. 2021CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flowscited by 320
  5. 2020Multiresolution Knowledge Distillation for Anomaly Detectioncited by 285
  6. 2023SimpleNet: A Simple Network for Image Anomaly Detection and Localizationcited by 256
  7. 2020Same Same But DifferNet: Semi-Supervised Defect Detection with Normalizing Flowscited by 212
  8. 2021The MVTec 3D-AD Dataset for Unsupervised 3D Anomaly Detection and Localizationcited by 174
  9. 2022DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detectioncited by 153
  10. 2021Student-Teacher Feature Pyramid Matching for Anomaly Detectioncited by 145
  11. 2023Multimodal Industrial Anomaly Detection via Hybrid Fusioncited by 138
  12. 2021Fully Convolutional Cross-Scale-Flows for Image-based Defect Detectioncited by 117
  13. 2021Natural Synthetic Anomalies for Self-Supervised Anomaly Detection and Localizationcited by 116
  14. 2022Asymmetric Student-Teacher Networks for Industrial Anomaly Detectioncited by 115
  15. 2022DSR -- A dual subspace re-projection network for surface anomaly detectioncited by 89
  16. 2024Harnessing Large Language Models for Training-free Video Anomaly Detectioncited by 85
  17. 2021Divide-and-Assemble: Learning Block-wise Memory for Unsupervised Anomaly Detectioncited by 83
  18. 2022Explicit Boundary Guided Semi-Push-Pull Contrastive Learning for Supervised Anomaly Detectioncited by 72
  19. 2022Factorizing Knowledge in Neural Networkscited by 59
  20. 2023Focus the Discrepancy: Intra- and Inter-Correlation Learning for Image Anomaly Detectioncited by 32
  21. 2022Stabilizing Adversarially Learned One-Class Novelty Detection Using Pseudo Anomaliescited by 14
  22. 2019Structuring Autoencoderscited by 11
  23. 2022Unsupervised Visual Defect Detection with Score-Based Generative Modelcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Action Hints: Semantic Typicality and Context Uniqueness for Generalizable Skeleton-based Video Anomaly Detectioncited by 1
  2. 2026DiFA: Dual Evidence Fusion and Aggregation for Token-Level Text Anomaly Detectioncited by 0