mlsanityData to 14 Oct 2026

Anomaly Detection via Reverse Distillation from One-Class Embedding

arXiv 2201.10703 ↗First posted 26 Jan 2022cs.CVcited by 421

On the map: Knowledge distillation → Detection and dense prediction distillation

Abstract (arXiv)

Knowledge distillation (KD) achieves promising results on the challenging problem of unsupervised anomaly detection (AD).The representation discrepancy of anomalies in the teacher-student (T-S) model provides essential evidence for AD. However, using similar or identical architectures to build the teacher and student models in previous studies hinders the diversity of anomalous representations. To tackle this problem, we propose a novel T-S model consisting of a teacher encoder and a student decoder and introduce a simple yet effective "reverse distillation" paradigm accordingly. Instead of receiving raw images directly, the student network takes teacher model's one-class embedding as input and targets to restore the teacher's multiscale representations. Inherently, knowledge distillation in this study starts from abstract, high-level presentations to low-level features. In addition, we introduce a trainable one-class bottleneck embedding (OCBE) module in our T-S model. The obtained compact embedding effectively preserves essential information on normal patterns, but abandons anomaly perturbations. Extensive experimentation on AD and one-class novelty detection benchmarks shows that our method surpasses SOTA performance, demonstrating our proposed approach's effectiveness and generalizability.

Key facts. Cited by 421 papers in the map (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 31 of its 52 references that are in the map. First posted on arXiv 26 Jan 2022.

Builds on

31 of 52 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  6. 2016Wide Residual Networkscited by 3,070
  7. 2020PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localizationcited by 594
  8. 2019Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detectioncited by 564
  9. 2021CutPaste: Self-Supervised Learning for Anomaly Detection and Localizationcited by 531
  10. 2019Similarity-Preserving Knowledge Distillationcited by 516
  11. 2018GANomaly: Semi-Supervised Anomaly Detection via Adversarial Trainingcited by 409
  12. 2017Future Frame Prediction for Anomaly Detection -- A New Baselinecited by 391
19 more
  1. 2019Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddingscited by 386
  2. 2020Sub-Image Anomaly Detection with Deep Pyramid Correspondencescited by 300
  3. 2021Distilling Knowledge via Knowledge Reviewcited by 288
  4. 2020Multiresolution Knowledge Distillation for Anomaly Detectioncited by 285
  5. 2020Learning Memory-guided Normality for Anomaly Detectioncited by 278
  6. 2018Deep Anomaly Detection Using Geometric Transformationscited by 269
  7. 2018Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoderscited by 264
  8. 2020Patch SVDD: Patch-level SVDD for Anomaly Detection and Segmentationcited by 212
  9. 2019OCGAN: One-class Novelty Detection Using GANs with Constrained Latent Representationscited by 199
  10. 2020PANDA: Adapting Pretrained Features for Anomaly Detection and Segmentationcited by 175
  11. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  12. 2018Latent Space Autoregression for Novelty Detectioncited by 149
  13. 2021Student-Teacher Feature Pyramid Matching for Anomaly Detectioncited by 146
  14. 2020Old is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigmcited by 115
  15. 2020Modeling the Distribution of Normal Data in Pre-Trained Deep Features for Anomaly Detectioncited by 115
  16. 2021Divide-and-Assemble: Learning Block-wise Memory for Unsupervised Anomaly Detectioncited by 83
  17. 2020Encoding Structure-Texture Relation with P-Net for Anomaly Detection in Retinal Imagescited by 52
  18. 2020G2D: Generate to Detect Anomalycited by 42
  19. 2021Complementary Relation Contrastive Distillationcited by 42

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 16; 2023: 79; 2024: 113; 2025: 119; 2026: 94.

421 papers.

Sort:

By area:

  1. 2022SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentationcited by 400
  2. 2023AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detectioncited by 271
  3. 2022A Unified Model for Multi-class Anomaly Detectioncited by 268
  4. 2023SimpleNet: A Simple Network for Image Anomaly Detection and Localizationcited by 256
  5. 2022DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detectioncited by 154
  6. 2023Deep Industrial Image Anomaly Detection: A Surveycited by 148
  7. 2023Multimodal Industrial Anomaly Detection via Hybrid Fusioncited by 139
  8. 2024Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detectioncited by 139
  9. 2024AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detectioncited by 127
  10. 2024RealNet: A Feature Selection Network with Realistic Synthetic Anomaly for Anomaly Detectioncited by 116
  11. 2024MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detectioncited by 113
  12. 2023AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Modelcited by 111
38 more
  1. 2024Dinomaly: The Less Is More Philosophy in Multi-Class Unsupervised Anomaly Detectioncited by 93
  2. 2024Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Promptscited by 89
  3. 2023Real3D-AD: A Dataset of Point Cloud Anomaly Detectioncited by 87
  4. 2023BMAD: Benchmarks for Medical Anomaly Detectioncited by 82
  5. 2023Pushing the Limits of Fewshot Anomaly Detection in Industry Vision: Graphcorecited by 75
  6. 2023Complementary Pseudo Multimodal Feature for Point Cloud Anomaly Detectioncited by 75
  7. 2024A Unified Anomaly Synthesis Strategy with Gradient Ascent for Industrial Anomaly Detection and Localizationcited by 74
  8. 2025AA-CLIP: Enhancing Zero-shot Anomaly Detection via Anomaly-Aware CLIPcited by 73
  9. 2022PNI : Industrial Anomaly Detection using Position and Neighborhood Informationcited by 69
  10. 2024A Survey on Visual Anomaly Detection: Challenge, Approach, and Prospectcited by 69
  11. 2024Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Imagescited by 69
  12. 2023Hierarchical Vector Quantized Transformer for Multi-class Unsupervised Anomaly Detectioncited by 66
  13. 2023IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturingcited by 64
  14. 2023ReContrast: Domain-Specific Anomaly Detection via Contrastive Reconstructioncited by 63
  15. 2023Segment Any Anomaly without Training via Hybrid Prompt Regularizationcited by 60
  16. 2023Collaborative Discrepancy Optimization for Reliable Image Anomaly Localizationcited by 57
  17. 2022Prototypical Residual Networks for Anomaly Detection and Localizationcited by 56
  18. 2023Anomaly Detection with Conditioned Denoising Diffusion Modelscited by 56
  19. 2024SoftPatch: Unsupervised Anomaly Detection with Noisy Datacited by 55
  20. 2025Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Modelscited by 54
  21. 2024FiLo: Zero-Shot Anomaly Detection by Fine-Grained Description and High-Quality Localizationcited by 51
  22. 2025Exploring Intrinsic Normal Prototypes within a Single Image for Universal Anomaly Detectioncited by 48
  23. 2023Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detectioncited by 47
  24. 2024MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detectioncited by 47
  25. 2023DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detectioncited by 46
  26. 2023PAD: A Dataset and Benchmark for Pose-agnostic Anomaly Detectioncited by 46
  27. 2023EasyNet: An Easy Network for 3D Industrial Anomaly Detectioncited by 43
  28. 2023TransFusion -- A Transparency-Based Diffusion Model for Anomaly Detectioncited by 43
  29. 2023Multimodal Industrial Anomaly Detection by Crossmodal Feature Mappingcited by 42
  30. 2024GLAD: Towards Better Reconstruction with Global and Local Adaptive Diffusion Models for Unsupervised Anomaly Detectioncited by 42
  31. 2023Diversity-Measurable Anomaly Detectioncited by 41
  32. 2023Myriad: Large Multimodal Model by Applying Vision Experts for Industrial Anomaly Detectioncited by 39
  33. 2023Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectorscited by 37
  34. 2023GenDet: Towards Good Generalizations for AI-Generated Image Detectioncited by 36
  35. 2023Anomaly Detection under Distribution Shiftcited by 35
  36. 2023DiAD: A Diffusion-based Framework for Multi-class Anomaly Detectioncited by 33
  37. 2023Focus the Discrepancy: Intra- and Inter-Correlation Learning for Image Anomaly Detectioncited by 32
  38. 2024Dual-Interrelated Diffusion Model for Few-Shot Anomaly Image Generationcited by 32

50 most cited of 421 All 421 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Dual-Modeling Decouple Distillation for Unsupervised Anomaly Detection
  2. 2023Knowledge Distillation for Anomaly Detection
  3. 2021Distilling Object Detectors via Decoupled Features
  4. 2024CNC: Cross-modal Normality Constraint for Unsupervised Multi-class Anomaly Detection
  5. 2021Beyond Classification: Knowledge Distillation using Multi-Object Impressions
  6. 2021Brittle Features May Help Anomaly Detection
  7. 2025Architectural Insights into Knowledge Distillation for Object Detection: A Comprehensive Review
  8. 2022PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficient