mlsanityData to 7 Oct 2026

Multiresolution Knowledge Distillation for Anomaly Detection

arXiv 2011.11108 ↗First posted 22 Nov 2020cs.CVcited by 285

Not on the map. Why?

Abstract (arXiv)

Unsupervised representation learning has proved to be a critical component of anomaly detection/localization in images. The challenges to learn such a representation are two-fold. Firstly, the sample size is not often large enough to learn a rich generalizable representation through conventional techniques. Secondly, while only normal samples are available at training, the learned features should be discriminative of normal and anomalous samples. Here, we propose to use the "distillation" of features at various layers of an expert network, pre-trained on ImageNet, into a simpler cloner network to tackle both issues. We detect and localize anomalies using the discrepancy between the expert and cloner networks' intermediate activation values given the input data. We show that considering multiple intermediate hints in distillation leads to better exploiting the expert's knowledge and more distinctive discrepancy compared to solely utilizing the last layer activation values. Notably, previous methods either fail in precise anomaly localization or need expensive region-based training. In contrast, with no need for any special or intensive training procedure, we incorporate interpretability algorithms in our novel framework for the localization of anomalous regions. Despite the striking contrast between some test datasets and ImageNet, we achieve competitive or significantly superior results compared to the SOTA methods on MNIST, F-MNIST, CIFAR-10, MVTecAD, Retinal-OCT, and two Medical datasets on both anomaly detection and localization.

Key facts. Cited by 285 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 40; 2023: 58; 2024: 82; 2025: 49; 2026: 38.

285 papers.

Sort:

By area:

  1. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  2. 2021Towards Total Recall in Industrial Anomaly Detectioncited by 772
  3. 2022Anomaly Detection via Reverse Distillation from One-Class Embeddingcited by 419
  4. 2021CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flowscited by 320
  5. 2023WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentationcited by 311
  6. 2023AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detectioncited by 270
  7. 2022A Unified Model for Multi-class Anomaly Detectioncited by 267
  8. 2023SimpleNet: A Simple Network for Image Anomaly Detection and Localizationcited by 256
  9. 2021The MVTec 3D-AD Dataset for Unsupervised 3D Anomaly Detection and Localizationcited by 174
  10. 2023EfficientAD: Accurate Visual Anomaly Detection at Millisecond-Level Latenciescited by 157
  11. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  12. 2022DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detectioncited by 153
38 more
  1. 2023Deep Industrial Image Anomaly Detection: A Surveycited by 147
  2. 2022Registration based Few-Shot Anomaly Detectioncited by 133
  3. 2023APRIL-GAN: A Zero-/Few-Shot Anomaly Classification and Segmentation Method for CVPR 2023 VAND Workshop Challenge Tracks 1&2: 1st Place on Zero-shot AD and 4th Place on Few-shot ADcited by 128
  4. 2021Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detectioncited by 119
  5. 2022Catching Both Gray and Black Swans: Open-set Supervised Anomaly Detectioncited by 104
  6. 2024PromptAD: Learning Prompts with only Normal Samples for Few-Shot Anomaly Detectioncited by 97
  7. 2024Dinomaly: The Less Is More Philosophy in Multi-Class Unsupervised Anomaly Detectioncited by 92
  8. 2024Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Promptscited by 88
  9. 2023BMAD: Benchmarks for Medical Anomaly Detectioncited by 81
  10. 2024A Unified Anomaly Synthesis Strategy with Gradient Ascent for Industrial Anomaly Detection and Localizationcited by 74
  11. 2022Explicit Boundary Guided Semi-Push-Pull Contrastive Learning for Supervised Anomaly Detectioncited by 72
  12. 2021Explainable Deep Few-shot Anomaly Detection with Deviation Networkscited by 69
  13. 2024Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Imagescited by 69
  14. 2021Deep One-Class Classification via Interpolated Gaussian Descriptorcited by 67
  15. 2021Deep Graph-level Anomaly Detection by Glocal Knowledge Distillationcited by 67
  16. 2022Deep Learning for Unsupervised Anomaly Localization in Industrial Images: A Surveycited by 67
  17. 2022Anomaly Detection in 3D Point Clouds using Deep Geometric Descriptorscited by 66
  18. 2023Hierarchical Vector Quantized Transformer for Multi-class Unsupervised Anomaly Detectioncited by 66
  19. 2023IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturingcited by 64
  20. 2023ReContrast: Domain-Specific Anomaly Detection via Contrastive Reconstructioncited by 62
  21. 2023Segment Any Anomaly without Training via Hybrid Prompt Regularizationcited by 59
  22. 2022Prototypical Residual Networks for Anomaly Detection and Localizationcited by 56
  23. 2023Collaborative Discrepancy Optimization for Reliable Image Anomaly Localizationcited by 56
  24. 2024SoftPatch: Unsupervised Anomaly Detection with Noisy Datacited by 55
  25. 2023Mask, Stitch, and Re-Sample: Enhancing Robustness and Generalizability in Anomaly Detection through Automatic Diffusion Modelscited by 48
  26. 2023DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detectioncited by 46
  27. 2022Self-Supervised Masking for Unsupervised Anomaly Detection and Localizationcited by 43
  28. 2023Multimodal Industrial Anomaly Detection by Crossmodal Feature Mappingcited by 41
  29. 2023Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectorscited by 37
  30. 2023GenDet: Towards Good Generalizations for AI-Generated Image Detectioncited by 36
  31. 2023Anomaly Detection under Distribution Shiftcited by 35
  32. 2024Do LLMs Understand Visual Anomalies? Uncovering LLM's Capabilities in Zero-shot Anomaly Detectioncited by 35
  33. 2021SQUID: Deep Feature In-Painting for Unsupervised Anomaly Detectioncited by 34
  34. 2022Self-Supervised Masked Convolutional Transformer Block for Anomaly Detectioncited by 34
  35. 2023DiAD: A Diffusion-based Framework for Multi-class Anomaly Detectioncited by 33
  36. 2023Focus the Discrepancy: Intra- and Inter-Correlation Learning for Image Anomaly Detectioncited by 32
  37. 2022ADTR: Anomaly Detection Transformer with Feature Reconstructioncited by 31
  38. 2024Unsupervised Continual Anomaly Detection with Contrastively-learned Promptcited by 31

50 most cited of 285 All 285 (JSON)