mlsanityData to 14 Oct 2026

MixedTeacher : Knowledge Distillation for fast inference textural anomaly detection

arXiv 2306.09859 ↗First posted 16 Jun 2023cs.CVcited by 2

On the map: Knowledge distillation → Detection and dense prediction distillation

Abstract (arXiv)

For a very long time, unsupervised learning for anomaly detection has been at the heart of image processing research and a stepping stone for high performance industrial automation process. With the emergence of CNN, several methods have been proposed such as Autoencoders, GAN, deep feature extraction, etc. In this paper, we propose a new method based on the promising concept of knowledge distillation which consists of training a network (the student) on normal samples while considering the output of a larger pretrained network (the teacher). The main contributions of this paper are twofold: First, a reduced student architecture with optimal layer selection is proposed, then a new Student-Teacher architecture with network bias reduction combining two teachers is proposed in order to jointly enhance the performance of anomaly detection and its localization accuracy. The proposed texture anomaly detector has an outstanding capability to detect defects in any texture and a fast inference time compared to the SOTA methods.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 15 of its 20 references that are in the map. First posted on arXiv 16 Jun 2023.

Builds on

15 of 20 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Anomaly Detection via Reverse Distillation from One-Class Embedding26 Jan

Other references · 14

  1. 2021Towards Total Recall in Industrial Anomaly Detectioncited by 776
  2. 2021CutPaste: Self-Supervised Learning for Anomaly Detection and Localizationcited by 531
  3. 2021FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flowscited by 227
  4. 2021VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localizationcited by 223
  5. 2020Same Same But DifferNet: Semi-Supervised Defect Detection with Normalizing Flowscited by 212
  6. 2021Student-Teacher Feature Pyramid Matching for Anomaly Detectioncited by 146
  7. 2022CFA: Coupled-hypersphere-based Feature Adaptation for Target-Oriented Anomaly Localizationcited by 135
  8. 2021Fully Convolutional Cross-Scale-Flows for Image-based Defect Detectioncited by 117
  9. 2022Omni-frequency Channel-selection Representations for Unsupervised Anomaly Detectioncited by 80
  10. 2020G2D: Generate to Detect Anomalycited by 42
  11. 2019GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detectioncited by 36
  12. 2021Reconstruction Student with Attention for Student-Teacher Pyramid Matchingcited by 20
2 more
  1. 2019AnoNet: Weakly Supervised Anomaly Detection in Textured Surfacescited by 2
  2. 2022Unsupervised Industrial Anomaly Detection via Pattern Generative and Contrastive Networkscited by 2

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024Distillation-based fabric anomaly detectioncited by 2
  2. 2024CSE: Surface Anomaly Detection with Contrastively Selected Embeddingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Advancing Pre-trained Teacher: Towards Robust Feature Discrepancy for Anomaly Detection
  2. 2024Dual-Modeling Decouple Distillation for Unsupervised Anomaly Detection
  3. 2024Unlocking the Potential of Reverse Distillation for Anomaly Detection
  4. 2024Tuned Reverse Distillation: Enhancing Multimodal Industrial Anomaly Detection with Crossmodal Tuners
  5. 2024Dual-Student Knowledge Distillation Networks for Unsupervised Anomaly Detection
  6. 2023Contextual Affinity Distillation for Image Anomaly Detection
  7. 2025Generalist Multi-Class Anomaly Detection via Distillation to Two Heterogeneous Student Networks
  8. 2022Lightning Fast Video Anomaly Detection via Adversarial Knowledge Distillation