# MixedTeacher : Knowledge Distillation for fast inference textural anomaly detection

arXiv: https://arxiv.org/abs/2306.09859 · HTML: https://mlsanity.com/paper/2306.09859/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 16 Jun 2023.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 15 of 15)

- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows](https://mlsanity.com/paper/2111.07677/) (2021)
- [VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2104.10036/) (2021)
- [Same Same But DifferNet: Semi-Supervised Defect Detection with Normalizing Flows](https://mlsanity.com/paper/2008.12577/) (2020)
- [Student-Teacher Feature Pyramid Matching for Anomaly Detection](https://mlsanity.com/paper/2103.04257/) (2021)
- [CFA: Coupled-hypersphere-based Feature Adaptation for Target-Oriented Anomaly Localization](https://mlsanity.com/paper/2206.04325/) (2022)
- [Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection](https://mlsanity.com/paper/2110.02855/) (2021)
- [Omni-frequency Channel-selection Representations for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2203.00259/) (2022)
- [G2D: Generate to Detect Anomaly](https://mlsanity.com/paper/2006.11629/) (2020)
- [GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection](https://mlsanity.com/paper/1903.06661/) (2019)
- [Reconstruction Student with Attention for Student-Teacher Pyramid Matching](https://mlsanity.com/paper/2111.15376/) (2021)
- [AnoNet: Weakly Supervised Anomaly Detection in Textured Surfaces](https://mlsanity.com/paper/1911.10608/) (2019)
- [Unsupervised Industrial Anomaly Detection via Pattern Generative and Contrastive Networks](https://mlsanity.com/paper/2207.09792/) (2022)

## Built on by (most cited first, 2 of 2)

- [Distillation-based fabric anomaly detection](https://mlsanity.com/paper/2401.02287/) (2024)
- [CSE: Surface Anomaly Detection with Contrastively Selected Embedding](https://mlsanity.com/paper/2403.01859/) (2024)

## Abstract (arXiv)

For a very long time, unsupervised learning for anomaly detection has been at the heart of image processing research and a stepping stone for high performance industrial automation process. With the emergence of CNN, several methods have been proposed such as Autoencoders, GAN, deep feature extraction, etc. In this paper, we propose a new method based on the promising concept of knowledge distillation which consists of training a network (the student) on normal samples while considering the output of a larger pretrained network (the teacher). The main contributions of this paper are twofold: First, a reduced student architecture with optimal layer selection is proposed, then a new Student-Teacher architecture with network bias reduction combining two teachers is proposed in order to jointly enhance the performance of anomaly detection and its localization accuracy. The proposed texture anomaly detector has an outstanding capability to detect defects in any texture and a fast inference time compared to the SOTA methods.