# Knowledge Distillation for Anomaly Detection

arXiv: https://arxiv.org/abs/2310.06047 · HTML: https://mlsanity.com/paper/2310.06047/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 8 of its 16 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 8 of 8)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Improved Knowledge Distillation via Teacher Assistant](https://mlsanity.com/paper/1902.03393/) (2019)
- [Multiresolution Knowledge Distillation for Anomaly Detection](https://mlsanity.com/paper/2011.11108/) (2020)
- [Does Knowledge Distillation Really Work?](https://mlsanity.com/paper/2106.05945/) (2021)
- [Transferring Inductive Biases through Knowledge Distillation](https://mlsanity.com/paper/2006.00555/) (2020)
- [Deep Model Compression: Distilling Knowledge from Noisy Teachers](https://mlsanity.com/paper/1610.09650/) (2016)
- [Detector monitoring with artificial neural networks at the CMS experiment at the CERN Large Hadron Collider](https://mlsanity.com/paper/1808.00911/) (2018)
- [Knowledge distillation using unlabeled mismatched images](https://mlsanity.com/paper/1703.07131/) (2017)

## Built on by (most cited first, 2 of 2)

- [Distillation-based fabric anomaly detection](https://mlsanity.com/paper/2401.02287/) (2024)
- [DevPrompt: Deviation-Based Prompt Learning for One-Normal ShotImage Anomaly Detection](https://mlsanity.com/paper/2601.15453/) (2026)

## Abstract (arXiv)

Unsupervised deep learning techniques are widely used to identify anomalous behaviour. The performance of such methods is a product of the amount of training data and the model size. However, the size is often a limiting factor for the deployment on resource-constrained devices. We present a novel procedure based on knowledge distillation for compressing an unsupervised anomaly detection model into a supervised deployable one and we suggest a set of techniques to improve the detection sensitivity. Compressed models perform comparably to their larger counterparts while significantly reducing the size and memory footprint.