# Contrastive Knowledge Distillation for Anomaly Detection in Multi-Illumination/Focus Display Images

arXiv: https://arxiv.org/abs/2609.05520 · HTML: https://mlsanity.com/paper/2609.05520/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 23 of its 27 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances](https://mlsanity.com/paper/2007.08176/) (2020)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [Multiresolution Knowledge Distillation for Anomaly Detection](https://mlsanity.com/paper/2011.11108/) (2020)
- [VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2104.10036/) (2021)
- [PANDA: Adapting Pretrained Features for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2010.05903/) (2020)
- [OpenGAN: Open-Set Recognition via Open Data Generation](https://mlsanity.com/paper/2104.02939/) (2021)
- [CFA: Coupled-hypersphere-based Feature Adaptation for Target-Oriented Anomaly Localization](https://mlsanity.com/paper/2206.04325/) (2022)
- [Learning and Evaluating Representations for Deep One-class Classification](https://mlsanity.com/paper/2011.02578/) (2020)
- [Explainable Deep One-Class Classification](https://mlsanity.com/paper/2007.01760/) (2020)
- [Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection](https://mlsanity.com/paper/2111.09099/) (2021)
- [Asymmetric Student-Teacher Networks for Industrial Anomaly Detection](https://mlsanity.com/paper/2210.07829/) (2022)
- [Deep Nearest Neighbor Anomaly Detection](https://mlsanity.com/paper/2002.10445/) (2020)
- [Catching Both Gray and Black Swans: Open-set Supervised Anomaly Detection](https://mlsanity.com/paper/2203.14506/) (2022)
- [Mean-Shifted Contrastive Loss for Anomaly Detection](https://mlsanity.com/paper/2106.03844/) (2021)
- [Omni-frequency Channel-selection Representations for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2203.00259/) (2022)
- [MemSeg: A semi-supervised method for image surface defect detection using differences and commonalities](https://mlsanity.com/paper/2205.00908/) (2022)
- [Driver Anomaly Detection: A Dataset and Contrastive Learning Approach](https://mlsanity.com/paper/2009.14660/) (2020)
- [Transformaly -- Two (Feature Spaces) Are Better Than One](https://mlsanity.com/paper/2112.04185/) (2021)
- [Deep Unsupervised Image Anomaly Detection: An Information Theoretic Framework](https://mlsanity.com/paper/2012.04837/) (2020)
- [Fake It Till You Make It: Towards Accurate Near-Distribution Novelty Detection](https://mlsanity.com/paper/2205.14297/) (2022)

## Abstract (arXiv)

In this paper, we tackle automatic anomaly detection in multi-illumination and multi-focus display images. The minute defects on the display surface are hard to spot out in RGB images and by a model trained with only normal data. To address this, we propose a novel contrastive learning scheme for knowledge distillation-based anomaly detection. In our framework, Multiresolution Knowledge Distillation (MKD) is adopted as a baseline, which operates by measuring feature similarities between the teacher and student networks. Based on MKD, we propose a novel contrastive learning method, namely Multiresolution Contrastive Distillation (MCD), which does not require positive/negative pairs with an anchor but operates by pulling/pushing the distance between the teacher and student features. Furthermore, we propose the blending module that transforms and aggregate multi-channel information to the three-channel input layer of MCD. Our proposed method significantly outperforms competitive state-of-the-art methods in both AUROC and accuracy metrics on the collected Multi-illumination and Multi-focus display image dataset for Anomaly Detection (MMdAD).