# Unlocking the Potential of Reverse Distillation for Anomaly Detection

arXiv: https://arxiv.org/abs/2412.07579 · HTML: https://mlsanity.com/paper/2412.07579/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 10 Dec 2024.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 18 of 18)

- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detection](https://mlsanity.com/paper/2108.07610/) (2021)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings](https://mlsanity.com/paper/1911.02357/) (2019)
- [Multiresolution Knowledge Distillation for Anomaly Detection](https://mlsanity.com/paper/2011.11108/) (2020)
- [Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders](https://mlsanity.com/paper/1807.02011/) (2018)
- [VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2104.10036/) (2021)
- [DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection](https://mlsanity.com/paper/2211.11317/) (2022)
- [Student-Teacher Feature Pyramid Matching for Anomaly Detection](https://mlsanity.com/paper/2103.04257/) (2021)
- [RealNet: A Feature Selection Network with Realistic Synthetic Anomaly for Anomaly Detection](https://mlsanity.com/paper/2403.05897/) (2024)
- [PNI : Industrial Anomaly Detection using Position and Neighborhood Information](https://mlsanity.com/paper/2211.12634/) (2022)
- [ReContrast: Domain-Specific Anomaly Detection via Contrastive Reconstruction](https://mlsanity.com/paper/2306.02602/) (2023)
- [Target before Shooting: Accurate Anomaly Detection and Localization under One Millisecond via Cascade Patch Retrieval](https://mlsanity.com/paper/2308.06748/) (2023)
- [A Comprehensive Augmentation Framework for Anomaly Detection](https://mlsanity.com/paper/2308.15068/) (2023)

## Built on by (most cited first, 8 of 8)

- [A Survey on Foundation-Model-Based Industrial Defect Detection](https://mlsanity.com/paper/2502.19106/) (2025)
- [Isolating to Harness: Cross-Division Distillation for Fully Unsupervised Anomaly Detection](https://mlsanity.com/paper/2508.18007/) (2025)
- [HarmoniAD: Harmonizing Local Structures and Global Semantics for Anomaly Detection](https://mlsanity.com/paper/2601.00327/) (2026)
- [Res2CLIP: Few-Shot Generalist Anomaly Detection with Residual-to-Residual Alignment](https://mlsanity.com/paper/2605.16171/) (2026)
- [CFR-Net:Collaborative Feature Refinement Network for Medical Image Anomaly Detection](https://mlsanity.com/paper/2607.11509/) (2026)
- [ReMoE: Report-Guided Mixture-of-Experts for Multimodal OCT/OCTA Anomaly Detection](https://mlsanity.com/paper/2607.29039/) (2026)
- [See More, Detect Less? Taming Information Leakage in Multi-View Anomaly Detection](https://mlsanity.com/paper/2608.25168/) (2026)
- [GeoMAD: Geometry-Aware Multi-View Anomaly Detection via Deformable Fusion and Distributional Alignment](https://mlsanity.com/paper/2608.26724/) (2026)

## Abstract (arXiv)

Knowledge Distillation (KD) is a promising approach for unsupervised Anomaly Detection (AD). However, the student network's over-generalization often diminishes the crucial representation differences between teacher and student in anomalous regions, leading to detection failures. To addresses this problem, the widely accepted Reverse Distillation (RD) paradigm designs the asymmetry teacher and student, using an encoder as teacher and a decoder as student. Yet, the design of RD does not ensure that the teacher encoder effectively distinguishes between normal and abnormal features or that the student decoder generates anomaly-free features. Additionally, the absence of skip connections results in a loss of fine details during feature reconstruction. To address these issues, we propose RD with Expert, which introduces a novel Expert-Teacher-Student network for simultaneous distillation of both the teacher encoder and student decoder. The added expert network enhances the student's ability to generate normal features and optimizes the teacher's differentiation between normal and abnormal features, reducing missed detections. Additionally, Guided Information Injection is designed to filter and transfer features from teacher to student, improving detail reconstruction and minimizing false positives. Experiments on several benchmarks prove that our method outperforms existing unsupervised AD methods under RD paradigm, fully unlocking RD's potential.