# Scale-Aware Contrastive Reverse Distillation for Unsupervised Medical Anomaly Detection

arXiv: https://arxiv.org/abs/2503.13828 · HTML: https://mlsanity.com/paper/2503.13828/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 36 of its 47 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings](https://mlsanity.com/paper/1911.02357/) (2019)
- [CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows](https://mlsanity.com/paper/2107.12571/) (2021)
- [Multiresolution Knowledge Distillation for Anomaly Detection](https://mlsanity.com/paper/2011.11108/) (2020)
- [Learning Memory-guided Normality for Anomaly Detection](https://mlsanity.com/paper/2003.13228/) (2020)
- [A Unified Model for Multi-class Anomaly Detection](https://mlsanity.com/paper/2206.03687/) (2022)
- [SimpleNet: A Simple Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2303.15140/) (2023)
- [FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows](https://mlsanity.com/paper/2111.07677/) (2021)
- [Same Same But DifferNet: Semi-Supervised Defect Detection with Normalizing Flows](https://mlsanity.com/paper/2008.12577/) (2020)
- [EfficientAD: Accurate Visual Anomaly Detection at Millisecond-Level Latencies](https://mlsanity.com/paper/2303.14535/) (2023)
- [Student-Teacher Feature Pyramid Matching for Anomaly Detection](https://mlsanity.com/paper/2103.04257/) (2021)
- [Learning and Evaluating Representations for Deep One-class Classification](https://mlsanity.com/paper/2011.02578/) (2020)
- [Natural Synthetic Anomalies for Self-Supervised Anomaly Detection and Localization](https://mlsanity.com/paper/2109.15222/) (2021)
- [Context-encoding Variational Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1812.05941/) (2018)
- [ReContrast: Domain-Specific Anomaly Detection via Contrastive Reconstruction](https://mlsanity.com/paper/2306.02602/) (2023)
- [MADGAN: unsupervised Medical Anomaly Detection GAN using multiple adjacent brain MRI slice reconstruction](https://mlsanity.com/paper/2007.13559/) (2020)
- [Anomaly detection through latent space restoration using vector-quantized variational autoencoders](https://mlsanity.com/paper/2012.06765/) (2020)
- [Detecting Outliers with Poisson Image Interpolation](https://mlsanity.com/paper/2107.02622/) (2021)
- [SQUID: Deep Feature In-Painting for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2111.13495/) (2021)
- [Unsupervised Continual Anomaly Detection with Contrastively-learned Prompt](https://mlsanity.com/paper/2401.01010/) (2024)
- [Anomaly Detection in Medical Imaging with Deep Perceptual Autoencoders](https://mlsanity.com/paper/2006.13265/) (2020)
- [MedIAnomaly: A comparative study of anomaly detection in medical images](https://mlsanity.com/paper/2404.04518/) (2024)

## Built on by (most cited first, 6 of 6)

- [Dual Distillation for Few-Shot Anomaly Detection](https://mlsanity.com/paper/2603.01713/) (2026)
- [Transcending the Annotation Bottleneck: AI-Powered Discovery in Biology and Medicine](https://mlsanity.com/paper/2602.20100/) (2026)
- [Evidential Reasoning Advances Interpretable Real-World Disease Screening](https://mlsanity.com/paper/2605.15171/) (2026)
- [On Revisiting Entropy for Identifying Mislabeled Images](https://mlsanity.com/paper/2605.31090/) (2026)
- [ArcAD: Anomaly-Rectified Calibration for Cold-Start Supervised Anomaly Detection](https://mlsanity.com/paper/2607.02252/) (2026)
- [MIRTO: a registration-gated, multiverse-tested evaluation protocol for unsupervised anomaly segmentation in brain MRI](https://mlsanity.com/paper/2610.02136/) (2026)

## Abstract (arXiv)

Unsupervised anomaly detection using deep learning has garnered significant research attention due to its broad applicability, particularly in medical imaging where labeled anomalous data are scarce. While earlier approaches leverage generative models like autoencoders and generative adversarial networks (GANs), they often fall short due to overgeneralization. Recent methods explore various strategies, including memory banks, normalizing flows, self-supervised learning, and knowledge distillation, to enhance discrimination. Among these, knowledge distillation, particularly reverse distillation, has shown promise. Following this paradigm, we propose a novel scale-aware contrastive reverse distillation model that addresses two key limitations of existing reverse distillation methods: insufficient feature discriminability and inability to handle anomaly scale variations. Specifically, we introduce a contrastive student-teacher learning approach to derive more discriminative representations by generating and exploring out-of-normal distributions. Further, we design a scale adaptation mechanism to softly weight contrastive distillation losses at different scales to account for the scale variation issue. Extensive experiments on benchmark datasets demonstrate state-of-the-art performance, validating the efficacy of the proposed method. Code is available at https://github.com/MedAITech/SCRD4AD.