# Dual Distillation for Few-Shot Anomaly Detection

arXiv: https://arxiv.org/abs/2603.01713 · HTML: https://mlsanity.com/paper/2603.01713/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 52 references that are in the map.
- First posted on arXiv 2 Mar 2026.

## Builds on (most cited first, 30 of 39)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows](https://mlsanity.com/paper/2107.12571/) (2021)
- [WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation](https://mlsanity.com/paper/2303.14814/) (2023)
- [Multiresolution Knowledge Distillation for Anomaly Detection](https://mlsanity.com/paper/2011.11108/) (2020)
- [Learning Memory-guided Normality for Anomaly Detection](https://mlsanity.com/paper/2003.13228/) (2020)
- [SimpleNet: A Simple Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2303.15140/) (2023)
- [FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows](https://mlsanity.com/paper/2111.07677/) (2021)
- [Same Same But DifferNet: Semi-Supervised Defect Detection with Normalizing Flows](https://mlsanity.com/paper/2008.12577/) (2020)
- [AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models](https://mlsanity.com/paper/2308.15366/) (2023)
- [EfficientAD: Accurate Visual Anomaly Detection at Millisecond-Level Latencies](https://mlsanity.com/paper/2303.14535/) (2023)
- [Learning and Evaluating Representations for Deep One-class Classification](https://mlsanity.com/paper/2011.02578/) (2020)
- [Registration based Few-Shot Anomaly Detection](https://mlsanity.com/paper/2207.07361/) (2022)
- [Deep Learning for Medical Anomaly Detection -- A Survey](https://mlsanity.com/paper/2012.02364/) (2020)
- [Natural Synthetic Anomalies for Self-Supervised Anomaly Detection and Localization](https://mlsanity.com/paper/2109.15222/) (2021)
- [Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts](https://mlsanity.com/paper/2403.06495/) (2024)
- [Attention Based Glaucoma Detection: A Large-scale Database and CNN Model](https://mlsanity.com/paper/1903.10831/) (2019)
- [Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images](https://mlsanity.com/paper/2403.12570/) (2024)
- [ReContrast: Domain-Specific Anomaly Detection via Contrastive Reconstruction](https://mlsanity.com/paper/2306.02602/) (2023)
- [A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection](https://mlsanity.com/paper/2104.14535/) (2021)
- [Deep Weakly-supervised Anomaly Detection](https://mlsanity.com/paper/1910.13601/) (2019)
- [Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series](https://mlsanity.com/paper/1809.04758/) (2018)
- [Exploring Intrinsic Normal Prototypes within a Single Image for Universal Anomaly Detection](https://mlsanity.com/paper/2503.02424/) (2025)
- [Anomaly detection through latent space restoration using vector-quantized variational autoencoders](https://mlsanity.com/paper/2012.06765/) (2020)

## Built on by (most cited first, 3 of 3)

- [Semantic Iterative Reconstruction: One-Shot Universal Anomaly Detection](https://mlsanity.com/paper/2603.23766/) (2026)
- [Beyond Normal References: Discriminative Few-Shot Anomaly Detection](https://mlsanity.com/paper/2605.23231/) (2026)
- [On Revisiting Entropy for Identifying Mislabeled Images](https://mlsanity.com/paper/2605.31090/) (2026)

## Abstract (arXiv)

Anomaly detection is a critical task in computer vision with profound implications for medical imaging, where identifying pathologies early can directly impact patient outcomes. While recent unsupervised anomaly detection approaches show promise, they require substantial normal training data and struggle to generalize across anatomical contexts. We introduce D24FAD, a novel dual distillation framework for few-shot anomaly detection that identifies anomalies in previously unseen tasks using only a small number of normal reference images. Our approach leverages a pre-trained encoder as a teacher network to extract multi-scale features from both support and query images, while a student decoder learns to distill knowledge from the teacher on query images and self-distill on support images. We further propose a learn-to-weight mechanism that dynamically assesses the reference value of each support image conditioned on the query, optimizing anomaly detection performance. To evaluate our method, we curate a comprehensive benchmark dataset comprising 13,084 images across four organs, four imaging modalities, and five disease categories. Extensive experiments demonstrate that D24FAD significantly outperforms existing approaches, establishing a new state-of-the-art in few-shot medical anomaly detection. Code is available at https://github.com/ttttqz/D24FAD.