# Uncertainty-Aware Multi-Expert Knowledge Distillation for Imbalanced Disease Grading

arXiv: https://arxiv.org/abs/2505.00592 · HTML: https://mlsanity.com/paper/2505.00592/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation.
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 1 May 2025.

## Placed in

- Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/)

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [Decoupled Knowledge Distillation](https://mlsanity.com/paper/2203.08679/) (2022)
- [One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillation](https://mlsanity.com/paper/2310.19444/) (2023)
- [Going Deeper through the Gleason Scoring Scale: An Automatic end-to-end System for Histology Prostate Grading and Cribriform Pattern Detection](https://mlsanity.com/paper/2105.10490/) (2021)
- [BayeSeg: Bayesian Modeling for Medical Image Segmentation with Interpretable Generalizability](https://mlsanity.com/paper/2303.01710/) (2023)
- [Ord2Seq: Regarding Ordinal Regression as Label Sequence Prediction](https://mlsanity.com/paper/2307.09004/) (2023)
- [Robust Image Ordinal Regression with Controllable Image Generation](https://mlsanity.com/paper/2305.04213/) (2023)
- [Contrastive Knowledge Amalgamation for Unsupervised Image Classification](https://mlsanity.com/paper/2307.14781/) (2023)
- [KA2ER: Knowledge Adaptive Amalgamation of ExpeRts for Medical Images Segmentation](https://mlsanity.com/paper/2410.21085/) (2024)

## Built on by (most cited first, 1 of 1)

- [Two-Stage Cross-Domain Cervical Abnormality Screening with Cytopathological Image Synthesis and Knowledge Distillation](https://mlsanity.com/paper/2606.27678/) (2026)

## Abstract (arXiv)

Automatic disease image grading is a significant application of artificial intelligence for healthcare, enabling faster and more accurate patient assessments. However, domain shifts, which are exacerbated by data imbalance, introduce bias into the model, posing deployment difficulties in clinical applications. To address the problem, we propose a novel Uncertainty-aware Multi-experts Knowledge Distillation (UMKD) framework to transfer knowledge from multiple expert models to a single student model. Specifically, to extract discriminative features, UMKD decouples task-agnostic and task-specific features with shallow and compact feature alignment in the feature space. At the output space, an uncertainty-aware decoupled distillation (UDD) mechanism dynamically adjusts knowledge transfer weights based on expert model uncertainties, ensuring robust and reliable distillation. Additionally, UMKD also tackles the problems of model architecture heterogeneity and distribution discrepancies between source and target domains, which are inadequately tackled by previous KD approaches. Extensive experiments on histology prostate grading (textitSICAPv2) and fundus image grading (textitAPTOS) demonstrate that UMKD achieves a new state-of-the-art in both source-imbalanced and target-imbalanced scenarios, offering a robust and practical solution for real-world disease image grading.