# A Knowledge Distillation-Based Approach to Enhance Transparency of Classifier Models

arXiv: https://arxiv.org/abs/2502.15959 · HTML: https://mlsanity.com/paper/2502.15959/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation.
- Builds on 9 of its 28 references that are in the map.
- First posted on arXiv 21 Feb 2025.

## Placed in

- Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/)

## Builds on (most cited first, 9 of 9)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [AI can evolve without labels: self-evolving vision transformer for chest X-ray diagnosis through knowledge distillation](https://mlsanity.com/paper/2202.06431/) (2022)
- [SSD-KD: A Self-supervised Diverse Knowledge Distillation Method for Lightweight Skin Lesion Classification Using Dermoscopic Images](https://mlsanity.com/paper/2203.11490/) (2022)
- [Explainable Knowledge Distillation for On-device Chest X-Ray Classification](https://mlsanity.com/paper/2305.06244/) (2023)
- [Segmentation with mixed supervision: Confidence maximization helps knowledge distillation](https://mlsanity.com/paper/2109.10902/) (2021)

## Built on by (most cited first, 1 of 1)

- [Towards a Transparent and Interpretable AI Model for Medical Image Classifications](https://mlsanity.com/paper/2509.16685/) (2025)

## Abstract (arXiv)

With the rapid development of artificial intelligence (AI), especially in the medical field, the need for its explainability has grown. In medical image analysis, a high degree of transparency and model interpretability can help clinicians better understand and trust the decision-making process of AI models. In this study, we propose a Knowledge Distillation (KD)-based approach that aims to enhance the transparency of the AI model in medical image analysis. The initial step is to use traditional CNN to obtain a teacher model and then use KD to simplify the CNN architecture, retain most of the features of the data set, and reduce the number of network layers. It also uses the feature map of the student model to perform hierarchical analysis to identify key features and decision-making processes. This leads to intuitive visual explanations. We selected three public medical data sets (brain tumor, eye disease, and Alzheimer's disease) to test our method. It shows that even when the number of layers is reduced, our model provides a remarkable result in the test set and reduces the time required for the interpretability analysis.