# Explainable Knowledge Distillation for On-device Chest X-Ray Classification

arXiv: https://arxiv.org/abs/2305.06244 · HTML: https://mlsanity.com/paper/2305.06244/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation.
- Builds on 21 of its 52 references that are in the map.
- First posted on arXiv 10 May 2023.

## Placed in

- Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/)

## Builds on (most cited first, 21 of 21)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI](https://mlsanity.com/paper/1907.07374/) (2019)
- [Deep Learning for Medical Image Analysis](https://mlsanity.com/paper/1708.08987/) (2017)
- [PadChest: A large chest x-ray image dataset with multi-label annotated reports](https://mlsanity.com/paper/1901.07441/) (2019)
- [A Survey on Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2008.10937/) (2020)
- [Visformer: The Vision-friendly Transformer](https://mlsanity.com/paper/2104.12533/) (2021)
- [Large-scale Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image Classification](https://mlsanity.com/paper/2012.03173/) (2020)
- [Deep Learning for Chest X-ray Analysis: A Survey](https://mlsanity.com/paper/2103.08700/) (2021)
- [Searching the Search Space of Vision Transformer](https://mlsanity.com/paper/2111.14725/) (2021)
- [Dynamic Fusion based Federated Learning for COVID-19 Detection](https://mlsanity.com/paper/2009.10401/) (2020)
- [Vision Transformer using Low-level Chest X-ray Feature Corpus for COVID-19 Diagnosis and Severity Quantification](https://mlsanity.com/paper/2104.07235/) (2021)
- [Variational Knowledge Distillation for Disease Classification in Chest X-Rays](https://mlsanity.com/paper/2103.10825/) (2021)
- [EEEA-Net: An Early Exit Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2108.06156/) (2021)

## Built on by (most cited first, 8 of 8)

- [Applications of Knowledge Distillation in Remote Sensing: A Survey](https://mlsanity.com/paper/2409.12111/) (2024)
- [A Novel Multi-Task Teacher-Student Architecture with Self-Supervised Pretraining for 48-Hour Vasoactive-Inotropic Trend Analysis in Sepsis Mortality Prediction](https://mlsanity.com/paper/2502.16834/) (2025)
- [Multiple Teachers-Meticulous Student: A Domain Adaptive Meta-Knowledge Distillation Model for Medical Image Classification](https://mlsanity.com/paper/2403.11226/) (2024)
- [Low-Resolution Chest X-ray Classification via Knowledge Distillation and Multi-task Learning](https://mlsanity.com/paper/2405.13370/) (2024)
- [SalNAS: Efficient Saliency-prediction Neural Architecture Search with self-knowledge distillation](https://mlsanity.com/paper/2407.20062/) (2024)
- [A Knowledge Distillation-Based Approach to Enhance Transparency of Classifier Models](https://mlsanity.com/paper/2502.15959/) (2025)
- [Weakly Supervised Tuberculosis Localization in Chest X-rays through Knowledge Distillation](https://mlsanity.com/paper/2512.11057/) (2025)
- [Exponential Pixelating Integral transform with dual fractal features for enhanced chest X-ray abnormality detection](https://mlsanity.com/paper/2609.10988/) (2026)

## Abstract (arXiv)

Automated multi-label chest X-rays (CXR) image classification has achieved substantial progress in clinical diagnosis via utilizing sophisticated deep learning approaches. However, most deep models have high computational demands, which makes them less feasible for compact devices with low computational requirements. To overcome this problem, we propose a knowledge distillation (KD) strategy to create the compact deep learning model for the real-time multi-label CXR image classification. We study different alternatives of CNNs and Transforms as the teacher to distill the knowledge to a smaller student. Then, we employed explainable artificial intelligence (XAI) to provide the visual explanation for the model decision improved by the KD. Our results on three benchmark CXR datasets show that our KD strategy provides the improved performance on the compact student model, thus being the feasible choice for many limited hardware platforms. For instance, when using DenseNet161 as the teacher network, EEEA-Net-C2 achieved an AUC of 83.7%, 87.1%, and 88.7% on the ChestX-ray14, CheXpert, and PadChest datasets, respectively, with fewer parameters of 4.7 million and computational cost of 0.3 billion FLOPS.