# MED-TEX: Transferring and Explaining Knowledge with Less Data from Pretrained Medical Imaging Models

arXiv: https://arxiv.org/abs/2008.02593 · HTML: https://mlsanity.com/paper/2008.02593/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2020.

## Built on by (most cited first, 2 of 2)

- [Light-weight Deformable Registration using Adversarial Learning with Distilling Knowledge](https://mlsanity.com/paper/2110.01293/) (2021)
- [VALD-MD: Visual Attribution via Latent Diffusion for Medical Diagnostics](https://mlsanity.com/paper/2401.01414/) (2024)

## Abstract (arXiv)

Deep learning methods usually require a large amount of training data and lack interpretability. In this paper, we propose a novel knowledge distillation and model interpretation framework for medical image classification that jointly solves the above two issues. Specifically, to address the data-hungry issue, a small student model is learned with less data by distilling knowledge from a cumbersome pretrained teacher model. To interpret the teacher model and assist the learning of the student, an explainer module is introduced to highlight the regions of an input that are important for the predictions of the teacher model. Furthermore, the joint framework is trained by a principled way derived from the information-theoretic perspective. Our framework outperforms on the knowledge distillation and model interpretation tasks compared to state-of-the-art methods on a fundus dataset.