# Multiple Teachers-Meticulous Student: A Domain Adaptive Meta-Knowledge Distillation Model for Medical Image Classification arXiv: https://arxiv.org/abs/2403.11226 ยท HTML: https://mlsanity.com/paper/2403.11226/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation. - Builds on 28 of its 54 references that are in the map. - First posted on arXiv 17 Mar 2024. ## Placed in - Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/) ## Builds on (most cited first, 28 of 28) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014) - [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020) - [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021) - [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019) - [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020) - [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018) - [Domain Adaptation for Medical Image Analysis: A Survey](https://mlsanity.com/paper/2102.09508/) (2021) - [Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks](https://mlsanity.com/paper/2004.05937/) (2020) - [A Survey of Deep Meta-Learning](https://mlsanity.com/paper/2010.03522/) (2020) - [Source-Free Unsupervised Domain Adaptation: A Survey](https://mlsanity.com/paper/2301.00265/) (2022) - [Unsupervised Multi-Target Domain Adaptation Through Knowledge Distillation](https://mlsanity.com/paper/2007.07077/) (2020) - [MetaFed: Federated Learning among Federations with Cyclic Knowledge Distillation for Personalized Healthcare](https://mlsanity.com/paper/2206.08516/) (2022) - [Multi-Source Unsupervised Domain Adaptation via Pseudo Target Domain](https://mlsanity.com/paper/2202.10725/) (2022) - [Bootstrapping Semi-supervised Medical Image Segmentation with Anatomical-aware Contrastive Distillation](https://mlsanity.com/paper/2206.02307/) (2022) - [Automatic CNN-based detection of cardiac MR motion artefacts using k-space data augmentation and curriculum learning](https://mlsanity.com/paper/1810.12185/) (2018) - [PA-Seg: Learning from Point Annotations for 3D Medical Image Segmentation using Contextual Regularization and Cross Knowledge Distillation](https://mlsanity.com/paper/2208.05669/) (2022) - [Knowledge Distillation Methods for Efficient Unsupervised Adaptation Across Multiple Domains](https://mlsanity.com/paper/2101.07308/) (2021) - [Categorical Relation-Preserving Contrastive Knowledge Distillation for Medical Image Classification](https://mlsanity.com/paper/2107.03225/) (2021) - [SSD-KD: A Self-supervised Diverse Knowledge Distillation Method for Lightweight Skin Lesion Classification Using Dermoscopic Images](https://mlsanity.com/paper/2203.11490/) (2022) - [FoPro-KD: Fourier Prompted Effective Knowledge Distillation for Long-Tailed Medical Image Recognition](https://mlsanity.com/paper/2305.17421/) (2023) - [Enhancing and Adapting in the Clinic: Source-free Unsupervised Domain Adaptation for Medical Image Enhancement](https://mlsanity.com/paper/2312.01338/) (2023) - [Explainable Knowledge Distillation for On-device Chest X-Ray Classification](https://mlsanity.com/paper/2305.06244/) (2023) - [Taxonomy Adaptive Cross-Domain Adaptation in Medical Imaging via Optimization Trajectory Distillation](https://mlsanity.com/paper/2307.14709/) (2023) - [A Generalised Deep Meta-Learning Model for Automated Quality Control of Cardiovascular Magnetic Resonance Images](https://mlsanity.com/paper/2303.13324/) (2023) - [Automatic Multi-Class Cardiovascular Magnetic Resonance Image Quality Assessment using Unsupervised Domain Adaptation in Spatial and Frequency Domains](https://mlsanity.com/paper/2112.06806/) (2021) - [Fully Automated Assessment of Cardiac Coverage in Cine Cardiovascular Magnetic Resonance Images using an Explainable Deep Visual Salient Region Detection Model](https://mlsanity.com/paper/2206.06844/) (2022) ## Built on by (most cited first, 1 of 1) - [GENRE-CMR: Generalizable Deep Learning for Diverse Multi-Domain Cardiac MRI Reconstruction](https://mlsanity.com/paper/2508.20600/) (2025) ## Abstract (arXiv) Background: Image classification can be considered one of the key pillars of medical image analysis. Deep learning (DL) faces challenges that prevent its practical applications despite the remarkable improvement in medical image classification. The data distribution differences can lead to a drop in the efficiency of DL, known as the domain shift problem. Besides, requiring bulk annotated data for model training, the large size of models, and the privacy-preserving of patients are other challenges of using DL in medical image classification. This study presents a strategy that can address the mentioned issues simultaneously. Method: The proposed domain adaptive model based on knowledge distillation can classify images by receiving limited annotated data of different distributions. The designed multiple teachers-meticulous student model trains a student network that tries to solve the challenges by receiving the parameters of several teacher networks. The proposed model was evaluated using six available datasets of different distributions by defining the respiratory motion artefact detection task. Results: The results of extensive experiments using several datasets show the superiority of the proposed model in addressing the domain shift problem and lack of access to bulk annotated data. Besides, the privacy preservation of patients by receiving only the teacher network parameters instead of the original data and consolidating the knowledge of several DL models into a model with almost similar performance are other advantages of the proposed model. Conclusions: The proposed model can pave the way for practical clinical applications of deep classification methods by achieving the mentioned objectives simultaneously.