mlsanityData to 14 Oct 2026

Distillation for medical imaging

191 papers0 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: segmentation, medical, knowledge, distillation, image

Key facts. Distillation for medical imaging: 191 papers on arXiv since 2021 (data through 14 Oct 2026). 14 of its 191 mapped papers (7%) cite “Efficient Medical Image Segmentation Based on Knowledge Distillation” (2021, arXiv 2108.09987). 3 new papers 25 Sep–14 Oct.

What’s new

25 Sep – 14 Oct 2026

3 new since 25 Sep

  1. Efficient Multi-Granularity Knowledge Transfer for Radiology Report Generation
  2. Multimodal Knowledge Distillation for Gastric Adenocarcinoma Classification from Whole-Slide Images
  3. From UNI2-h to ConvNeXt-T: Lightweight Nuclei Instance Segmentation via Knowledge Distillation

Foundations

Most cited by papers here

  1. Efficient Medical Image Segmentation Based on Knowledge Distillation2021cited by 14 here
  2. Knowledge distillation from multi-modal to mono-modal segmentation networks2021cited by 11 here
  3. DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification2022cited by 6 here
  4. Categorical Relation-Preserving Contrastive Knowledge Distillation for Medical Image Classification2021cited by 4 here
  5. Alternate Diverse Teaching for Semi-supervised Medical Image Segmentation2023cited by 4 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2021Distilling Knowledge via Knowledge Reviewcited by 4 here
  2. 2022Decoupled Knowledge Distillationcited by 5 here
  3. 2023CrossKD: Cross-Head Knowledge Distillation for Object Detectioncited by 3 here
  4. 2025ACAM-KD: Adaptive and Cooperative Attention Masking for Knowledge Distillationcited by 4 here
20 more it builds on, not in this chain
  1. 2021Show, Attend and Distill:Knowledge Distillation via Attention-based Feature Matchingcited by 4 here
  2. 2021Variational Knowledge Distillation for Disease Classification in Chest X-Rayscited by 3 here
  3. 2021Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillationcited by 4 here
  4. 2021Knowledge distillation from multi-modal to mono-modal segmentation networkscited by 11 here
  5. 2021Categorical Relation-Preserving Contrastive Knowledge Distillation for Medical Image Classificationcited by 4 here
  6. 2021Efficient Medical Image Segmentation Based on Knowledge Distillationcited by 14 here
  7. 2021Cross-modal Knowledge Distillation for Vision-to-Sensor Action Recognitioncited by 3 here
  8. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 3 here
  9. 2022SSD-KD: A Self-supervised Diverse Knowledge Distillation Method for Lightweight Skin Lesion Classification Using Dermoscopic Imagescited by 3 here
  10. 2022DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classificationcited by 6 here
  11. 2022Knowledge Distillation with the Reused Teacher Classifiercited by 4 here
  12. 2022Cross-Image Relational Knowledge Distillation for Semantic Segmentationcited by 6 here
  13. 2022Knowledge Distillation from A Stronger Teachercited by 7 here
  14. 2022MetaFed: Federated Learning among Federations with Cyclic Knowledge Distillation for Personalized Healthcarecited by 3 here
  15. 2023Gradient-Guided Knowledge Distillation for Object Detectorscited by 3 here
  16. 2023Understanding the Role of the Projector in Knowledge Distillationcited by 3 here
  17. 2023Explainable Knowledge Distillation for On-device Chest X-Ray Classificationcited by 3 here
  18. 2023Learnable Cross-modal Knowledge Distillation for Multi-modal Learning with Missing Modalitycited by 3 here
  19. 2023Switching Temporary Teachers for Semi-Supervised Semantic Segmentationcited by 4 here
  20. 2023Alternate Diverse Teaching for Semi-supervised Medical Image Segmentationcited by 4 here

The 24 most cited of 151.

Browse

30 of 191, ranked 6–35

  1. 6ReCo-KD: Region- and Context-Aware Knowledge Distillation for Efficient 3D Medical Image Segmentation20263
  2. 7Explainable Knowledge Distillation for On-device Chest X-Ray Classification20233
  3. 8SSD-KD: A Self-supervised Diverse Knowledge Distillation Method for Lightweight Skin Lesion Classification Using Dermoscopic Images20223
  4. 9Variational Knowledge Distillation for Disease Classification in Chest X-Rays20213
  5. 10Cross-modal Knowledge Distillation for Vision-to-Sensor Action Recognition20213
  6. 11Prototype Knowledge Distillation for Medical Segmentation with Missing Modality20232
  7. 12FoPro-KD: Fourier Prompted Effective Knowledge Distillation for Long-Tailed Medical Image Recognition20232
  8. 13Role of Data Augmentation Strategies in Knowledge Distillation for Wearable Sensor Data20222
  9. 14AI can evolve without labels: self-evolving vision transformer for chest X-ray diagnosis through knowledge distillation20222
  10. 15CoCo DistillNet: a Cross-layer Correlation Distillation Network for Pathological Gastric Cancer Segmentation20212
All 30
  1. 16Uncertainty-Aware Multi-Expert Knowledge Distillation for Imbalanced Disease Grading20251
  2. 17DCSNet: A Lightweight Knowledge Distillation-Based Model with Explainable AI for Lung Cancer Diagnosis from Histopathological Images20251
  3. 18Cross-Architecture Knowledge Distillation (KD) for Retinal Fundus Image Anomaly Detection on NVIDIA Jetson Nano20251
  4. 19Bridging the Gap in Missing Modalities: Leveraging Knowledge Distillation and Style Matching for Brain Tumor Segmentation20251
  5. 20Fine-tuning a Multiple Instance Learning Feature Extractor with Masked Context Modelling and Knowledge Distillation20241
  6. 21Histo-Genomic Knowledge Distillation For Cancer Prognosis From Histopathology Whole Slide Images20241
  7. 22SlideGCD: Slide-based Graph Collaborative Training with Knowledge Distillation for Whole Slide Image Classification20241
  8. 23Shape-intensity knowledge distillation for robust medical image segmentation20241
  9. 24Hybrid Attention Model Using Feature Decomposition and Knowledge Distillation for Glucose Forecasting20241
  10. 25Unsupervised Deep Digital Staining For Microscopic Cell Images Via Knowledge Distillation20231
  11. 26Whole-slide-imaging Cancer Metastases Detection and Localization with Limited Tumorous Data20231
  12. 27Vision Transformers for Small Histological Datasets Learned through Knowledge Distillation20231
  13. 28Self-Knowledge Distillation for Surgical Phase Recognition20231
  14. 29Enhancing and Adapting in the Clinic: Source-free Unsupervised Domain Adaptation for Medical Image Enhancement20231
  15. 30Self-distillation Augmented Masked Autoencoders for Histopathological Image Classification20221
  16. 31Leukocyte Classification using Multimodal Architecture Enhanced by Knowledge Distillation20221
  17. 32TASKED: Transformer-based Adversarial learning for human activity recognition using wearable sensors via Self-KnowledgE Distillation20221
  18. 33Bi-directional Weakly Supervised Knowledge Distillation for Whole Slide Image Classification20221
  19. 34Knowledge Distillation approach towards Melanoma Detection20221
  20. 35Segmentation with mixed supervision: Confidence maximization helps knowledge distillation20211