mlsanityData to 7 Oct 2026

Knowledge distillation from multi-modal to mono-modal segmentation networks

arXiv 2106.09564 ↗First posted 17 Jun 2021cs.CV cs.AI stat.MLcited by 68

Not on the map. Why?

Abstract (arXiv)

The joint use of multiple imaging modalities for medical image segmentation has been widely studied in recent years. The fusion of information from different modalities has demonstrated to improve the segmentation accuracy, with respect to mono-modal segmentations, in several applications. However, acquiring multiple modalities is usually not possible in a clinical setting due to a limited number of physicians and scanners, and to limit costs and scan time. Most of the time, only one modality is acquired. In this paper, we propose KD-Net, a framework to transfer knowledge from a trained multi-modal network (teacher) to a mono-modal one (student). The proposed method is an adaptation of the generalized distillation framework where the student network is trained on a subset (1 modality) of the teacher's inputs (n modalities). We illustrate the effectiveness of the proposed framework in brain tumor segmentation with the BraTS 2018 dataset. Using different architectures, we show that the student network effectively learns from the teacher and always outperforms the baseline mono-modal network in terms of segmentation accuracy.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Builds on 11 of its 15 references that are in the map. First posted on arXiv 17 Jun 2021.

Builds on

11 of 15 references.

References · 11

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2019Structured Knowledge Distillation for Dense Predictioncited by 277
  3. 2015Unifying distillation and privileged informationcited by 218
  4. 2019MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentationcited by 195
  5. 2016HeMIS: Hetero-Modal Image Segmentationcited by 102
  6. 2019Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentationcited by 68
  7. 2020Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusioncited by 67
  8. 2018No New-Netcited by 61
  9. 2018Improving Fast Segmentation With Teacher-student Learningcited by 34
  10. 2018Cross-Modality Synthesis from CT to PET using FCN and GAN Networks for Improved Automated Lesion Detectioncited by 29
  11. 2018Simultaneous synthesis of FLAIR and segmentation of white matter hypointensities from T1 MRIscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 10; 2023: 15; 2024: 19; 2025: 9; 2026: 9.

68 papers.

Sort:

By area:

  1. 2023Multi-modal Learning with Missing Modality via Shared-Specific Feature Modellingcited by 159
  2. 2022mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentationcited by 94
  3. 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 89
  4. 2023M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalitiescited by 56
  5. 2021Modality-aware Mutual Learning for Multi-modal Medical Image Segmentationcited by 47
  6. 2022The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillationcited by 44
  7. 2023Learnable Cross-modal Knowledge Distillation for Multi-modal Learning with Missing Modalitycited by 44
  8. 2021ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalitiescited by 43
  9. 2021Efficient Medical Image Segmentation Based on Knowledge Distillationcited by 43
  10. 2023MMANet: Margin-aware Distillation and Modality-aware Regularization for Incomplete Multimodal Learningcited by 40
  11. 2024Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 32
  12. 2023Directional Connectivity-based Segmentation of Medical Imagescited by 30
38 more
  1. 2023Prototype Knowledge Distillation for Medical Segmentation with Missing Modalitycited by 24
  2. 2022ModDrop++: A Dynamic Filter Network with Intra-subject Co-training for Multiple Sclerosis Lesion Segmentation with Missing Modalitiescited by 22
  3. 2025SimMLM: A Simple Framework for Multi-modal Learning with Missing Modalitycited by 22
  4. 2022Medical Image Segmentation on MRI Images with Missing Modalities: A Reviewcited by 18
  5. 2023Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarkscited by 17
  6. 2023COSST: Multi-organ Segmentation with Partially Labeled Datasets Using Comprehensive Supervisions and Self-trainingcited by 16
  7. 2024Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learningcited by 16
  8. 2023Multimodal Machine Learning in Image-Based and Clinical Biomedicine: Survey and Prospectscited by 15
  9. 2022SFusion: Self-attention based N-to-One Multimodal Fusion Blockcited by 14
  10. 2023Enhancing Modality-Agnostic Representations via Meta-Learning for Brain Tumor Segmentationcited by 14
  11. 2021Feature-enhanced Generation and Multi-modality Fusion based Deep Neural Network for Brain Tumor Segmentation with Missing MR Modalitiescited by 13
  12. 2022Deep Multimodal Guidance for Medical Image Classificationcited by 11
  13. 2022Point-Unet: A Context-aware Point-based Neural Network for Volumetric Segmentationcited by 11
  14. 2023Enhanced Multimodal Representation Learning with Cross-modal KDcited by 11
  15. 2024Multimodal Data Integration for Precision Oncology: Challenges and Future Directionscited by 10
  16. 2024Federated Modality-specific Encoders and Multimodal Anchors for Personalized Brain Tumor Segmentationcited by 8
  17. 2024Feasibility and benefits of joint learning from MRI databases with different brain diseases and modalities for segmentationcited by 8
  18. 2022MISSU: 3D Medical Image Segmentation via Self-distilling TransUNetcited by 7
  19. 2021Segmentation with mixed supervision: Confidence maximization helps knowledge distillationcited by 6
  20. 2024ADAPT: Multimodal Learning for Detecting Physiological Changes under Missing Modalitiescited by 6
  21. 2024Meta-Learned Modality-Weighted Knowledge Distillation for Robust Multi-Modal Learning with Missing Datacited by 5
  22. 2024Feasibility of Federated Learning from Client Databases with Different Brain Diseases and MRI Modalitiescited by 5
  23. 2024Make a Strong Teacher with Label Assistance: A Novel Knowledge Distillation Approach for Semantic Segmentationcited by 5
  24. 2025DC-Seg: Disentangled Contrastive Learning for Brain Tumor Segmentation with Missing Modalitiescited by 5
  25. 2024Dealing with All-stage Missing Modality: Towards A Universal Model with Robust Reconstruction and Personalizationcited by 4
  26. 2024MedMAP: Promoting Incomplete Multi-modal Brain Tumor Segmentation with Alignmentcited by 4
  27. 2024Shape-intensity knowledge distillation for robust medical image segmentationcited by 4
  28. 2025Revisiting Cross-Modal Knowledge Distillation: A Disentanglement Approach for RGBD Semantic Segmentationcited by 4
  29. 2023CoNeS: Conditional neural fields with shift modulation for multi-sequence MRI translationcited by 3
  30. 2024Focus on Focus: Focus-oriented Representation Learning and Multi-view Cross-modal Alignment for Glioma Gradingcited by 3
  31. 2025Prototype-Guided Cross-Modal Knowledge Enhancement for Adaptive Survival Predictioncited by 3
  32. 2025Multimodal Masked Autoencoder Pre-training for 3D MRI-Based Brain Tumor Analysis with Missing Modalitiescited by 3
  33. 2026Federated Modality-specific Encoders and Partially Personalized Fusion Decoder for Multimodal Brain Tumor Segmentationcited by 3
  34. 2023Modality-Agnostic Learning for Medical Image Segmentation Using Multi-modality Self-distillationcited by 2
  35. 2024Anatomical Consistency Distillation and Inconsistency Synthesis for Brain Tumor Segmentation with Missing Modalitiescited by 2
  36. 2025MIND: Modality-Informed Knowledge Distillation Framework for Multimodal Clinical Prediction Taskscited by 2
  37. 2021A Few-shot Learning Graph Multi-Trajectory Evolution Network for Forecasting Multimodal Baby Connectivity Development from a Baseline Timepointcited by 1
  38. 2022LightVessel: Exploring Lightweight Coronary Artery Vessel Segmentation via Similarity Knowledge Distillationcited by 1

50 most cited of 68 All 68 (JSON)