# KD-MRI: A knowledge distillation framework for image reconstruction and image restoration in MRI workflow

arXiv: https://arxiv.org/abs/2004.05319 · HTML: https://mlsanity.com/paper/2004.05319/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2020.

## Built on by (most cited first, 10 of 10)

- [Universal Undersampled MRI Reconstruction](https://mlsanity.com/paper/2103.05214/) (2021)
- [Challenges for machine learning in clinical translation of big data imaging studies](https://mlsanity.com/paper/2107.05630/) (2021)
- [Unpaired cross-modality educed distillation (CMEDL) for medical image segmentation](https://mlsanity.com/paper/2107.07985/) (2021)
- [SFT-KD-Recon: Learning a Student-friendly Teacher for Knowledge Distillation in Magnetic Resonance Image Reconstruction](https://mlsanity.com/paper/2304.05057/) (2023)
- [Highly Constrained Coded Aperture Imaging Systems Design Via a Knowledge Distillation Approach](https://mlsanity.com/paper/2406.17970/) (2024)
- [KDPhys: An Attention Guided 3D to 2D Knowledge Distillation for Real-time Video-Based Physiological Measurement](https://mlsanity.com/paper/2601.00714/) (2026)
- [Accelerated Proton Resonance Frequency-based Magnetic Resonance Thermometry by Optimized Deep Learning Method](https://mlsanity.com/paper/2407.03308/) (2024)
- [Defense against Unauthorized Distillation in Image Restoration via Feature Space Perturbation](https://mlsanity.com/paper/2510.08925/) (2025)
- [Pruning Overparameterized Multi-Task Networks for Degraded Web Image Restoration](https://mlsanity.com/paper/2510.14463/) (2025)
- [DIPA: Distilled Preconditioned Algorithms for Solving Imaging Inverse Problems](https://mlsanity.com/paper/2605.15456/) (2026)

## Abstract (arXiv)

Deep learning networks are being developed in every stage of the MRI workflow and have provided state-of-the-art results. However, this has come at the cost of increased computation requirement and storage. Hence, replacing the networks with compact models at various stages in the MRI workflow can significantly reduce the required storage space and provide considerable speedup. In computer vision, knowledge distillation is a commonly used method for model compression. In our work, we propose a knowledge distillation (KD) framework for the image to image problems in the MRI workflow in order to develop compact, low-parameter models without a significant drop in performance. We propose a combination of the attention-based feature distillation method and imitation loss and demonstrate its effectiveness on the popular MRI reconstruction architecture, DC-CNN. We conduct extensive experiments using Cardiac, Brain, and Knee MRI datasets for 4x, 5x and 8x accelerations. We observed that the student network trained with the assistance of the teacher using our proposed KD framework provided significant improvement over the student network trained without assistance across all the datasets and acceleration factors. Specifically, for the Knee dataset, the student network achieves 65% parameter reduction, 2x faster CPU running time, and 1.5x faster GPU running time compared to the teacher. Furthermore, we compare our attention-based feature distillation method with other feature distillation methods. We also conduct an ablative study to understand the significance of attention-based distillation and imitation loss. We also extend our KD framework for MRI super-resolution and show encouraging results.