# Multiple Degradation and Reconstruction Network for Single Image Denoising via Knowledge Distillation

arXiv: https://arxiv.org/abs/2204.13873 · HTML: https://mlsanity.com/paper/2204.13873/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 29 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [Residual Non-local Attention Networks for Image Restoration](https://mlsanity.com/paper/1903.10082/) (2019)
- [Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration](https://mlsanity.com/paper/1508.02848/) (2015)
- [Residual Dense Network for Image Restoration](https://mlsanity.com/paper/1812.10477/) (2018)
- [Multi-level Wavelet Convolutional Neural Networks](https://mlsanity.com/paper/1907.03128/) (2019)
- [Seven ways to improve example-based single image super resolution](https://mlsanity.com/paper/1511.02228/) (2015)
- [PAMS: Quantized Super-Resolution via Parameterized Max Scale](https://mlsanity.com/paper/2011.04212/) (2020)
- [MDCN: Multi-scale Dense Cross Network for Image Super-Resolution](https://mlsanity.com/paper/2008.13084/) (2020)
- [Efficient and Accurate Multi-scale Topological Network for Single Image Dehazing](https://mlsanity.com/paper/2102.12135/) (2021)

## Built on by (most cited first, 5 of 5)

- [A cross Transformer for image denoising](https://mlsanity.com/paper/2310.10408/) (2023)
- [Real Image Denoising with Knowledge Distillation for High-Performance Mobile NPUs](https://mlsanity.com/paper/2605.03680/) (2026)
- [Spherical Image Inpainting with Frame Transformation and Data-driven Prior Deep Networks](https://mlsanity.com/paper/2209.14604/) (2022)
- [Defense against Unauthorized Distillation in Image Restoration via Feature Space Perturbation](https://mlsanity.com/paper/2510.08925/) (2025)
- [DIPA: Distilled Preconditioned Algorithms for Solving Imaging Inverse Problems](https://mlsanity.com/paper/2605.15456/) (2026)

## Abstract (arXiv)

Single image denoising (SID) has achieved significant breakthroughs with the development of deep learning. However, the proposed methods are often accompanied by plenty of parameters, which greatly limits their application scenarios. Different from previous works that blindly increase the depth of the network, we explore the degradation mechanism of the noisy image and propose a lightweight Multiple Degradation and Reconstruction Network (MDRN) to progressively remove noise. Meanwhile, we propose two novel Heterogeneous Knowledge Distillation Strategies (HMDS) to enable MDRN to learn richer and more accurate features from heterogeneous models, which make it possible to reconstruct higher-quality denoised images under extreme conditions. Extensive experiments show that our MDRN achieves favorable performance against other SID models with fewer parameters. Meanwhile, plenty of ablation studies demonstrate that the introduced HMDS can improve the performance of tiny models or the model under high noise levels, which is extremely useful for related applications.