# Knowledge Distillation based Degradation Estimation for Blind Super-Resolution

arXiv: https://arxiv.org/abs/2211.16928 · HTML: https://mlsanity.com/paper/2211.16928/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 46 of its 61 references that are in the map.
- First posted on arXiv 30 Nov 2022.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 30 of 46)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Designing a Practical Degradation Model for Deep Blind Image Super-Resolution](https://mlsanity.com/paper/2103.14006/) (2021)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Model](https://mlsanity.com/paper/1904.00523/) (2019)
- [Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform](https://mlsanity.com/paper/1804.02815/) (2018)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [Variational Information Distillation for Knowledge Transfer](https://mlsanity.com/paper/1904.05835/) (2019)
- ["Zero-Shot" Super-Resolution using Deep Internal Learning](https://mlsanity.com/paper/1712.06087/) (2017)
- [Structured Knowledge Distillation for Dense Prediction](https://mlsanity.com/paper/1903.04197/) (2019)
- [Deep Unfolding Network for Image Super-Resolution](https://mlsanity.com/paper/2003.10428/) (2020)
- [Component Divide-and-Conquer for Real-World Image Super-Resolution](https://mlsanity.com/paper/2008.01928/) (2020)
- [Learning a Single Convolutional Super-Resolution Network for Multiple Degradations](https://mlsanity.com/paper/1712.06116/) (2017)
- [Blind Super-Resolution With Iterative Kernel Correction](https://mlsanity.com/paper/1904.03377/) (2019)
- [Unsupervised Degradation Representation Learning for Blind Super-Resolution](https://mlsanity.com/paper/2104.00416/) (2021)
- [Unfolding the Alternating Optimization for Blind Super Resolution](https://mlsanity.com/paper/2010.02631/) (2020)
- [Meta-Transfer Learning for Zero-Shot Super-Resolution](https://mlsanity.com/paper/2002.12213/) (2020)
- [Structure-Preserving Super Resolution with Gradient Guidance](https://mlsanity.com/paper/2003.13081/) (2020)

## Built on by (most cited first, 28 of 28)

- [DiffIR: Efficient Diffusion Model for Image Restoration](https://mlsanity.com/paper/2303.09472/) (2023)
- [VmambaIR: Visual State Space Model for Image Restoration](https://mlsanity.com/paper/2403.11423/) (2024)
- [AddSR: Accelerating Diffusion-based Blind Super-Resolution with Adversarial Diffusion Distillation](https://mlsanity.com/paper/2404.01717/) (2024)
- [DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curation](https://mlsanity.com/paper/2410.18666/) (2024)
- [Efficient Real-world Image Super-Resolution Via Adaptive Directional Gradient Convolution](https://mlsanity.com/paper/2405.07023/) (2024)
- [LLMGA: Multimodal Large Language Model based Generation Assistant](https://mlsanity.com/paper/2311.16500/) (2023)
- [DiffI2I: Efficient Diffusion Model for Image-to-Image Translation](https://mlsanity.com/paper/2308.13767/) (2023)
- [Deep Equilibrium Diffusion Restoration with Parallel Sampling](https://mlsanity.com/paper/2311.11600/) (2023)
- [Degradation-Aware Self-Attention Based Transformer for Blind Image Super-Resolution](https://mlsanity.com/paper/2310.04180/) (2023)
- [DORNet: A Degradation Oriented and Regularized Network for Blind Depth Super-Resolution](https://mlsanity.com/paper/2410.11666/) (2024)
- [CDFormer:When Degradation Prediction Embraces Diffusion Model for Blind Image Super-Resolution](https://mlsanity.com/paper/2405.07648/) (2024)
- [Blind Super-Resolution via Meta-learning and Markov Chain Monte Carlo Simulation](https://mlsanity.com/paper/2406.08896/) (2024)
- [Blind Video Super-Resolution based on Implicit Kernels](https://mlsanity.com/paper/2503.07856/) (2025)
- [OSMamba: Omnidirectional Spectral Mamba with Dual-Domain Prior Generator for Exposure Correction](https://mlsanity.com/paper/2411.15255/) (2024)
- [INDIGO+: A Unified INN-Guided Probabilistic Diffusion Algorithm for Blind and Non-Blind Image Restoration](https://mlsanity.com/paper/2501.14014/) (2025)
- [Suppressing Uncertainties in Degradation Estimation for Blind Super-Resolution](https://mlsanity.com/paper/2406.16459/) (2024)
- [Kernelized Back-Projection Networks for Blind Super Resolution](https://mlsanity.com/paper/2302.08478/) (2023)
- [Preserving Full Degradation Details for Blind Image Super-Resolution](https://mlsanity.com/paper/2407.01299/) (2024)
- [BUFF: Bayesian Uncertainty Guided Diffusion Probabilistic Model for Single Image Super-Resolution](https://mlsanity.com/paper/2504.03490/) (2025)
- [Adaptive Blind Super-Resolution Network for Spatial-Specific and Spatial-Agnostic Degradations](https://mlsanity.com/paper/2506.07705/) (2025)
- [DACESR: Degradation-Aware Conditional Embedding for Real-World Image Super-Resolution](https://mlsanity.com/paper/2602.23890/) (2026)
- [Learning Correction Errors via Frequency-Self Attention for Blind Image Super-Resolution](https://mlsanity.com/paper/2403.07390/) (2024)
- [Low-Trace Adaptation of Zero-shot Self-supervised Blind Image Denoising](https://mlsanity.com/paper/2403.12382/) (2024)
- [Content-decoupled Contrastive Learning-based Implicit Degradation Modeling for Blind Image Super-Resolution](https://mlsanity.com/paper/2408.05440/) (2024)
- [Pluggable Style Representation Learning for Multi-Style Transfer](https://mlsanity.com/paper/2503.20368/) (2025)
- [LatentINDIGO: An INN-Guided Latent Diffusion Algorithm for Image Restoration](https://mlsanity.com/paper/2505.12935/) (2025)
- [LightBSR: Towards Lightweight Blind Super-Resolution via Discriminative Implicit Degradation Representation Learning](https://mlsanity.com/paper/2506.22710/) (2025)
- [Interest Entanglement: The Hidden Barrier to Blind Super-Resolution Optimization](https://mlsanity.com/paper/2606.22353/) (2026)

## Abstract (arXiv)

Blind image super-resolution (Blind-SR) aims to recover a high-resolution (HR) image from its corresponding low-resolution (LR) input image with unknown degradations. Most of the existing works design an explicit degradation estimator for each degradation to guide SR. However, it is infeasible to provide concrete labels of multiple degradation combinations (e.g., blur, noise, jpeg compression) to supervise the degradation estimator training. In addition, these special designs for certain degradation, such as blur, impedes the models from being generalized to handle different degradations. To this end, it is necessary to design an implicit degradation estimator that can extract discriminative degradation representation for all degradations without relying on the supervision of degradation ground-truth. In this paper, we propose a Knowledge Distillation based Blind-SR network (KDSR). It consists of a knowledge distillation based implicit degradation estimator network (KD-IDE) and an efficient SR network. To learn the KDSR model, we first train a teacher network: KD-IDET. It takes paired HR and LR patches as inputs and is optimized with the SR network jointly. Then, we further train a student network KD-IDES, which only takes LR images as input and learns to extract the same implicit degradation representation (IDR) as KD-IDET. In addition, to fully use extracted IDR, we design a simple, strong, and efficient IDR based dynamic convolution residual block (IDR-DCRB) to build an SR network. We conduct extensive experiments under classic and real-world degradation settings. The results show that KDSR achieves SOTA performance and can generalize to various degradation processes. The source codes and pre-trained models will be released.