# Compressing Deep Image Super-resolution Models arXiv: https://arxiv.org/abs/2401.00523 · HTML: https://mlsanity.com/paper/2401.00523/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation. - Builds on 29 of its 46 references that are in the map. - First posted on arXiv 31 Dec 2023. ## Placed in - Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/) ## Builds on (most cited first, 29 of 29) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021) - [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017) - [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018) - [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014) - [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015) - [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021) - [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017) - [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017) - [Deep Learning for Image Super-resolution: A Survey](https://mlsanity.com/paper/1902.06068/) (2019) - [A Survey of Model Compression and Acceleration for Deep Neural Networks](https://mlsanity.com/paper/1710.09282/) (2017) - [Distilling Knowledge via Knowledge Review](https://mlsanity.com/paper/2104.09044/) (2021) - [Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network](https://mlsanity.com/paper/1803.08664/) (2018) - [Knowledge distillation: A good teacher is patient and consistent](https://mlsanity.com/paper/2106.05237/) (2021) - [Lightweight Image Super-Resolution with Information Multi-distillation Network](https://mlsanity.com/paper/1909.11856/) (2019) - [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022) - [Context-aware Synthesis for Video Frame Interpolation](https://mlsanity.com/paper/1803.10967/) (2018) - [Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration](https://mlsanity.com/paper/2209.11345/) (2022) - [HiNeRV: Video Compression with Hierarchical Encoding-based Neural Representation](https://mlsanity.com/paper/2306.09818/) (2023) - [Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search](https://mlsanity.com/paper/1901.07261/) (2019) - [CDFI: Compression-Driven Network Design for Frame Interpolation](https://mlsanity.com/paper/2103.10559/) (2021) - [Residual Local Feature Network for Efficient Super-Resolution](https://mlsanity.com/paper/2205.07514/) (2022) - [Orthant Based Proximal Stochastic Gradient Method for ℓ1-Regularized Optimization](https://mlsanity.com/paper/2004.03639/) (2020) - [MFRNet: A New CNN Architecture for Post-Processing and In-loop Filtering](https://mlsanity.com/paper/2007.07099/) (2020) - [CVEGAN: A Perceptually-inspired GAN for Compressed Video Enhancement](https://mlsanity.com/paper/2011.09190/) (2020) - [ST-MFNet Mini: Knowledge Distillation-Driven Frame Interpolation](https://mlsanity.com/paper/2302.08455/) (2023) - [AsConvSR: Fast and Lightweight Super-Resolution Network with Assembled Convolutions](https://mlsanity.com/paper/2305.03387/) (2023) - [ViSTRA2: Video Coding using Spatial Resolution and Effective Bit Depth Adaptation](https://mlsanity.com/paper/1911.02833/) (2019) ## Built on by (most cited first, 7 of 7) - [The Tenth NTIRE 2025 Efficient Super-Resolution Challenge Report](https://mlsanity.com/paper/2504.10686/) (2025) - [MTKD: Multi-Teacher Knowledge Distillation for Image Super-Resolution](https://mlsanity.com/paper/2404.09571/) (2024) - [AIM 2024 Challenge on Efficient Video Super-Resolution for AV1 Compressed Content](https://mlsanity.com/paper/2409.17256/) (2024) - [C2D-ISR: Optimizing Attention-based Image Super-resolution from Continuous to Discrete Scales](https://mlsanity.com/paper/2503.13740/) (2025) - [RTSR: A Real-Time Super-Resolution Model for AV1 Compressed Content](https://mlsanity.com/paper/2411.13362/) (2024) - [Compressed Video Super-Resolution based on Hierarchical Encoding](https://mlsanity.com/paper/2506.14381/) (2025) - [Pruning Overparameterized Multi-Task Networks for Degraded Web Image Restoration](https://mlsanity.com/paper/2510.14463/) (2025) ## Abstract (arXiv) Deep learning techniques have been applied in the context of image super-resolution (SR), achieving remarkable advances in terms of reconstruction performance. Existing techniques typically employ highly complex model structures which result in large model sizes and slow inference speeds. This often leads to high energy consumption and restricts their adoption for practical applications. To address this issue, this work employs a three-stage workflow for compressing deep SR models which significantly reduces their memory requirement. Restoration performance has been maintained through teacher-student knowledge distillation using a newly designed distillation loss. We have applied this approach to two popular image super-resolution networks, SwinIR and EDSR, to demonstrate its effectiveness. The resulting compact models, SwinIRmini and EDSRmini, attain an 89% and 96% reduction in both model size and floating-point operations (FLOPs) respectively, compared to their original versions. They also retain competitive super-resolution performance compared to their original models and other commonly used SR approaches. The source code and pre-trained models for these two lightweight SR approaches are released at https://pikapi22.github.io/CDISM/.