# Distillation-Supervised Convolutional Low-Rank Adaptation for Efficient Image Super-Resolution

arXiv: https://arxiv.org/abs/2504.11271 · HTML: https://mlsanity.com/paper/2504.11271/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 15 references that are in the map.
- First posted on arXiv 15 Apr 2025.

## Builds on (most cited first, 8 of 8)

- [Patient Knowledge Distillation for BERT Model Compression](https://mlsanity.com/paper/1908.09355/) (2019)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Sparse Low-rank Adaptation of Pre-trained Language Models](https://mlsanity.com/paper/2311.11696/) (2023)
- [Spatially-Adaptive Feature Modulation for Efficient Image Super-Resolution](https://mlsanity.com/paper/2302.13800/) (2023)
- [The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report](https://mlsanity.com/paper/2404.10343/) (2024)
- [DIPNet: Efficiency Distillation and Iterative Pruning for Image Super-Resolution](https://mlsanity.com/paper/2304.07018/) (2023)
- [MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning](https://mlsanity.com/paper/2410.18035/) (2024)
- [ConvLoRA and AdaBN based Domain Adaptation via Self-Training](https://mlsanity.com/paper/2402.04964/) (2024)

## Built on by (most cited first, 2 of 2)

- [The Eleventh NTIRE 2026 Efficient Super-Resolution Challenge Report](https://mlsanity.com/paper/2604.03198/) (2026)
- [LOREN: Low Rank-Based Code-Rate Adaptation in Neural Receivers](https://mlsanity.com/paper/2602.10770/) (2026)

## Abstract (arXiv)

Convolutional neural networks (CNNs) have been widely used in efficient image super-resolution. However, for CNN-based methods, performance gains often require deeper networks and larger feature maps, which increase complexity and inference costs. Inspired by LoRA's success in fine-tuning large language models, we explore its application to lightweight models and propose Distillation-Supervised Convolutional Low-Rank Adaptation (DSCLoRA), which improves model performance without increasing architectural complexity or inference costs. Specifically, we integrate ConvLoRA into the efficient SR network SPAN by replacing the SPAB module with the proposed SConvLB module and incorporating ConvLoRA layers into both the pixel shuffle block and its preceding convolutional layer. DSCLoRA leverages low-rank decomposition for parameter updates and employs a spatial feature affinity-based knowledge distillation strategy to transfer second-order statistical information from teacher models (pre-trained SPAN) to student models (ours). This method preserves the core knowledge of lightweight models and facilitates optimal solution discovery under certain conditions. Experiments on benchmark datasets show that DSCLoRA improves PSNR and SSIM over SPAN while maintaining its efficiency and competitive image quality. Notably, DSCLoRA ranked first in the Overall Performance Track of the NTIRE 2025 Efficient Super-Resolution Challenge. Our code and models are made publicly available at https://github.com/Yaozzz666/DSCF-SR.