# ClassSR: A General Framework to Accelerate Super-Resolution Networks by Data Characteristic

arXiv: https://arxiv.org/abs/2103.04039 · HTML: https://mlsanity.com/paper/2103.04039/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 6 Mar 2021.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform](https://mlsanity.com/paper/1804.02815/) (2018)
- [Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network](https://mlsanity.com/paper/1803.08664/) (2018)
- [Lightweight Image Super-Resolution with Information Multi-distillation Network](https://mlsanity.com/paper/1909.11856/) (2019)
- [Efficient Image Super-Resolution Using Pixel Attention](https://mlsanity.com/paper/2010.01073/) (2020)
- [Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning](https://mlsanity.com/paper/1804.03312/) (2018)
- [Path-Restore: Learning Network Path Selection for Image Restoration](https://mlsanity.com/paper/1904.10343/) (2019)
- [RAISR: Rapid and Accurate Image Super Resolution](https://mlsanity.com/paper/1606.01299/) (2016)

## Built on by (most cited first, 30 of 81)

- [Activating More Pixels in Image Super-Resolution Transformer](https://mlsanity.com/paper/2205.04437/) (2022)
- [Reference-based Image Super-Resolution with Deformable Attention Transformer](https://mlsanity.com/paper/2207.11938/) (2022)
- [Towards Practical Certifiable Patch Defense with Vision Transformer](https://mlsanity.com/paper/2203.08519/) (2022)
- [Reflash Dropout in Image Super-Resolution](https://mlsanity.com/paper/2112.12089/) (2021)
- [CAMixerSR: Only Details Need More "Attention"](https://mlsanity.com/paper/2402.19289/) (2024)
- [Directing Mamba to Complex Textures: An Efficient Texture-Aware State Space Model for Image Restoration](https://mlsanity.com/paper/2501.16583/) (2025)
- [A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advances](https://mlsanity.com/paper/2505.17799/) (2025)
- [CADyQ: Content-Aware Dynamic Quantization for Image Super-Resolution](https://mlsanity.com/paper/2207.10345/) (2022)
- [AdaRevD: Adaptive Patch Exiting Reversible Decoder Pushes the Limit of Image Deblurring](https://mlsanity.com/paper/2406.09135/) (2024)
- [DVMSR: Distillated Vision Mamba for Efficient Super-Resolution](https://mlsanity.com/paper/2405.03008/) (2024)
- [Fine-structure Preserved Real-world Image Super-resolution via Transfer VAE Training](https://mlsanity.com/paper/2507.20291/) (2025)
- [Self-Supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations](https://mlsanity.com/paper/2203.01325/) (2022)
- [Human Guided Ground-truth Generation for Realistic Image Super-resolution](https://mlsanity.com/paper/2303.13069/) (2023)
- [ARM: Any-Time Super-Resolution Method](https://mlsanity.com/paper/2203.10812/) (2022)
- [Adaptive Patch Exiting for Scalable Single Image Super-Resolution](https://mlsanity.com/paper/2203.11589/) (2022)
- [CABM: Content-Aware Bit Mapping for Single Image Super-Resolution Network with Large Input](https://mlsanity.com/paper/2304.06454/) (2023)
- [CATANet: Efficient Content-Aware Token Aggregation for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2503.06896/) (2025)
- [HybridFlow: Infusing Continuity into Masked Codebook for Extreme Low-Bitrate Image Compression](https://mlsanity.com/paper/2404.13372/) (2024)
- [Towards Lightweight Super-Resolution with Dual Regression Learning](https://mlsanity.com/paper/2207.07929/) (2022)
- [Real-Time Image Demoireing on Mobile Devices](https://mlsanity.com/paper/2302.02184/) (2023)
- [Attentive Fine-Grained Structured Sparsity for Image Restoration](https://mlsanity.com/paper/2204.12266/) (2022)
- [CFAT: Unleashing TriangularWindows for Image Super-resolution](https://mlsanity.com/paper/2403.16143/) (2024)
- [Dynamic Slimmable Denoising Network](https://mlsanity.com/paper/2110.08940/) (2021)
- [AdaBM: On-the-Fly Adaptive Bit Mapping for Image Super-Resolution](https://mlsanity.com/paper/2404.03296/) (2024)
- [AsConvSR: Fast and Lightweight Super-Resolution Network with Assembled Convolutions](https://mlsanity.com/paper/2305.03387/) (2023)
- [TMP: Temporal Motion Propagation for Online Video Super-Resolution](https://mlsanity.com/paper/2312.09909/) (2023)
- [RefQSR: Reference-based Quantization for Image Super-Resolution Networks](https://mlsanity.com/paper/2404.01690/) (2024)
- [CDFormer:When Degradation Prediction Embraces Diffusion Model for Blind Image Super-Resolution](https://mlsanity.com/paper/2405.07648/) (2024)
- [Accelerating Image Super-Resolution Networks with Pixel-Level Classification](https://mlsanity.com/paper/2407.21448/) (2024)
- [AdaDiffSR: Adaptive Region-aware Dynamic Acceleration Diffusion Model for Real-World Image Super-Resolution](https://mlsanity.com/paper/2410.17752/) (2024)

## Abstract (arXiv)

We aim at accelerating super-resolution (SR) networks on large images (2K-8K). The large images are usually decomposed into small sub-images in practical usages. Based on this processing, we found that different image regions have different restoration difficulties and can be processed by networks with different capacities. Intuitively, smooth areas are easier to super-solve than complex textures. To utilize this property, we can adopt appropriate SR networks to process different sub-images after the decomposition. On this basis, we propose a new solution pipeline -- ClassSR that combines classification and SR in a unified framework. In particular, it first uses a Class-Module to classify the sub-images into different classes according to restoration difficulties, then applies an SR-Module to perform SR for different classes. The Class-Module is a conventional classification network, while the SR-Module is a network container that consists of the to-be-accelerated SR network and its simplified versions. We further introduce a new classification method with two losses -- Class-Loss and Average-Loss to produce the classification results. After joint training, a majority of sub-images will pass through smaller networks, thus the computational cost can be significantly reduced. Experiments show that our ClassSR can help most existing methods (e.g., FSRCNN, CARN, SRResNet, RCAN) save up to 50% FLOPs on DIV8K datasets. This general framework can also be applied in other low-level vision tasks.