# Accelerating Image Super-Resolution Networks with Pixel-Level Classification

arXiv: https://arxiv.org/abs/2407.21448 · HTML: https://mlsanity.com/paper/2407.21448/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 31 Jul 2024.

## Builds on (most cited first, 19 of 19)

- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network](https://mlsanity.com/paper/1803.08664/) (2018)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [Efficient Image Super-Resolution Using Pixel Attention](https://mlsanity.com/paper/2010.01073/) (2020)
- [ClassSR: A General Framework to Accelerate Super-Resolution Networks by Data Characteristic](https://mlsanity.com/paper/2103.04039/) (2021)
- [Blueprint Separable Residual Network for Efficient Image Super-Resolution](https://mlsanity.com/paper/2205.05996/) (2022)
- [Learning Frequency-aware Dynamic Network for Efficient Super-Resolution](https://mlsanity.com/paper/2103.08357/) (2021)
- [Lightweight Bimodal Network for Single-Image Super-Resolution via Symmetric CNN and Recursive Transformer](https://mlsanity.com/paper/2204.13286/) (2022)
- [ARM: Any-Time Super-Resolution Method](https://mlsanity.com/paper/2203.10812/) (2022)
- [Adaptive Patch Exiting for Scalable Single Image Super-Resolution](https://mlsanity.com/paper/2203.11589/) (2022)
- [Deep Adaptive Inference Networks for Single Image Super-Resolution](https://mlsanity.com/paper/2004.03915/) (2020)

## Built on by (most cited first, 8 of 8)

- [Directing Mamba to Complex Textures: An Efficient Texture-Aware State Space Model for Image Restoration](https://mlsanity.com/paper/2501.16583/) (2025)
- [Towards Robust and Generalizable Continuous Space-Time Video Super-Resolution with Events](https://mlsanity.com/paper/2510.03833/) (2025)
- [Human Vision Constrained Super-Resolution](https://mlsanity.com/paper/2411.17513/) (2024)
- [High-Frequency Prior-Driven Adaptive Masking for Accelerating Image Super-Resolution](https://mlsanity.com/paper/2505.06975/) (2025)
- [Motion-Aware Adaptive Pixel Pruning for Efficient Local Motion Deblurring](https://mlsanity.com/paper/2507.07708/) (2025)
- [Adaptive Densification for High-Fidelity and Efficient Sparse Gaussian Splatting in Arbitrary-Scale Super-Resolution](https://mlsanity.com/paper/2606.29400/) (2026)
- [PixelSR: Efficient Screen Content Super-Resolution via Pixel Classification](https://mlsanity.com/paper/2608.00646/) (2026)
- [Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References](https://mlsanity.com/paper/2608.26476/) (2026)

## Abstract (arXiv)

In recent times, the need for effective super-resolution (SR) techniques has surged, especially for large-scale images ranging 2K to 8K resolutions. For DNN-based SISR, decomposing images into overlapping patches is typically necessary due to computational constraints. In such patch-decomposing scheme, one can allocate computational resources differently based on each patch's difficulty to further improve efficiency while maintaining SR performance. However, this approach has a limitation: computational resources is uniformly allocated within a patch, leading to lower efficiency when the patch contain pixels with varying levels of restoration difficulty. To address the issue, we propose the Pixel-level Classifier for Single Image Super-Resolution (PCSR), a novel method designed to distribute computational resources adaptively at the pixel level. A PCSR model comprises a backbone, a pixel-level classifier, and a set of pixel-level upsamplers with varying capacities. The pixel-level classifier assigns each pixel to an appropriate upsampler based on its restoration difficulty, thereby optimizing computational resource usage. Our method allows for performance and computational cost balance during inference without re-training. Our experiments demonstrate PCSR's advantage over existing patch-distributing methods in PSNR-FLOP trade-offs across different backbone models and benchmarks. The code is available at https://github.com/3587jjh/PCSR.