# High-Frequency Prior-Driven Adaptive Masking for Accelerating Image Super-Resolution

arXiv: https://arxiv.org/abs/2505.06975 · HTML: https://mlsanity.com/paper/2505.06975/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 11 May 2025.

## Builds on (most cited first, 24 of 24)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Designing a Practical Degradation Model for Deep Blind Image Super-Resolution](https://mlsanity.com/paper/2103.14006/) (2021)
- [Activating More Pixels in Image Super-Resolution Transformer](https://mlsanity.com/paper/2205.04437/) (2022)
- [Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network](https://mlsanity.com/paper/1803.08664/) (2018)
- [Efficient Image Super-Resolution Using Pixel Attention](https://mlsanity.com/paper/2010.01073/) (2020)
- [ClassSR: A General Framework to Accelerate Super-Resolution Networks by Data Characteristic](https://mlsanity.com/paper/2103.04039/) (2021)
- [Learning with Privileged Information for Efficient Image Super-Resolution](https://mlsanity.com/paper/2007.07524/) (2020)
- [PAMS: Quantized Super-Resolution via Parameterized Max Scale](https://mlsanity.com/paper/2011.04212/) (2020)
- [Blueprint Separable Residual Network for Efficient Image Super-Resolution](https://mlsanity.com/paper/2205.05996/) (2022)
- [Learning Frequency-aware Dynamic Network for Efficient Super-Resolution](https://mlsanity.com/paper/2103.08357/) (2021)
- [Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search](https://mlsanity.com/paper/2108.08910/) (2021)
- [Journey Towards Tiny Perceptual Super-Resolution](https://mlsanity.com/paper/2007.04356/) (2020)
- [ARM: Any-Time Super-Resolution Method](https://mlsanity.com/paper/2203.10812/) (2022)
- [Adaptive Patch Exiting for Scalable Single Image Super-Resolution](https://mlsanity.com/paper/2203.11589/) (2022)
- [Deep Adaptive Inference Networks for Single Image Super-Resolution](https://mlsanity.com/paper/2004.03915/) (2020)
- [2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution](https://mlsanity.com/paper/2406.06649/) (2024)
- [Accelerating Image Super-Resolution Networks with Pixel-Level Classification](https://mlsanity.com/paper/2407.21448/) (2024)

## Abstract (arXiv)

The primary challenge in accelerating image super-resolution lies in reducing computation while maintaining performance and adaptability. Motivated by the observation that high-frequency regions (e.g., edges and textures) are most critical for reconstruction, we propose a training-free adaptive masking module for acceleration that dynamically focuses computation on these challenging areas. Specifically, our method first extracts high-frequency components via Gaussian blur subtraction and adaptively generates binary masks using K-means clustering to identify regions requiring intensive processing. Our method can be easily integrated with both CNNs and Transformers. For CNN-based architectures, we replace standard 3 × 3 convolutions with an unfold operation followed by 1 × 1 convolutions, enabling pixel-wise sparse computation guided by the mask. For Transformer-based models, we partition the mask into non-overlapping windows and selectively process tokens based on their average values. During inference, unnecessary pixels or windows are pruned, significantly reducing computation. Moreover, our method supports dilation-based mask adjustment to control the processing scope without retraining, and is robust to unseen degradations (e.g., noise, compression). Extensive experiments on benchmarks demonstrate that our method reduces FLOPs by 24--43% for state-of-the-art models (e.g., CARN, SwinIR) while achieving comparable or better quantitative metrics. The source code is available at https://github.com/shangwei5/AMSR