# Learning Frequency-aware Dynamic Network for Efficient Super-Resolution

arXiv: https://arxiv.org/abs/2103.08357 · HTML: https://mlsanity.com/paper/2103.08357/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 15 Mar 2021.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network](https://mlsanity.com/paper/1803.08664/) (2018)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [Lightweight Image Super-Resolution with Information Multi-distillation Network](https://mlsanity.com/paper/1909.11856/) (2019)
- [Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution](https://mlsanity.com/paper/1904.05049/) (2019)
- [Distilling Knowledge from Graph Convolutional Networks](https://mlsanity.com/paper/2003.10477/) (2020)
- [Fast and Accurate Single Image Super-Resolution via Information Distillation Network](https://mlsanity.com/paper/1803.09454/) (2018)
- [Cross-Scale Internal Graph Neural Network for Image Super-Resolution](https://mlsanity.com/paper/2006.16673/) (2020)
- [Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference](https://mlsanity.com/paper/1912.03203/) (2019)
- [Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search](https://mlsanity.com/paper/1901.07261/) (2019)
- [Efficient Residual Dense Block Search for Image Super-Resolution](https://mlsanity.com/paper/1909.11409/) (2019)
- [Deep Graph-Convolutional Image Denoising](https://mlsanity.com/paper/1907.08448/) (2019)
- [Efficient Super Resolution Using Binarized Neural Network](https://mlsanity.com/paper/1812.06378/) (2018)
- [Image Super-Resolution via Attention based Back Projection Networks](https://mlsanity.com/paper/1910.04476/) (2019)
- [Deep Adaptive Inference Networks for Single Image Super-Resolution](https://mlsanity.com/paper/2004.03915/) (2020)

## Built on by (most cited first, 30 of 36)

- [Efficient Real-world Image Super-Resolution Via Adaptive Directional Gradient Convolution](https://mlsanity.com/paper/2405.07023/) (2024)
- [CADyQ: Content-Aware Dynamic Quantization for Image Super-Resolution](https://mlsanity.com/paper/2207.10345/) (2022)
- [Self-Supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations](https://mlsanity.com/paper/2203.01325/) (2022)
- [LoFormer: Local Frequency Transformer for Image Deblurring](https://mlsanity.com/paper/2407.16993/) (2024)
- [ARM: Any-Time Super-Resolution Method](https://mlsanity.com/paper/2203.10812/) (2022)
- [Learning to Distill Global Representation for Sparse-View CT](https://mlsanity.com/paper/2308.08463/) (2023)
- [Frequency-Adaptive Pan-Sharpening with Mixture of Experts](https://mlsanity.com/paper/2401.02151/) (2024)
- [Frequency-Controlled Diffusion Model for Versatile Text-Guided Image-to-Image Translation](https://mlsanity.com/paper/2407.03006/) (2024)
- [RS-vHeat: Heat Conduction Guided Efficient Remote Sensing Foundation Model](https://mlsanity.com/paper/2411.17984/) (2024)
- [AdaBM: On-the-Fly Adaptive Bit Mapping for Image Super-Resolution](https://mlsanity.com/paper/2404.03296/) (2024)
- [Incremental Cross-view Mutual Distillation for Self-supervised Medical CT Synthesis](https://mlsanity.com/paper/2112.10325/) (2021)
- [RefQSR: Reference-based Quantization for Image Super-Resolution Networks](https://mlsanity.com/paper/2404.01690/) (2024)
- [Dynamic Pre-training: Towards Efficient and Scalable All-in-One Image Restoration](https://mlsanity.com/paper/2404.02154/) (2024)
- [Accelerating Image Super-Resolution Networks with Pixel-Level Classification](https://mlsanity.com/paper/2407.21448/) (2024)
- [Dual-domain Modulation Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2503.10047/) (2025)
- [Spatial-Frequency Attention for Image Denoising](https://mlsanity.com/paper/2302.13598/) (2023)
- [RBSR: Efficient and Flexible Recurrent Network for Burst Super-Resolution](https://mlsanity.com/paper/2306.17595/) (2023)
- [I3Net: Inter-Intra-slice Interpolation Network for Medical Slice Synthesis](https://mlsanity.com/paper/2405.02857/) (2024)
- [FBSDiff: Plug-and-Play Frequency Band Substitution of Diffusion Features for Highly Controllable Text-Driven Image Translation](https://mlsanity.com/paper/2408.00998/) (2024)
- [Self-Supervised Learning for Real-World Super-Resolution from Dual and Multiple Zoomed Observations](https://mlsanity.com/paper/2405.02171/) (2024)
- [Scale-Invariant Adversarial Attack against Arbitrary-scale Super-resolution](https://mlsanity.com/paper/2503.04385/) (2025)
- [Image super-resolution via dynamic network](https://mlsanity.com/paper/2310.10413/) (2023)
- [Invertible Residual Rescaling Models](https://mlsanity.com/paper/2405.02945/) (2024)
- [GlobalMamba: Global Image Serialization for Vision Mamba](https://mlsanity.com/paper/2410.10316/) (2024)
- [AFANet: Adaptive Frequency-Aware Network for Weakly-Supervised Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2412.17601/) (2024)
- [FADPNet: Frequency-Aware Dual-Path Network for Face Super-Resolution](https://mlsanity.com/paper/2506.14121/) (2025)
- [Deep Learning Empowered Super-Resolution: A Comprehensive Survey and Future Prospects](https://mlsanity.com/paper/2509.22692/) (2025)
- [Beyond Redundancy: Diverse and Specialized Multi-Expert Sparse Autoencoder](https://mlsanity.com/paper/2511.05745/) (2025)
- [FBSDiff++: Improved Frequency Band Substitution of Diffusion Features for Efficient and Highly Controllable Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2601.19115/) (2026)
- [DRNet: All-in-One Image Restoration via Prior-Guided Dynamic Reparameterization](https://mlsanity.com/paper/2605.08627/) (2026)

## Abstract (arXiv)

Deep learning based methods, especially convolutional neural networks (CNNs) have been successfully applied in the field of single image super-resolution (SISR). To obtain better fidelity and visual quality, most of existing networks are of heavy design with massive computation. However, the computation resources of modern mobile devices are limited, which cannot easily support the expensive cost. To this end, this paper explores a novel frequency-aware dynamic network for dividing the input into multiple parts according to its coefficients in the discrete cosine transform (DCT) domain. In practice, the high-frequency part will be processed using expensive operations and the lower-frequency part is assigned with cheap operations to relieve the computation burden. Since pixels or image patches belong to low-frequency areas contain relatively few textural details, this dynamic network will not affect the quality of resulting super-resolution images. In addition, we embed predictors into the proposed dynamic network to end-to-end fine-tune the handcrafted frequency-aware masks. Extensive experiments conducted on benchmark SISR models and datasets show that the frequency-aware dynamic network can be employed for various SISR neural architectures to obtain the better tradeoff between visual quality and computational complexity. For instance, we can reduce the FLOPs of SR models by approximate 50% while preserving state-of-the-art SISR performance.