# Fast and Memory-Efficient Network Towards Efficient Image Super-Resolution

arXiv: https://arxiv.org/abs/2204.08397 · HTML: https://mlsanity.com/paper/2204.08397/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 54 references that are in the map.
- First posted on arXiv 18 Apr 2022.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [Deeply-Recursive Convolutional Network for Image Super-Resolution](https://mlsanity.com/paper/1511.04491/) (2015)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Single Image Super-Resolution via a Holistic Attention Network](https://mlsanity.com/paper/2008.08767/) (2020)
- [Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network](https://mlsanity.com/paper/1803.08664/) (2018)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [Residual Non-local Attention Networks for Image Restoration](https://mlsanity.com/paper/1903.10082/) (2019)
- [Lightweight Image Super-Resolution with Information Multi-distillation Network](https://mlsanity.com/paper/1909.11856/) (2019)
- [Self-Distillation Amplifies Regularization in Hilbert Space](https://mlsanity.com/paper/2002.05715/) (2020)
- [Feedback Network for Image Super-Resolution](https://mlsanity.com/paper/1903.09814/) (2019)
- [Residual Feature Distillation Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2009.11551/) (2020)
- [Fast and Accurate Single Image Super-Resolution via Information Distillation Network](https://mlsanity.com/paper/1803.09454/) (2018)

## Built on by (most cited first, 27 of 27)

- [The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report](https://mlsanity.com/paper/2404.10343/) (2024)
- [N-Gram in Swin Transformers for Efficient Lightweight Image Super-Resolution](https://mlsanity.com/paper/2211.11436/) (2022)
- [HiT-SR: Hierarchical Transformer for Efficient Image Super-Resolution](https://mlsanity.com/paper/2407.05878/) (2024)
- [CAMixerSR: Only Details Need More "Attention"](https://mlsanity.com/paper/2402.19289/) (2024)
- [The Tenth NTIRE 2025 Efficient Super-Resolution Challenge Report](https://mlsanity.com/paper/2504.10686/) (2025)
- [The Eleventh NTIRE 2026 Efficient Super-Resolution Challenge Report](https://mlsanity.com/paper/2604.03198/) (2026)
- [AIM 2022 Challenge on Super-Resolution of Compressed Image and Video: Dataset, Methods and Results](https://mlsanity.com/paper/2208.11184/) (2022)
- [SSUMamba: Spatial-Spectral Selective State Space Model for Hyperspectral Image Denoising](https://mlsanity.com/paper/2405.01726/) (2024)
- [DIPNet: Efficiency Distillation and Iterative Pruning for Image Super-Resolution](https://mlsanity.com/paper/2304.07018/) (2023)
- [Real-Time 4K Super-Resolution of Compressed AVIF Images. AIS 2024 Challenge Survey](https://mlsanity.com/paper/2404.16484/) (2024)
- [Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Report](https://mlsanity.com/paper/2211.05910/) (2022)
- [Efficient Image Super-Resolution with Feature Interaction Weighted Hybrid Network](https://mlsanity.com/paper/2212.14181/) (2022)
- [AsConvSR: Fast and Lightweight Super-Resolution Network with Assembled Convolutions](https://mlsanity.com/paper/2305.03387/) (2023)
- [Reciprocal Attention Mixing Transformer for Lightweight Image Restoration](https://mlsanity.com/paper/2305.11474/) (2023)
- [Spiking Meets Attention: Efficient Remote Sensing Image Super-Resolution with Attention Spiking Neural Networks](https://mlsanity.com/paper/2503.04223/) (2025)
- [C2D-ISR: Optimizing Attention-based Image Super-resolution from Continuous to Discrete Scales](https://mlsanity.com/paper/2503.13740/) (2025)
- [Hundred-Kilobyte Lookup Tables for Efficient Single-Image Super-Resolution](https://mlsanity.com/paper/2312.06101/) (2023)
- [Channel-Partitioned Windowed Attention And Frequency Learning for Single Image Super-Resolution](https://mlsanity.com/paper/2407.16232/) (2024)
- [Lightweight network towards real-time image denoising on mobile devices](https://mlsanity.com/paper/2211.04687/) (2022)
- [AnySR: Realizing Image Super-Resolution as Any-Scale, Any-Resource](https://mlsanity.com/paper/2407.04241/) (2024)
- [Exploration of Lightweight Single Image Denoising with Transformers and Truly Fair Training](https://mlsanity.com/paper/2304.01805/) (2023)
- [LMLT: Low-to-high Multi-Level Vision Transformer for Image Super-Resolution](https://mlsanity.com/paper/2409.03516/) (2024)
- [UNet--: Memory-Efficient and Feature-Enhanced Network Architecture based on U-Net with Reduced Skip-Connections](https://mlsanity.com/paper/2412.18276/) (2024)
- [Involution and BSConv Multi-Depth Distillation Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2503.14779/) (2025)
- [DiMoSR: Feature Modulation via Multi-Branch Dilated Convolutions for Efficient Image Super-Resolution](https://mlsanity.com/paper/2505.21262/) (2025)
- [LCS: An AI-based Low-Complexity Scaler for Power-Efficient Super-Resolution of Game Content](https://mlsanity.com/paper/2507.22873/) (2025)
- [CUST: Clustered Unit-level Similarity Transformer for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2607.11088/) (2026)

## Abstract (arXiv)

Runtime and memory consumption are two important aspects for efficient image super-resolution (EISR) models to be deployed on resource-constrained devices. Recent advances in EISR exploit distillation and aggregation strategies with plenty of channel split and concatenation operations to make full use of limited hierarchical features. In contrast, sequential network operations avoid frequently accessing preceding states and extra nodes, and thus are beneficial to reducing the memory consumption and runtime overhead. Following this idea, we design our lightweight network backbone by mainly stacking multiple highly optimized convolution and activation layers and decreasing the usage of feature fusion. We propose a novel sequential attention branch, where every pixel is assigned an important factor according to local and global contexts, to enhance high-frequency details. In addition, we tailor the residual block for EISR and propose an enhanced residual block (ERB) to further accelerate the network inference. Finally, combining all the above techniques, we construct a fast and memory-efficient network (FMEN) and its small version FMEN-S, which runs 33% faster and reduces 74% memory consumption compared with the state-of-the-art EISR model: E-RFDN, the champion in AIM 2020 efficient super-resolution challenge. Besides, FMEN-S achieves the lowest memory consumption and the second shortest runtime in NTIRE 2022 challenge on efficient super-resolution. Code is available at https://github.com/NJU-Jet/FMEN.