# Efficient Long-Range Attention Network for Image Super-resolution

arXiv: https://arxiv.org/abs/2203.06697 · HTML: https://mlsanity.com/paper/2203.06697/

## Key facts

- Cited by 151 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 70 references that are in the map.
- First posted on arXiv 13 Mar 2022.

## Builds on (most cited first, 30 of 50)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)

## Built on by (most cited first, 30 of 151)

- [MambaIR: A Simple Baseline for Image Restoration with State-Space Model](https://mlsanity.com/paper/2402.15648/) (2024)
- [SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution](https://mlsanity.com/paper/2311.16518/) (2023)
- [One-Step Effective Diffusion Network for Real-World Image Super-Resolution](https://mlsanity.com/paper/2406.08177/) (2024)
- [Dual Aggregation Transformer for Image Super-Resolution](https://mlsanity.com/paper/2308.03364/) (2023)
- [Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach](https://mlsanity.com/paper/2412.03017/) (2024)
- [MambaIRv2: Attentive State Space Restoration](https://mlsanity.com/paper/2411.15269/) (2024)
- [VmambaIR: Visual State Space Model for Image Restoration](https://mlsanity.com/paper/2403.11423/) (2024)
- [TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution](https://mlsanity.com/paper/2411.18263/) (2024)
- [A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shift](https://mlsanity.com/paper/2206.10810/) (2022)
- [AddSR: Accelerating Diffusion-based Blind Super-Resolution with Adversarial Diffusion Distillation](https://mlsanity.com/paper/2404.01717/) (2024)
- [Transcending the Limit of Local Window: Advanced Super-Resolution Transformer with Adaptive Token Dictionary](https://mlsanity.com/paper/2401.08209/) (2024)
- [Spatially-Adaptive Feature Modulation for Efficient Image Super-Resolution](https://mlsanity.com/paper/2302.13800/) (2023)
- [DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolution](https://mlsanity.com/paper/2503.23580/) (2025)
- [N-Gram in Swin Transformers for Efficient Lightweight Image Super-Resolution](https://mlsanity.com/paper/2211.11436/) (2022)
- [MaIR: A Locality- and Continuity-Preserving Mamba for Image Restoration](https://mlsanity.com/paper/2412.20066/) (2024)
- [HiT-SR: Hierarchical Transformer for Efficient Image Super-Resolution](https://mlsanity.com/paper/2407.05878/) (2024)
- [DRCT: Saving Image Super-resolution away from Information Bottleneck](https://mlsanity.com/paper/2404.00722/) (2024)
- [NTIRE 2024 Challenge on Image Super-Resolution (x4): Methods and Results](https://mlsanity.com/paper/2404.09790/) (2024)
- [CAMixerSR: Only Details Need More "Attention"](https://mlsanity.com/paper/2402.19289/) (2024)
- [Multi-scale Attention Network for Single Image Super-Resolution](https://mlsanity.com/paper/2209.14145/) (2022)
- [Recursive Generalization Transformer for Image Super-Resolution](https://mlsanity.com/paper/2303.06373/) (2023)
- [Progressive Focused Transformer for Single Image Super-Resolution](https://mlsanity.com/paper/2503.20337/) (2025)
- [ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution](https://mlsanity.com/paper/2307.14010/) (2023)
- [See More Details: Efficient Image Super-Resolution by Experts Mining](https://mlsanity.com/paper/2402.03412/) (2024)
- [DifIISR: A Diffusion Model with Gradient Guidance for Infrared Image Super-Resolution](https://mlsanity.com/paper/2503.01187/) (2025)
- [Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolution](https://mlsanity.com/paper/2501.06838/) (2025)
- [Fine-structure Preserved Real-world Image Super-resolution via Transfer VAE Training](https://mlsanity.com/paper/2507.20291/) (2025)
- [Human Guided Ground-truth Generation for Realistic Image Super-resolution](https://mlsanity.com/paper/2303.13069/) (2023)
- [SRFormerV2: Taking a Closer Look at Permuted Self-Attention for Image Super-Resolution](https://mlsanity.com/paper/2303.09735/) (2023)
- [A Benchmark for Chinese-English Scene Text Image Super-resolution](https://mlsanity.com/paper/2308.03262/) (2023)

## Abstract (arXiv)

Recently, transformer-based methods have demonstrated impressive results in various vision tasks, including image super-resolution (SR), by exploiting the self-attention (SA) for feature extraction. However, the computation of SA in most existing transformer based models is very expensive, while some employed operations may be redundant for the SR task. This limits the range of SA computation and consequently the SR performance. In this work, we propose an efficient long-range attention network (ELAN) for image SR. Specifically, we first employ shift convolution (shift-conv) to effectively extract the image local structural information while maintaining the same level of complexity as 1x1 convolution, then propose a group-wise multi-scale self-attention (GMSA) module, which calculates SA on non-overlapped groups of features using different window sizes to exploit the long-range image dependency. A highly efficient long-range attention block (ELAB) is then built by simply cascading two shift-conv with a GMSA module, which is further accelerated by using a shared attention mechanism. Without bells and whistles, our ELAN follows a fairly simple design by sequentially cascading the ELABs. Extensive experiments demonstrate that ELAN obtains even better results against the transformer-based SR models but with significantly less complexity. The source code can be found at https://github.com/xindongzhang/ELAN.