mlsanityData to 7 Oct 2026

Efficient Long-Range Attention Network for Image Super-resolution

arXiv 2203.06697 ↗First posted 13 Mar 2022cs.CVcited by 151

Not on the map. Why?

Cited by 24 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recently, transformer-based methods have demonstrated impressive results in various vision tasks, including image super-resolution (SR), by exploiting the self-attention (SA) for feature extraction. However, the computation of SA in most existing transformer based models is very expensive, while some employed operations may be redundant for the SR task. This limits the range of SA computation and consequently the SR performance. In this work, we propose an efficient long-range attention network (ELAN) for image SR. Specifically, we first employ shift convolution (shift-conv) to effectively extract the image local structural information while maintaining the same level of complexity as 1x1 convolution, then propose a group-wise multi-scale self-attention (GMSA) module, which calculates SA on non-overlapped groups of features using different window sizes to exploit the long-range image dependency. A highly efficient long-range attention block (ELAB) is then built by simply cascading two shift-conv with a GMSA module, which is further accelerated by using a shared attention mechanism. Without bells and whistles, our ELAN follows a fairly simple design by sequentially cascading the ELABs. Extensive experiments demonstrate that ELAN obtains even better results against the transformer-based SR models but with significantly less complexity. The source code can be found at https://github.com/xindongzhang/ELAN.

Key facts. Cited by 151 papers in the map (data through 7 Oct 2026). Builds on 50 of its 70 references that are in the map. First posted on arXiv 13 Mar 2022.

Builds on

50 of 70 references.

References · 50

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  9. 2016Densely Connected Convolutional Networkscited by 6,691
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  11. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  12. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
38 more
  1. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  2. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
  3. 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,543
  4. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  5. 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,256
  6. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
  7. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
  8. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,218
  9. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  10. 2015Accurate Image Super-Resolution Using Very Deep Convolutional Networkscited by 902
  11. 2020Pre-Trained Image Processing Transformercited by 891
  12. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 864
  13. 2018Residual Dense Network for Image Super-Resolutioncited by 796
  14. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  15. 2019EDVR: Video Restoration with Enhanced Deformable Convolutional Networkscited by 492
  16. 2020Plug-and-Play Image Restoration with Deep Denoiser Priorcited by 467
  17. 2017Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolutioncited by 452
  18. 2016Accelerating the Super-Resolution Convolutional Neural Networkcited by 393
  19. 2018Deep Learning for Generic Object Detection: A Surveycited by 392
  20. 2020Visual Transformers: Token-based Image Representation and Processing for Computer Visioncited by 333
  21. 2015Sketch-based Manga Retrieval using Manga109 Datasetcited by 326
  22. 2020Single Image Super-Resolution via a Holistic Attention Networkcited by 265
  23. 2018Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Networkcited by 263
  24. 2017MemNet: A Persistent Memory Network for Image Restorationcited by 244
  25. 2019Residual Non-local Attention Networks for Image Restorationcited by 236
  26. 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
  27. 2019Lightweight Image Super-Resolution with Information Multi-distillation Networkcited by 227
  28. 2016EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesiscited by 177
  29. 2018Frame-Recurrent Video Super-Resolutioncited by 160
  30. 2018Non-Local Recurrent Network for Image Restorationcited by 160
  31. 2016Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensationcited by 156
  32. 2019Feedback Network for Image Super-Resolutioncited by 155
  33. 2017Detail-revealing Deep Video Super-resolutioncited by 147
  34. 2020Residual Feature Distillation Network for Lightweight Image Super-Resolutioncited by 122
  35. 2020Cross-Scale Internal Graph Neural Network for Image Super-Resolutioncited by 94
  36. 2017Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutionscited by 93
  37. 2021LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-Resolution and Beyondcited by 88
  38. 2018Identifying Recurring Patterns with Deep Neural Networks for Natural Image Denoisingcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 30; 2024: 46; 2025: 43; 2026: 24.

151 papers.

Sort:

By area:

  1. 2024MambaIR: A Simple Baseline for Image Restoration with State-Space Modelcited by 349
  2. 2023SeeSR: Towards Semantics-Aware Real-World Image Super-Resolutioncited by 311
  3. 2024One-Step Effective Diffusion Network for Real-World Image Super-Resolutioncited by 243
  4. 2023Dual Aggregation Transformer for Image Super-Resolutioncited by 124
  5. 2024Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approachcited by 93
  6. 2024VmambaIR: Visual State Space Model for Image Restorationcited by 90
  7. 2024MambaIRv2: Attentive State Space Restorationcited by 90
  8. 2024TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolutioncited by 83
  9. 2022A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shiftcited by 68
  10. 2024AddSR: Accelerating Diffusion-based Blind Super-Resolution with Adversarial Diffusion Distillationcited by 66
  11. 2024Transcending the Limit of Local Window: Advanced Super-Resolution Transformer with Adaptive Token Dictionarycited by 63
  12. 2023Spatially-Adaptive Feature Modulation for Efficient Image Super-Resolutioncited by 56
38 more
  1. 2025DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolutioncited by 50
  2. 2022N-Gram in Swin Transformers for Efficient Lightweight Image Super-Resolutioncited by 46
  3. 2024MaIR: A Locality- and Continuity-Preserving Mamba for Image Restorationcited by 43
  4. 2024HiT-SR: Hierarchical Transformer for Efficient Image Super-Resolutioncited by 42
  5. 2024DRCT: Saving Image Super-resolution away from Information Bottleneckcited by 40
  6. 2024NTIRE 2024 Challenge on Image Super-Resolution (x4): Methods and Resultscited by 40
  7. 2024CAMixerSR: Only Details Need More "Attention"cited by 30
  8. 2022Multi-scale Attention Network for Single Image Super-Resolutioncited by 29
  9. 2023Recursive Generalization Transformer for Image Super-Resolutioncited by 28
  10. 2025Progressive Focused Transformer for Single Image Super-Resolutioncited by 25
  11. 2023ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolutioncited by 23
  12. 2024See More Details: Efficient Image Super-Resolution by Experts Miningcited by 23
  13. 2025Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolutioncited by 21
  14. 2025DifIISR: A Diffusion Model with Gradient Guidance for Infrared Image Super-Resolutioncited by 21
  15. 2025Fine-structure Preserved Real-world Image Super-resolution via Transfer VAE Trainingcited by 21
  16. 2023Human Guided Ground-truth Generation for Realistic Image Super-resolutioncited by 20
  17. 2023SRFormerV2: Taking a Closer Look at Permuted Self-Attention for Image Super-Resolutioncited by 18
  18. 2023A Benchmark for Chinese-English Scene Text Image Super-resolutioncited by 17
  19. 2024Training Generative Image Super-Resolution Models by Wavelet-Domain Losses Enables Better Control of Artifactscited by 16
  20. 2023Improving the Stability and Efficiency of Diffusion Models for Content Consistent Super-Resolutioncited by 15
  21. 2025CATANet: Efficient Content-Aware Token Aggregation for Lightweight Image Super-Resolutioncited by 15
  22. 2022Towards Lightweight Super-Resolution with Dual Regression Learningcited by 13
  23. 2022Efficient Image Super-Resolution with Feature Interaction Weighted Hybrid Networkcited by 13
  24. 2023DLGSANet: Lightweight Dynamic Local and Global Self-Attention Networks for Image Super-Resolutioncited by 13
  25. 2023Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformerscited by 13
  26. 2024Transforming Image Super-Resolution: A ConvFormer-based Efficient Approachcited by 13
  27. 2022HIPA: Hierarchical Patch Transformer for Single Image Super Resolutioncited by 11
  28. 2023WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removalcited by 11
  29. 2023Perception-Distortion Balanced Super-Resolution: A Multi-Objective Optimization Perspectivecited by 11
  30. 2023Exploring Frequency-Inspired Optimization in Transformer for Efficient Single Image Super-Resolutioncited by 10
  31. 2024Training Transformer Models by Wavelet Losses Improves Quantitative and Visual Performance in Single Image Super-Resolutioncited by 10
  32. 2024Development of Skip Connection in Deep Neural Networks for Computer Vision and Medical Image Analysis: A Surveycited by 10
  33. 2023RViDeformer: Efficient Raw Video Denoising Transformer with a Larger Benchmark Datasetcited by 8
  34. 2024RefQSR: Reference-based Quantization for Image Super-Resolution Networkscited by 8
  35. 2025Dual-domain Modulation Network for Lightweight Image Super-Resolutioncited by 8
  36. 2024CasSR: Activating Image Power for Real-World Image Super-Resolutioncited by 7
  37. 2024Partial Large Kernel CNNs for Efficient Super-Resolutioncited by 7
  38. 2025DSwinIR: Rethinking Window-based Attention for Image Restorationcited by 7

50 most cited of 151 All 151 (JSON)