mlsanityData to 7 Oct 2026

Uformer: A General U-Shaped Transformer for Image Restoration

arXiv 2106.03106 ↗First posted 6 Jun 2021cs.CVcited by 864

Not on the map. Why?

Cited by 76 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper, we present Uformer, an effective and efficient Transformer-based architecture for image restoration, in which we build a hierarchical encoder-decoder network using the Transformer block. In Uformer, there are two core designs. First, we introduce a novel locally-enhanced window (LeWin) Transformer block, which performs nonoverlapping window-based self-attention instead of global self-attention. It significantly reduces the computational complexity on high resolution feature map while capturing local context. Second, we propose a learnable multi-scale restoration modulator in the form of a multi-scale spatial bias to adjust features in multiple layers of the Uformer decoder. Our modulator demonstrates superior capability for restoring details for various image restoration tasks while introducing marginal extra parameters and computational cost. Powered by these two designs, Uformer enjoys a high capability for capturing both local and global dependencies for image restoration. To evaluate our approach, extensive experiments are conducted on several image restoration tasks, including image denoising, motion deblurring, defocus deblurring and deraining. Without bells and whistles, our Uformer achieves superior or comparable performance compared with the state-of-the-art algorithms. The code and models are available at https://github.com/ZhendongWang6/Uformer.

Key facts. Cited by 864 papers in the map (data through 7 Oct 2026). Builds on 66 of its 98 references that are in the map. First posted on arXiv 6 Jun 2021.

Builds on

66 of 98 references.

References · 66

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2016Densely Connected Convolutional Networkscited by 6,691
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  8. 2016Layer Normalizationcited by 5,579
  9. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  10. 2017Squeeze-and-Excitation Networkscited by 4,763
  11. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  12. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
38 more
  1. 2017Non-local Neural Networkscited by 2,290
  2. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  3. 2021Vision Transformers for Dense Predictioncited by 1,789
  4. 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,543
  5. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  6. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,352
  7. 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,256
  8. 2018Self-Attention with Relative Position Representationscited by 1,112
  9. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  10. 2021Transformers in Vision: A Surveycited by 1,009
  11. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  12. 2020Pre-Trained Image Processing Transformercited by 891
  13. 2021Multi-Stage Progressive Image Restorationcited by 813
  14. 2018Residual Dense Network for Image Super-Resolutioncited by 796
  15. 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 722
  16. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  17. 2019Deep Learning for Image Super-resolution: A Surveycited by 368
  18. 2017DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networkscited by 366
  19. 2019DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Bettercited by 361
  20. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  21. 2020Learning Texture Transformer Network for Image Super-Resolutioncited by 346
  22. 2018Scale-recurrent Network for Deep Image Deblurringcited by 332
  23. 2020Learning Enriched Features for Real Image Restoration and Enhancementcited by 311
  24. 2019Progressive Image Deraining Networks: A Better and Simpler Baselinecited by 271
  25. 2021Incorporating Convolution Designs into Visual Transformerscited by 248
  26. 2019Residual Non-local Attention Networks for Image Restorationcited by 236
  27. 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
  28. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  29. 2020CycleISP: Real Image Restoration via Improved Data Synthesiscited by 212
  30. 2017Benchmarking Denoising Algorithms with Real Photographscited by 209
  31. 2018Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Derainingcited by 205
  32. 2019Deep Stacked Hierarchical Multi-patch Network for Image Deblurringcited by 205
  33. 2019Spatial Attentive Single-Image Deraining with a High Quality Real Rain Datasetcited by 192
  34. 2020Deblurring by Realistic Blurringcited by 187
  35. 2019Real Image Denoising with Feature Attentioncited by 176
  36. 2020Human-Aware Motion Deblurringcited by 174
  37. 2021Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Imagescited by 171
  38. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 20; 2022: 114; 2023: 208; 2024: 231; 2025: 180; 2026: 111.

864 papers.

Sort:

By area:

  1. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  2. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
  3. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  4. 2022Simple Baselines for Image Restorationcited by 701
  5. 2022Activating More Pixels in Image Super-Resolution Transformercited by 433
  6. 2024MambaIR: A Simple Baseline for Image Restoration with State-Space Modelcited by 349
  7. 2022V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformercited by 320
  8. 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 317
  9. 2022MAXIM: Multi-Axis MLP for Image Processingcited by 301
  10. 2022Vision Transformers for Single Image Dehazingcited by 255
  11. 2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Priorcited by 255
  12. 2021TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditionscited by 243
38 more
  1. 2023Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancementcited by 228
  2. 2022CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusioncited by 222
  3. 2022VRT: A Video Restoration Transformercited by 219
  4. 2023Learning A Sparse Transformer Network for Effective Image Derainingcited by 204
  5. 2022Medical Image Segmentation Review: The success of U-Netcited by 190
  6. 2023Image Restoration with Mean-Reverting Stochastic Differential Equationscited by 182
  7. 2022Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Methodcited by 174
  8. 2022Recurrent Video Restoration Transformer with Guided Deformable Attentioncited by 168
  9. 2022Efficient Frequency Domain-based Transformers for High-Quality Image Deblurringcited by 164
  10. 2022Efficient Long-Range Attention Network for Image Super-resolutioncited by 151
  11. 2023Efficient and Explicit Modelling of Image Hierarchies for Image Restorationcited by 145
  12. 2024InstructIR: High-Quality Image Restoration Following Human Instructionscited by 139
  13. 2023Low-Light Image Enhancement with Wavelet-based Diffusion Modelscited by 126
  14. 2023Dual Aggregation Transformer for Image Super-Resolutioncited by 124
  15. 2024Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformerscited by 116
  16. 2022Parallel Diffusion Models of Operator and Image for Blind Inverse Problemscited by 113
  17. 2023PromptIR: Prompting for All-in-One Blind Image Restorationcited by 113
  18. 2021Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstructioncited by 110
  19. 2021U-shape Transformer for Underwater Image Enhancementcited by 110
  20. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  21. 2024AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulationcited by 108
  22. 2022AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensingcited by 107
  23. 2021MISSFormer: An Effective Medical Image Segmentation Transformercited by 103
  24. 2022ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removalcited by 102
  25. 2022PanoFormer: Panorama Transformer for Indoor 360 Depth Estimationcited by 94
  26. 2022Cross Aggregation Transformer for Image Restorationcited by 92
  27. 2024VmambaIR: Visual State Space Model for Image Restorationcited by 90
  28. 2023Hierarchical Integration Diffusion Model for Realistic Image Deblurringcited by 87
  29. 2023Embedding Fourier for Ultra-High-Definition Low-Light Image Enhancementcited by 86
  30. 2023Refusion: Enabling Large-Size Realistic Image Restoration with Latent-Space Diffusion Modelscited by 86
  31. 2022Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesiscited by 85
  32. 2022Accurate Image Restoration with Attention Retractable Transformercited by 85
  33. 2022You Only Need 90K Parameters to Adapt Light: A Light Weight Transformer for Image Enhancement and Exposure Correctioncited by 81
  34. 2023MB-TaylorFormer: Multi-branch Efficient Transformer Expanded by Taylor Formula for Image Dehazingcited by 81
  35. 2021Intriguing Findings of Frequency Selection for Image Deblurringcited by 80
  36. 2023Low-Light Image Enhancement via Structure Modeling and Guidancecited by 78
  37. 2023ProRes: Exploring Degradation-aware Visual Prompt for Universal Image Restorationcited by 77
  38. 2022Multiscale Structure Guided Diffusion for Image Deblurringcited by 76

50 most cited of 864 All 864 (JSON)