mlsanityData to 7 Oct 2026

Learning A Sparse Transformer Network for Effective Image Deraining

arXiv 2303.11950 ↗First posted 21 Mar 2023cs.CVcited by 204

Not on the map. Why?

Abstract (arXiv)

Transformers-based methods have achieved significant performance in image deraining as they can model the non-local information which is vital for high-quality image reconstruction. In this paper, we find that most existing Transformers usually use all similarities of the tokens from the query-key pairs for the feature aggregation. However, if the tokens from the query are different from those of the key, the self-attention values estimated from these tokens also involve in feature aggregation, which accordingly interferes with the clear image restoration. To overcome this problem, we propose an effective DeRaining network, Sparse Transformer (DRSformer) that can adaptively keep the most useful self-attention values for feature aggregation so that the aggregated features better facilitate high-quality image reconstruction. Specifically, we develop a learnable top-k selection operator to adaptively retain the most crucial attention scores from the keys for each query for better feature aggregation. Simultaneously, as the naive feed-forward network in Transformers does not model the multi-scale information that is important for latent clear image restoration, we develop an effective mixed-scale feed-forward network to generate better features for image deraining. To learn an enriched set of hybrid features, which combines local context from CNN operators, we equip our model with mixture of experts feature compensator to present a cooperation refinement deraining scheme. Extensive experimental results on the commonly used benchmarks demonstrate that the proposed method achieves favorable performance against state-of-the-art approaches. The source code and trained models are available at https://github.com/cschenxiang/DRSformer.

Key facts. Cited by 204 papers in the map (data through 7 Oct 2026). Builds on 40 of its 66 references that are in the map. First posted on arXiv 21 Mar 2023.

Builds on

40 of 66 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2017Squeeze-and-Excitation Networkscited by 4,763
  7. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
  8. 2020Pre-Trained Image Processing Transformercited by 891
  9. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 864
  10. 2021Multi-Stage Progressive Image Restorationcited by 813
  11. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  12. 2016Deep Joint Rain Detection and Removal from a Single Imagecited by 351
28 more
  1. 2020Learning Texture Transformer Network for Image Super-Resolutioncited by 346
  2. 2019Progressive Image Deraining Networks: A Better and Simpler Baselinecited by 271
  3. 2020Multi-Scale Progressive Fusion Network for Single Image Derainingcited by 259
  4. 2018Density-aware Single Image De-raining using a Multi-stream Dense Networkcited by 248
  5. 2021Incorporating Convolution Designs into Visual Transformerscited by 248
  6. 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
  7. 2018Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Derainingcited by 205
  8. 2017Image De-raining Using a Conditional Generative Adversarial Networkcited by 197
  9. 2019Spatial Attentive Single-Image Deraining with a High Quality Real Rain Datasetcited by 192
  10. 2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 181
  11. 2019Adaptively Sparse Transformerscited by 154
  12. 2018Gated Fusion Network for Single Image Dehazingcited by 133
  13. 2020A Model-driven Deep Neural Network for Single Image Rain Removalcited by 125
  14. 2022Accurate Image Restoration with Attention Retractable Transformercited by 85
  15. 2019Single Image Deraining: From Model-Based to Data-Driven and Beyondcited by 77
  16. 2021KVT: k-NN Attention for Boosting Vision Transformerscited by 64
  17. 2021Structure-Preserving Deraining with Residue Channel Prior Guidancecited by 63
  18. 2020Syn2Real Transfer Learning for Image Deraining using Gaussian Processescited by 62
  19. 2019Explicit Sparse Transformer: Concentrated Attention Through Explicit Selectioncited by 61
  20. 2021Unpaired Deep Image Deraining Using Dual Contrastive Learningcited by 60
  21. 2022NFormer: Robust Person Re-identification with Neighbor Transformercited by 57
  22. 2020DCSFN: Deep Cross-scale Fusion Network for Single Image Rain Removalcited by 44
  23. 2021Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translationcited by 36
  24. 2022SparseTT: Visual Tracking with Sparse Transformerscited by 36
  25. 2022DRT: A Lightweight Single Image Deraining Recursive Transformercited by 26
  26. 2018Attention-based Adaptive Selection of Operations for Image Restoration in the Presence of Unknown Combined Distortionscited by 18
  27. 2022Magic ELF: Image Deraining Meets Association Learning and Transformercited by 17
  28. 2020Restoring Spatially-Heterogeneous Distortions using Mixture of Experts Networkcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 19; 2024: 68; 2025: 72; 2026: 45.

204 papers.

Sort:

By area:

  1. 2024OneRestore: A Universal Restoration Framework for Composite Degradationcited by 91
  2. 2024Selective Hourglass Mapping for Universal Image Restoration Based on Diffusion Modelcited by 73
  3. 2024Restoring Images in Adverse Weather Conditions via Histogram Transformercited by 73
  4. 2024DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curationcited by 64
  5. 2024I2EBench: A Comprehensive Benchmark for Instruction-based Image Editingcited by 61
  6. 2024SCTransNet: Spatial-channel Cross Transformer Network for Infrared Small Target Detectioncited by 60
  7. 2024Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restorationcited by 60
  8. 2024A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trendscited by 58
  9. 2024Bidirectional Multi-Scale Implicit Neural Representations for Image Derainingcited by 54
  10. 2024Frequency-Assisted Mamba for Remote Sensing Image Super-Resolutioncited by 47
  11. 2025Pixel to Gaussian: Ultra-Fast Continuous Super-Resolution with 2D Gaussian Modelingcited by 47
  12. 2023Sparse Sampling Transformer with Uncertainty-Driven Ranking for Unified Removal of Raindrops and Rain Streakscited by 45
38 more
  1. 2023Prompt-In-Prompt Learning for Universal Image Restorationcited by 43
  2. 2024RestoreAgent: Autonomous Image Restoration Agent via Multimodal Large Language Modelscited by 41
  3. 2023From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removalcited by 39
  4. 2024Scene Adaptive Sparse Transformer for Event-based Object Detectioncited by 39
  5. 2024FoundIR: Unleashing Million-scale Training Data to Advance Foundation Models for Image Restorationcited by 37
  6. 2023Language-driven All-in-one Adverse Weather Removalcited by 36
  7. 2023Improving Image Restoration through Removing Degradations in Textual Representationscited by 36
  8. 2023Towards Unified Deep Image Deraining: A Survey and A New Benchmarkcited by 34
  9. 2025JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restorationcited by 32
  10. 2024Frequency Decoupling for Motion Magnification via Multi-Level Isomorphic Architecturecited by 28
  11. 2023Textual Prompt Guided Image Restorationcited by 27
  12. 2024Efficient Visual State Space Model for Image Deblurringcited by 27
  13. 2025Directing Mamba to Complex Textures: An Efficient Texture-Aware State Space Model for Image Restorationcited by 27
  14. 2025Visual-Instructed Degradation Diffusion for All-in-One Image Restorationcited by 26
  15. 2024Learning Enriched Features via Selective State Spaces Model for Efficient Image Deblurringcited by 25
  16. 2025Progressive Focused Transformer for Single Image Super-Resolutioncited by 25
  17. 2024CU-Mamba: Selective State Space Models with Channel Learning for Image Restorationcited by 24
  18. 2024Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraintcited by 24
  19. 2023Prompt-based Ingredient-Oriented All-in-One Image Restorationcited by 22
  20. 2025From Zero to Detail: Deconstructing Ultra-High-Definition Image Restoration from Progressive Spectral Perspectivecited by 22
  21. 2024Degradation-Aware Residual-Conditioned Optimal Transport for Unified Image Restorationcited by 21
  22. 2025WeatherBench: A Real-World Benchmark Dataset for All-in-One Adverse Weather Image Restorationcited by 20
  23. 2024NightRain: Nighttime Video Deraining via Adaptive-Rain-Removal and Adaptive-Correctioncited by 19
  24. 2024DeRainGS: Gaussian Splatting for Enhanced Scene Reconstruction in Rainy Environmentscited by 19
  25. 2024Dropout the High-rate Downsampling: A Novel Design Paradigm for UHD Image Restorationcited by 19
  26. 2023Mutual Information-driven Triple Interaction Network for Efficient Image Dehazingcited by 18
  27. 2024Seeing the Unseen: A Frequency Prompt Guided Transformer for Image Restorationcited by 18
  28. 2024LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restorationcited by 18
  29. 2024Pick of the Bunch: Detecting Infrared Small Targets Beyond Hit-Miss Trade-Offs via Selective Rank-Aware Attentioncited by 17
  30. 2024Learning A Spiking Neural Network for Efficient Image Derainingcited by 16
  31. 2024UAV-Rain1k: A Benchmark for Raindrop Removal from UAV Aerial Imagerycited by 15
  32. 2024Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restorationcited by 15
  33. 2024Multi-Scale Representation Learning for Image Restoration with State-Space Modelcited by 15
  34. 2023Networks are Slacking Off: Understanding Generalization Problem in Image Derainingcited by 14
  35. 2023Multi-task Image Restoration Guided By Robust DINO Featurescited by 14
  36. 2024All-weather Multi-Modality Image Fusion: Unified Framework and 100k Benchmarkcited by 14
  37. 2024RSDehamba: Lightweight Vision Mamba for Remote Sensing Satellite Image Dehazingcited by 14
  38. 2024Sharing Key Semantics in Transformer Makes Efficient Image Restorationcited by 14

50 most cited of 204 All 204 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Learning A Coarse-to-Fine Diffusion Transformer for Image Restoration
  2. 2025RestoRect: Degraded Image Restoration via Latent Rectified Flow & Feature Distillation
  3. 2026Edge-Efficient Image Restoration: Transformer Distillation into State-Space Models
  4. 2024IPT-V2: Efficient Image Processing Transformer using Hierarchical Attentions
  5. 2026In-Token Learning for High-Fidelity Image Restoration via Diffusion Transformers
  6. 2024CLEAR: Conv-Like Linearization Revs Pre-Trained Diffusion Transformers Up
  7. 2025ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image Restoration
  8. 2025DeblurDiff: Real-World Image Deblurring with Generative Diffusion Models