# Sparse Sampling Transformer with Uncertainty-Driven Ranking for Unified Removal of Raindrops and Rain Streaks

arXiv: https://arxiv.org/abs/2308.14153 · HTML: https://mlsanity.com/paper/2308.14153/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 76 references that are in the map.
- First posted on arXiv 27 Aug 2023.

## Builds on (most cited first, 30 of 49)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [Vision Transformers for Single Image Dehazing](https://mlsanity.com/paper/2204.03883/) (2022)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [HINet: Half Instance Normalization Network for Image Restoration](https://mlsanity.com/paper/2105.06086/) (2021)
- [Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining](https://mlsanity.com/paper/1807.05698/) (2018)
- [Learning A Sparse Transformer Network for Effective Image Deraining](https://mlsanity.com/paper/2303.11950/) (2023)
- [Attentive Generative Adversarial Network for Raindrop Removal from a Single Image](https://mlsanity.com/paper/1711.10098/) (2017)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration](https://mlsanity.com/paper/2007.12142/) (2020)
- [Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning](https://mlsanity.com/paper/1904.05050/) (2019)
- [Data Uncertainty Learning in Face Recognition](https://mlsanity.com/paper/2003.11339/) (2020)
- [Deep Generalized Unfolding Networks for Image Restoration](https://mlsanity.com/paper/2204.13348/) (2022)
- [Uncertainty Guided Multi-Scale Residual Learning-using a Cycle Spinning CNN for Single Image De-Raining](https://mlsanity.com/paper/1906.11129/) (2019)
- [Accurate Image Restoration with Attention Retractable Transformer](https://mlsanity.com/paper/2210.01427/) (2022)

## Built on by (most cited first, 30 of 45)

- [Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model](https://mlsanity.com/paper/2311.11638/) (2023)
- [Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining](https://mlsanity.com/paper/2404.01547/) (2024)
- [RestoreAgent: Autonomous Image Restoration Agent via Multimodal Large Language Models](https://mlsanity.com/paper/2407.18035/) (2024)
- [POSTA: A Go-to Framework for Customized Artistic Poster Generation](https://mlsanity.com/paper/2503.14908/) (2025)
- [JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restoration](https://mlsanity.com/paper/2504.04158/) (2025)
- [Visual-Instructed Degradation Diffusion for All-in-One Image Restoration](https://mlsanity.com/paper/2506.16960/) (2025)
- [Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraint](https://mlsanity.com/paper/2409.15739/) (2024)
- [NTIRE 2025 Challenge on Day and Night Raindrop Removal for Dual-Focused Images: Methods and Results](https://mlsanity.com/paper/2504.12711/) (2025)
- [DeRainGS: Gaussian Splatting for Enhanced Scene Reconstruction in Rainy Environments](https://mlsanity.com/paper/2408.11540/) (2024)
- [Dropout the High-rate Downsampling: A Novel Design Paradigm for UHD Image Restoration](https://mlsanity.com/paper/2411.06456/) (2024)
- [VersaT2I: Improving Text-to-Image Models with Versatile Reward](https://mlsanity.com/paper/2403.18493/) (2024)
- [Raindrop Clarity: A Dual-Focused Dataset for Day and Night Raindrop Removal](https://mlsanity.com/paper/2407.16957/) (2024)
- [Integrating View Conditions for Image Synthesis](https://mlsanity.com/paper/2310.16002/) (2023)
- [AGLLDiff: Guiding Diffusion Models Towards Unsupervised Training-free Real-world Low-light Image Enhancement](https://mlsanity.com/paper/2407.14900/) (2024)
- [VQCNIR: Clearer Night Image Restoration with Vector-Quantized Codebook](https://mlsanity.com/paper/2312.08606/) (2023)
- [All-in-one Weather-degraded Image Restoration via Adaptive Degradation-aware Self-prompting Model](https://mlsanity.com/paper/2411.07445/) (2024)
- [NiteDR: Nighttime Image De-Raining with Cross-View Sensor Cooperative Learning for Dynamic Driving Scenes](https://mlsanity.com/paper/2402.18172/) (2024)
- [RainyScape: Unsupervised Rainy Scene Reconstruction using Decoupled Neural Rendering](https://mlsanity.com/paper/2404.11401/) (2024)
- [Scan Clusters, Not Pixels: A Cluster-Centric Paradigm for Efficient Ultra-high-definition Image Restoration](https://mlsanity.com/paper/2602.21917/) (2026)
- [Image Deraining with Frequency-Enhanced State Space Model](https://mlsanity.com/paper/2405.16470/) (2024)
- [3D Smoke Scene Reconstruction Guided by Vision Priors from Multimodal Large Language Models](https://mlsanity.com/paper/2604.05687/) (2026)
- [A Hybrid Transformer-Mamba Network for Single Image Deraining](https://mlsanity.com/paper/2409.00410/) (2024)
- [Rethinking Nighttime Image Deraining via Learnable Color Space Transformation](https://mlsanity.com/paper/2510.17440/) (2025)
- [Unsupervised Low-light Image Enhancement with Lookup Tables and Diffusion Priors](https://mlsanity.com/paper/2409.18899/) (2024)
- [Seeing Through the Rain: Resolving High-Frequency Conflicts in Deraining and Super-Resolution via Diffusion Guidance](https://mlsanity.com/paper/2511.12419/) (2025)
- [TPGDiff: Hierarchical Triple-Prior Guided Diffusion for Image Restoration](https://mlsanity.com/paper/2601.20306/) (2026)
- [Instruct2See: Learning to Remove Any Obstructions Across Distributions](https://mlsanity.com/paper/2505.17649/) (2025)
- [Global Modeling Matters: A Fast, Lightweight and Effective Baseline for Efficient Image Restoration](https://mlsanity.com/paper/2507.13663/) (2025)
- [RetinexDual: Retinex-based Dual Nature Approach for Generalized Ultra-High-Definition Image Restoration](https://mlsanity.com/paper/2508.04797/) (2025)
- [Multi-dimension Transformer with Attention-based Filtering for Medical Image Segmentation](https://mlsanity.com/paper/2405.12328/) (2024)

## Abstract (arXiv)

In the real world, image degradations caused by rain often exhibit a combination of rain streaks and raindrops, thereby increasing the challenges of recovering the underlying clean image. Note that the rain streaks and raindrops have diverse shapes, sizes, and locations in the captured image, and thus modeling the correlation relationship between irregular degradations caused by rain artifacts is a necessary prerequisite for image deraining. This paper aims to present an efficient and flexible mechanism to learn and model degradation relationships in a global view, thereby achieving a unified removal of intricate rain scenes. To do so, we propose a Sparse Sampling Transformer based on Uncertainty-Driven Ranking, dubbed UDR-S2Former. Compared to previous methods, our UDR-S2Former has three merits. First, it can adaptively sample relevant image degradation information to model underlying degradation relationships. Second, explicit application of the uncertainty-driven ranking strategy can facilitate the network to attend to degradation features and understand the reconstruction process. Finally, experimental results show that our UDR-S2Former clearly outperforms state-of-the-art methods for all benchmarks.