# Spectral-Structured Diffusion for Single-Image Rain Removal

arXiv: https://arxiv.org/abs/2603.09054 · HTML: https://mlsanity.com/paper/2603.09054/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 62 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Builds on (most cited first, 26 of 26)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [HINet: Half Instance Normalization Network for Image Restoration](https://mlsanity.com/paper/2105.06086/) (2021)
- [Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models](https://mlsanity.com/paper/2207.14626/) (2022)
- [Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining](https://mlsanity.com/paper/1807.05698/) (2018)
- [Learning A Sparse Transformer Network for Effective Image Deraining](https://mlsanity.com/paper/2303.11950/) (2023)
- [Image De-raining Using a Conditional Generative Adversarial Network](https://mlsanity.com/paper/1701.05957/) (2017)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning](https://mlsanity.com/paper/1904.05050/) (2019)
- [Clearing the Skies: A deep network architecture for single-image rain removal](https://mlsanity.com/paper/1609.02087/) (2016)
- [A Model-driven Deep Neural Network for Single Image Rain Removal](https://mlsanity.com/paper/2005.01333/) (2020)
- [Fast Algorithms for Convolutional Neural Networks](https://mlsanity.com/paper/1509.09308/) (2015)
- [Fast Training of Convolutional Networks through FFTs](https://mlsanity.com/paper/1312.5851/) (2013)
- [Spectral Representations for Convolutional Neural Networks](https://mlsanity.com/paper/1506.03767/) (2015)
- [DCSFN: Deep Cross-scale Fusion Network for Single Image Rain Removal](https://mlsanity.com/paper/2008.00767/) (2020)
- [From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removal](https://mlsanity.com/paper/2308.03867/) (2023)
- [Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining](https://mlsanity.com/paper/1808.01491/) (2018)
- [Fast Convolutional Nets With fbfft: A GPU Performance Evaluation](https://mlsanity.com/paper/1412.7580/) (2014)
- [Joint Sub-bands Learning with Clique Structures for Wavelet Domain Super-Resolution](https://mlsanity.com/paper/1809.04508/) (2018)
- [RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining](https://mlsanity.com/paper/2107.06808/) (2021)
- [FFC-SE: Fast Fourier Convolution for Speech Enhancement](https://mlsanity.com/paper/2204.03042/) (2022)
- [Dual-Path Multi-Scale Transformer for High-Quality Image Deraining](https://mlsanity.com/paper/2405.18124/) (2024)

## Abstract (arXiv)

Rain streaks manifest as directional and frequency-concentrated structures that overlap across multiple scales, making single-image rain removal particularly challenging. While diffusion-based restoration models provide a powerful framework for progressive denoising, standard spatial-domain diffusion does not explicitly account for such structured spectral characteristics. We introduce SpectralDiff, a spectral-structured diffusion-based framework tailored for single-image rain removal. Rather than redefining the diffusion formulation, our method incorporates structured spectral perturbations to guide the progressive suppression of multi-directional rain components. To support this design, we further propose a full-product U-Net architecture that leverages the convolution theorem to replace convolution operations with element-wise product layers, improving computational efficiency while preserving modeling capacity. Extensive experiments on synthetic and real-world benchmarks demonstrate that SpectralDiff achieves competitive rain removal performance with improved model compactness and favorable inference efficiency compared to existing diffusion-based approaches.