# Unpaired Photo-realistic Image Deraining with Energy-informed Diffusion Model arXiv: https://arxiv.org/abs/2407.17193 · HTML: https://mlsanity.com/paper/2407.17193/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. - Builds on 26 of its 50 references that are in the map. - First posted on arXiv 24 Jul 2024. ## Placed in - Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/) ## Builds on (most cited first, 26 of 26) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022) - [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021) - [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021) - [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021) - [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022) - [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021) - [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022) - [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021) - [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019) - [Density-aware Single Image De-raining using a Multi-stream Dense Network](https://mlsanity.com/paper/1802.07412/) (2018) - [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019) - [EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations](https://mlsanity.com/paper/2207.06635/) (2022) - [Diffusion Visual Counterfactual Explanations](https://mlsanity.com/paper/2210.11841/) (2022) - [Dual Contrastive Learning for Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2104.07689/) (2021) - [Unpaired Deep Image Deraining Using Dual Contrastive Learning](https://mlsanity.com/paper/2109.02973/) (2021) - [Unsupervised Attention-guided Image to Image Translation](https://mlsanity.com/paper/1806.02311/) (2018) - [DCSFN: Deep Cross-scale Fusion Network for Single Image Rain Removal](https://mlsanity.com/paper/2008.00767/) (2020) - [Joint Self-Attention and Scale-Aggregation for Self-Calibrated Deraining Network](https://mlsanity.com/paper/2008.02763/) (2020) - [Learn from Unpaired Data for Image Restoration: A Variational Bayes Approach](https://mlsanity.com/paper/2204.10090/) (2022) - [Online-updated High-order Collaborative Networks for Single Image Deraining](https://mlsanity.com/paper/2202.06568/) (2022) - [Unsupervised Image Deraining: Optimization Model Driven Deep CNN](https://mlsanity.com/paper/2203.13699/) (2022) - [From heavy rain removal to detail restoration: A faster and better network](https://mlsanity.com/paper/2205.03553/) (2022) ## Built on by (most cited first, 11 of 11) - [All-in-one Weather-degraded Image Restoration via Adaptive Degradation-aware Self-prompting Model](https://mlsanity.com/paper/2411.07445/) (2024) - [Enhancing Diffusion Model Stability for Image Restoration via Gradient Management](https://mlsanity.com/paper/2507.06656/) (2025) - [Zero-Shot Video Deraining with Video Diffusion Models](https://mlsanity.com/paper/2511.18537/) (2025) - [Learning Adaptive Dynamical Features via Multi-τ Liquid-Mamba for All-in-one Image Restoration](https://mlsanity.com/paper/2606.22801/) (2026) - [Unsupervised Methods for Video Quality Improvement: A Survey of Restoration and Enhancement Techniques](https://mlsanity.com/paper/2507.08375/) (2025) - [Causal-AgentIR: Self-Evolving Causal Memory for Adaptive Image Restoration Agents](https://mlsanity.com/paper/2607.21125/) (2026) - [VLM-based Prompts as the Optimal Assistant for Unpaired Histopathology Virtual Staining](https://mlsanity.com/paper/2504.15545/) (2025) - [Learning to Restore Multi-Degraded Images via Ingredient Decoupling and Task-Aware Path Adaptation](https://mlsanity.com/paper/2511.04920/) (2025) - [From Physical Degradation Models to Task-Aware All-in-One Image Restoration](https://mlsanity.com/paper/2601.10192/) (2026) - [Learning to Restore More: Continual Capability Expansion for Pretrained Image Restoration Models](https://mlsanity.com/paper/2608.30305/) (2026) - [GraLoD: Graphics-Inspired Continuous Level-of-Detail Learning for Image Restoration](https://mlsanity.com/paper/2609.16578/) (2026) ## Abstract (arXiv) Existing unpaired image deraining approaches face challenges in accurately capture the distinguishing characteristics between the rainy and clean domains, resulting in residual degradation and color distortion within the reconstructed images. To this end, we propose an energy-informed diffusion model for unpaired photo-realistic image deraining (UPID-EDM). Initially, we delve into the intricate visual-language priors embedded within the contrastive language-image pre-training model (CLIP), and demonstrate that the CLIP priors aid in the discrimination of rainy and clean images. Furthermore, we introduce a dual-consistent energy function (DEF) that retains the rain-irrelevant characteristics while eliminating the rain-relevant features. This energy function is trained by the non-corresponding rainy and clean images. In addition, we employ the rain-relevance discarding energy function (RDEF) and the rain-irrelevance preserving energy function (RPEF) to direct the reverse sampling procedure of a pre-trained diffusion model, effectively removing the rain streaks while preserving the image contents. Extensive experiments demonstrate that our energy-informed model surpasses the existing unpaired learning approaches in terms of both supervised and no-reference metrics.