# Joint Conditional Diffusion Model for Image Restoration with Mixed Degradations

arXiv: https://arxiv.org/abs/2404.07770 · HTML: https://mlsanity.com/paper/2404.07770/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 11 Apr 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 30 of 30)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [FFA-Net: Feature Fusion Attention Network for Single Image Dehazing](https://mlsanity.com/paper/1911.07559/) (2019)
- [Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction](https://mlsanity.com/paper/2112.05146/) (2021)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [Vision Transformers for Single Image Dehazing](https://mlsanity.com/paper/2204.03883/) (2022)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models](https://mlsanity.com/paper/2207.14626/) (2022)
- [Attentive Generative Adversarial Network for Raindrop Removal from a Single Image](https://mlsanity.com/paper/1711.10098/) (2017)
- [DesnowNet: Context-Aware Deep Network for Snow Removal](https://mlsanity.com/paper/1708.04512/) (2017)
- [NH-HAZE: An Image Dehazing Benchmark with Non-Homogeneous Hazy and Haze-Free Images](https://mlsanity.com/paper/2005.03560/) (2020)
- [Dense Haze: A benchmark for image dehazing with dense-haze and haze-free images](https://mlsanity.com/paper/1904.02904/) (2019)
- [Deep Dense Multi-scale Network for Snow Removal Using Semantic and Geometric Priors](https://mlsanity.com/paper/2103.11298/) (2021)
- [Spatially-Adaptive Image Restoration using Distortion-Guided Networks](https://mlsanity.com/paper/2108.08617/) (2021)
- [EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining](https://mlsanity.com/paper/2009.09238/) (2020)
- [Blind Image Decomposition](https://mlsanity.com/paper/2108.11364/) (2021)
- [RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining](https://mlsanity.com/paper/2107.06808/) (2021)
- [High-quality Image Dehazing with Diffusion Model](https://mlsanity.com/paper/2308.11949/) (2023)
- [LMQFormer: A Laplace-Prior-Guided Mask Query Transformer for Lightweight Snow Removal](https://mlsanity.com/paper/2210.04787/) (2022)
- [Rethinking Real-world Image Deraining via An Unpaired Degradation-Conditioned Diffusion Model](https://mlsanity.com/paper/2301.09430/) (2023)

## Built on by (most cited first, 1 of 1)

- [Diffusion Once and Done: Degradation-Aware LoRA for Efficient All-in-One Image Restoration](https://mlsanity.com/paper/2508.03373/) (2025)

## Abstract (arXiv)

Image restoration is rather challenging in adverse weather conditions, especially when multiple degradations occur simultaneously. Blind image decomposition was proposed to tackle this issue, however, its effectiveness heavily relies on the accurate estimation of each component. Although diffusion-based models exhibit strong generative abilities in image restoration tasks, they may generate irrelevant contents when the degraded images are severely corrupted. To address these issues, we leverage physical constraints to guide the whole restoration process, where a mixed degradation model based on atmosphere scattering model is constructed. Then we formulate our Joint Conditional Diffusion Model (JCDM) by incorporating the degraded image and degradation mask to provide precise guidance. To achieve better color and detail recovery results, we further integrate a refinement network to reconstruct the restored image, where Uncertainty Estimation Block (UEB) is employed to enhance the features. Extensive experiments performed on both multi-weather and weather-specific datasets demonstrate the superiority of our method over state-of-the-art competing methods.