# Generative Diffusion Prior for Unified Image Restoration and Enhancement

arXiv: https://arxiv.org/abs/2304.01247 · HTML: https://mlsanity.com/paper/2304.01247/

## Key facts

- Cited by 199 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 80 of its 108 references that are in the map.
- First posted on arXiv 3 Apr 2023.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 30 of 80)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)

## Built on by (most cited first, 30 of 199)

- [DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2308.15070/) (2023)
- [Diffusion Model-Based Image Editing: A Survey](https://mlsanity.com/paper/2402.17525/) (2024)
- [Low-Light Image Enhancement with Wavelet-based Diffusion Models](https://mlsanity.com/paper/2306.00306/) (2023)
- [Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2308.09388/) (2023)
- [AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion](https://mlsanity.com/paper/2310.10123/) (2023)
- [Arbitrary-steps Image Super-resolution via Diffusion Inversion](https://mlsanity.com/paper/2412.09013/) (2024)
- [LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Models](https://mlsanity.com/paper/2407.08939/) (2024)
- [Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model](https://mlsanity.com/paper/2311.11638/) (2023)
- [CoSeR: Bridging Image and Language for Cognitive Super-Resolution](https://mlsanity.com/paper/2311.16512/) (2023)
- [DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curation](https://mlsanity.com/paper/2410.18666/) (2024)
- [Diffusion Models, Image Super-Resolution And Everything: A Survey](https://mlsanity.com/paper/2401.00736/) (2024)
- [A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends](https://mlsanity.com/paper/2410.15067/) (2024)
- [Motion-Guided Latent Diffusion for Temporally Consistent Real-world Video Super-resolution](https://mlsanity.com/paper/2312.00853/) (2023)
- [Boosting Diffusion Models with Moving Average Sampling in Frequency Domain](https://mlsanity.com/paper/2403.17870/) (2024)
- [PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance](https://mlsanity.com/paper/2309.10810/) (2023)
- [Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Tasks](https://mlsanity.com/paper/2403.00644/) (2024)
- [XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution](https://mlsanity.com/paper/2403.05049/) (2024)
- [HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models](https://mlsanity.com/paper/2402.15865/) (2024)
- [Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving](https://mlsanity.com/paper/2404.04804/) (2024)
- [DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Models](https://mlsanity.com/paper/2407.01519/) (2024)
- [OSDFace: One-Step Diffusion Model for Face Restoration](https://mlsanity.com/paper/2411.17163/) (2024)
- [Text-DiFuse: An Interactive Multi-Modal Image Fusion Framework based on Text-modulated Diffusion Model](https://mlsanity.com/paper/2410.23905/) (2024)
- [Blind Image Restoration via Fast Diffusion Inversion](https://mlsanity.com/paper/2405.19572/) (2024)
- [Transfer CLIP for Generalizable Image Denoising](https://mlsanity.com/paper/2403.15132/) (2024)
- [Conditional Image Synthesis with Diffusion Models: A Survey](https://mlsanity.com/paper/2409.19365/) (2024)
- [Diff-Privacy: Diffusion-based Face Privacy Protection](https://mlsanity.com/paper/2309.05330/) (2023)
- [Diffusion-based Blind Text Image Super-Resolution](https://mlsanity.com/paper/2312.08886/) (2023)
- [Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution](https://mlsanity.com/paper/2403.17000/) (2024)
- [UnfoldIR: Rethinking Deep Unfolding Network in Illumination Degradation Image Restoration](https://mlsanity.com/paper/2505.06683/) (2025)
- [Motion-adaptive Separable Collaborative Filters for Blind Motion Deblurring](https://mlsanity.com/paper/2404.13153/) (2024)

## Abstract (arXiv)

Existing image restoration methods mostly leverage the posterior distribution of natural images. However, they often assume known degradation and also require supervised training, which restricts their adaptation to complex real applications. In this work, we propose the Generative Diffusion Prior (GDP) to effectively model the posterior distributions in an unsupervised sampling manner. GDP utilizes a pre-train denoising diffusion generative model (DDPM) for solving linear inverse, non-linear, or blind problems. Specifically, GDP systematically explores a protocol of conditional guidance, which is verified more practical than the commonly used guidance way. Furthermore, GDP is strength at optimizing the parameters of degradation model during the denoising process, achieving blind image restoration. Besides, we devise hierarchical guidance and patch-based methods, enabling the GDP to generate images of arbitrary resolutions. Experimentally, we demonstrate GDP's versatility on several image datasets for linear problems, such as super-resolution, deblurring, inpainting, and colorization, as well as non-linear and blind issues, such as low-light enhancement and HDR image recovery. GDP outperforms the current leading unsupervised methods on the diverse benchmarks in reconstruction quality and perceptual quality. Moreover, GDP also generalizes well for natural images or synthesized images with arbitrary sizes from various tasks out of the distribution of the ImageNet training set.