# Image Restoration by Denoising Diffusion Models with Iteratively Preconditioned Guidance

arXiv: https://arxiv.org/abs/2312.16519 · HTML: https://mlsanity.com/paper/2312.16519/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 38 of its 47 references that are in the map.
- First posted on arXiv 27 Dec 2023.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 30 of 38)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Solving Inverse Problems in Medical Imaging with Score-Based Generative Models](https://mlsanity.com/paper/2111.08005/) (2021)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Denoising Diffusion Models for Plug-and-Play Image Restoration](https://mlsanity.com/paper/2305.08995/) (2023)
- [The Little Engine that Could: Regularization by Denoising (RED)](https://mlsanity.com/paper/1611.02862/) (2016)
- ["Zero-Shot" Super-Resolution using Deep Internal Learning](https://mlsanity.com/paper/1712.06087/) (2017)
- [Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applications](https://mlsanity.com/paper/1605.01710/) (2016)
- [A Variational Perspective on Solving Inverse Problems with Diffusion Models](https://mlsanity.com/paper/2305.04391/) (2023)
- [Learning a Convolutional Neural Network for Non-uniform Motion Blur Removal](https://mlsanity.com/paper/1503.00593/) (2015)
- [Regularization by Denoising: Clarifications and New Interpretations](https://mlsanity.com/paper/1806.02296/) (2018)
- [Plug-and-Play Methods for Integrating Physical and Learned Models in Computational Imaging](https://mlsanity.com/paper/2203.17061/) (2022)
- [Image Denoising: The Deep Learning Revolution and Beyond -- A Survey Paper --](https://mlsanity.com/paper/2301.03362/) (2023)
- [An Online Plug-and-Play Algorithm for Regularized Image Reconstruction](https://mlsanity.com/paper/1809.04693/) (2018)
- [Correction Filter for Single Image Super-Resolution: Robustifying Off-the-Shelf Deep Super-Resolvers](https://mlsanity.com/paper/1912.00157/) (2019)

## Built on by (most cited first, 30 of 43)

- [PnP-Flow: Plug-and-Play Image Restoration with Flow Matching](https://mlsanity.com/paper/2410.02423/) (2024)
- [Gradient Guidance for Diffusion Models: An Optimization Perspective](https://mlsanity.com/paper/2404.14743/) (2024)
- [Zero-Shot Image Restoration Using Few-Step Guidance of Consistency Models (and Beyond)](https://mlsanity.com/paper/2412.20596/) (2024)
- [Noise-Inspired Diffusion Model for Generalizable Low-Dose CT Reconstruction](https://mlsanity.com/paper/2506.22012/) (2025)
- [Fast RoPE Attention: Combining the Polynomial Method and Fast Fourier Transform](https://mlsanity.com/paper/2505.11892/) (2025)
- [Inverse Problem Sampling in Latent Space Using Sequential Monte Carlo](https://mlsanity.com/paper/2502.05908/) (2025)
- [ZeroSmooth: Training-free Diffuser Adaptation for High Frame Rate Video Generation](https://mlsanity.com/paper/2406.00908/) (2024)
- [BlurDM: A Blur Diffusion Model for Image Deblurring](https://mlsanity.com/paper/2512.03979/) (2025)
- [System-Embedded Diffusion Bridge Models](https://mlsanity.com/paper/2506.23726/) (2025)
- [Masked Pre-training Enables Universal Zero-shot Denoiser](https://mlsanity.com/paper/2401.14966/) (2024)
- [Robo-SGG: Exploiting Layout-Oriented Normalization and Restitution Can Improve Robust Scene Graph Generation](https://mlsanity.com/paper/2504.12606/) (2025)
- [AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models](https://mlsanity.com/paper/2510.20348/) (2025)
- [Local Epistemic Uncertainty Guided Active Sampling for Plug-and-play Diffusive Image Restoration](https://mlsanity.com/paper/2608.06981/) (2026)
- [Robust Simultaneous Multislice MRI Reconstruction Using Slice-Wise Learned Generative Diffusion Priors](https://mlsanity.com/paper/2407.21600/) (2024)
- [Exploring Multimodal Foundation AI and Expert-in-the-Loop for Sustainable Management of Wild Salmon Fisheries in Indigenous Rivers](https://mlsanity.com/paper/2505.06637/) (2025)
- [Physics-Guided Image Dehazing Diffusion](https://mlsanity.com/paper/2504.21385/) (2025)
- [Self-Cascaded Diffusion Models for Arbitrary-Scale Image Super-Resolution](https://mlsanity.com/paper/2506.07813/) (2025)
- [Equivariant Sampling for Improving Diffusion Model-based Image Restoration](https://mlsanity.com/paper/2511.09965/) (2025)
- [A Diffusion-Based Generative Prior Approach to Sparse-view Computed Tomography](https://mlsanity.com/paper/2602.10722/) (2026)
- [FAST-DIPS: Adjoint-Free Analytic Steps and Hard-Constrained Likelihood Correction for Diffusion-Prior Inverse Problems](https://mlsanity.com/paper/2603.01591/) (2026)
- [PASDiff: Physics-Aware Semantic Guidance for Joint Real-World Low-Light Face Enhancement and Restoration](https://mlsanity.com/paper/2603.24969/) (2026)
- [A Noise Constrained Diffusion (NC-Diffusion) Framework for High Fidelity Image Compression](https://mlsanity.com/paper/2604.06568/) (2026)
- [Local Intrinsic Dimension Unveils Hallucinations in Diffusion Models](https://mlsanity.com/paper/2605.05026/) (2026)
- [HNDiff: Haze-Noise Diffusion for Image Dehazing](https://mlsanity.com/paper/2608.10995/) (2026)
- [Sewer Image Super-Resolution with Depth Priors and Its Lightweight Network](https://mlsanity.com/paper/2407.19271/) (2024)
- [RELD: Regularization by Latent Diffusion Models for Image Restoration](https://mlsanity.com/paper/2503.22563/) (2025)
- [A review of advancements in low-light image enhancement using deep learning](https://mlsanity.com/paper/2505.05759/) (2025)
- [Zero-Shot Solving of Imaging Inverse Problems via Noise-Refined Likelihood Guided Diffusion Models](https://mlsanity.com/paper/2506.13391/) (2025)
- [Plug-and-play Diffusion Models for Image Compressive Sensing with Data Consistency Projection](https://mlsanity.com/paper/2509.09365/) (2025)
- [Progressive Flow-inspired Unfolding for Spectral Compressive Imaging](https://mlsanity.com/paper/2509.12079/) (2025)

## Abstract (arXiv)

Training deep neural networks has become a common approach for addressing image restoration problems. An alternative for training a "task-specific" network for each observation model is to use pretrained deep denoisers for imposing only the signal's prior within iterative algorithms, without additional training. Recently, a sampling-based variant of this approach has become popular with the rise of diffusion/score-based generative models. Using denoisers for general purpose restoration requires guiding the iterations to ensure agreement of the signal with the observations. In low-noise settings, guidance that is based on back-projection (BP) has been shown to be a promising strategy (used recently also under the names "pseudoinverse" or "range/null-space" guidance). However, the presence of noise in the observations hinders the gains from this approach. In this paper, we propose a novel guidance technique, based on preconditioning that allows traversing from BP-based guidance to least squares based guidance along the restoration scheme. The proposed approach is robust to noise while still having much simpler implementation than alternative methods (e.g., it does not require SVD or a large number of iterations). We use it within both an optimization scheme and a sampling-based scheme, and demonstrate its advantages over existing methods for image deblurring and super-resolution.