# Zero-Shot Image Restoration Using Few-Step Guidance of Consistency Models (and Beyond)

arXiv: https://arxiv.org/abs/2412.20596 · HTML: https://mlsanity.com/paper/2412.20596/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 29 Dec 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Solving Inverse Problems in Medical Imaging with Score-Based Generative Models](https://mlsanity.com/paper/2111.08005/) (2021)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Denoising Diffusion Models for Plug-and-Play Image Restoration](https://mlsanity.com/paper/2305.08995/) (2023)
- [Compressed Sensing using Generative Models](https://mlsanity.com/paper/1703.03208/) (2017)
- ["Zero-Shot" Super-Resolution using Deep Internal Learning](https://mlsanity.com/paper/1712.06087/) (2017)
- [Image Restoration by Denoising Diffusion Models with Iteratively Preconditioned Guidance](https://mlsanity.com/paper/2312.16519/) (2023)
- [Image-Adaptive GAN based Reconstruction](https://mlsanity.com/paper/1906.05284/) (2019)
- [Correction Filter for Single Image Super-Resolution: Robustifying Off-the-Shelf Deep Super-Resolvers](https://mlsanity.com/paper/1912.00157/) (2019)
- [ADIR: Adaptive Diffusion for Image Reconstruction](https://mlsanity.com/paper/2212.03221/) (2022)
- [CoSIGN: Few-Step Guidance of ConSIstency Model to Solve General INverse Problems](https://mlsanity.com/paper/2407.12676/) (2024)
- [Super-Resolution via Image-Adapted Denoising CNNs: Incorporating External and Internal Learning](https://mlsanity.com/paper/1811.12866/) (2018)
- [Back-Projection based Fidelity Term for Ill-Posed Linear Inverse Problems](https://mlsanity.com/paper/1906.06794/) (2019)
- [Deep Internal Learning: Deep Learning from a Single Input](https://mlsanity.com/paper/2312.07425/) (2023)
- [On the Convergence Rate of Projected Gradient Descent for a Back-Projection based Objective](https://mlsanity.com/paper/2005.00959/) (2020)

## Built on by (most cited first, 14 of 14)

- [LATINO-PRO: LAtent consisTency INverse sOlver with PRompt Optimization](https://mlsanity.com/paper/2503.12615/) (2025)
- [Learning few-step posterior samplers by unfolding and distillation of diffusion models](https://mlsanity.com/paper/2507.02686/) (2025)
- [PnP-CM: Consistency Models as Plug-and-Play Priors for Inverse Problems](https://mlsanity.com/paper/2509.22736/) (2025)
- [LVTINO: LAtent Video consisTency INverse sOlver for High Definition Video Restoration](https://mlsanity.com/paper/2510.01339/) (2025)
- [Consistency Regularised Gradient Flows for Inverse Problems](https://mlsanity.com/paper/2605.07907/) (2026)
- [SP3: Spherical Priors for Plug-and-Play Restoration](https://mlsanity.com/paper/2606.16396/) (2026)
- [MACS: Measurement-Aware Consistency Sampling for Inverse Problems](https://mlsanity.com/paper/2510.02208/) (2025)
- [Jacobian-Aware Posterior Sampling for Inverse Problems](https://mlsanity.com/paper/2511.18471/) (2025)
- [Deep unfolding of MCMC kernels: scalable, modular & explainable GANs for high-dimensional posterior sampling](https://mlsanity.com/paper/2602.20758/) (2026)
- [Beyond Fixed Inference: Quantitative Flow Matching for Adaptive Image Denoising](https://mlsanity.com/paper/2604.02392/) (2026)
- [Reducing Diffusion Model Memorization with Higher Order Langevin Dynamics](https://mlsanity.com/paper/2605.19170/) (2026)
- [Consistency Models for Fast MRI Reconstruction Using Regularization by Denoising](https://mlsanity.com/paper/2608.20561/) (2026)
- [BAM! Bayesian Anything Model: a foundation model for generative computational imaging](https://mlsanity.com/paper/2609.39660/) (2026)
- [S3-PnP: Plug-and-Play Denoising for Scientific Imaging with Self-Supervised Score Priors](https://mlsanity.com/paper/2610.18458/) (2026)

## Abstract (arXiv)

In recent years, it has become popular to tackle image restoration tasks with a single pretrained diffusion model (DM) and data-fidelity guidance, instead of training a dedicated deep neural network per task. However, such "zero-shot" restoration schemes currently require many Neural Function Evaluations (NFEs) for performing well, which may be attributed to the many NFEs needed in the original generative functionality of the DMs. Recently, faster variants of DMs have been explored for image generation. These include Consistency Models (CMs), which can generate samples via a couple of NFEs. However, existing works that use guided CMs for restoration still require tens of NFEs or fine-tuning of the model per task that leads to performance drop if the assumptions during the fine-tuning are not accurate. In this paper, we propose a zero-shot restoration scheme that uses CMs and operates well with as little as 4 NFEs. It is based on a wise combination of several ingredients: better initialization, back-projection guidance, and above all a novel noise injection mechanism. We demonstrate the advantages of our approach for image super-resolution, deblurring and inpainting. Interestingly, we show that the usefulness of our noise injection technique goes beyond CMs: it can also mitigate the performance degradation of existing guided DM methods when reducing their NFE count.