# Restoration by Generation with Constrained Priors arXiv: https://arxiv.org/abs/2312.17161 ยท HTML: https://mlsanity.com/paper/2312.17161/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. - Builds on 51 of its 58 references that are in the map. - First posted on arXiv 28 Dec 2023. ## Placed in - Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/) ## Builds on (most cited first, 30 of 51) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021) - [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018) - [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021) - [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022) - [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022) - [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016) - [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021) - [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021) - [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021) - [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022) - [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016) - [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021) - [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022) - [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019) - [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021) - [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021) - [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022) - [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021) - [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021) - [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022) - [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017) - [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020) - [Universal Guidance for Diffusion Models](https://mlsanity.com/paper/2302.07121/) (2023) - [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017) - [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017) - [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019) - [Towards Real-World Blind Face Restoration with Generative Facial Prior](https://mlsanity.com/paper/2101.04061/) (2021) - [InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning](https://mlsanity.com/paper/2304.03411/) (2023) ## Built on by (most cited first, 9 of 9) - [Visual-Instructed Degradation Diffusion for All-in-One Image Restoration](https://mlsanity.com/paper/2506.16960/) (2025) - [InstantRestore: Single-Step Personalized Face Restoration with Shared-Image Attention](https://mlsanity.com/paper/2412.06753/) (2024) - [TPGDiff: Hierarchical Triple-Prior Guided Diffusion for Image Restoration](https://mlsanity.com/paper/2601.20306/) (2026) - [Towards Unsupervised Blind Face Restoration using Diffusion Prior](https://mlsanity.com/paper/2410.04618/) (2024) - [Personalized Generative Low-light Image Denoising and Enhancement](https://mlsanity.com/paper/2412.14327/) (2024) - [HonestFace: Towards Honest Face Restoration with One-Step Diffusion Model](https://mlsanity.com/paper/2505.18469/) (2025) - [Human Multi-View Synthesis from a Single-View Model:Transferred Body and Face Representations](https://mlsanity.com/paper/2412.03011/) (2024) - [Face2Scene: Using Facial Degradation as an Oracle for Diffusion-Based Scene Restoration](https://mlsanity.com/paper/2603.16570/) (2026) - [Beyond Fixed Inference: Quantitative Flow Matching for Adaptive Image Denoising](https://mlsanity.com/paper/2604.02392/) (2026) ## Abstract (arXiv) The inherent generative power of denoising diffusion models makes them well-suited for image restoration tasks where the objective is to find the optimal high-quality image within the generative space that closely resembles the input image. We propose a method to adapt a pretrained diffusion model for image restoration by simply adding noise to the input image to be restored and then denoise. Our method is based on the observation that the space of a generative model needs to be constrained. We impose this constraint by finetuning the generative model with a set of anchor images that capture the characteristics of the input image. With the constrained space, we can then leverage the sampling strategy used for generation to do image restoration. We evaluate against previous methods and show superior performances on multiple real-world restoration datasets in preserving identity and image quality. We also demonstrate an important and practical application on personalized restoration, where we use a personal album as the anchor images to constrain the generative space. This approach allows us to produce results that accurately preserve high-frequency details, which previous works are unable to do. Project webpage: https://gen2res.github.io.