# Posterior Sampling for Image Restoration using Explicit Patch Priors

arXiv: https://arxiv.org/abs/2104.09895 · HTML: https://mlsanity.com/paper/2104.09895/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 20 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- ["Zero-Shot" Super-Resolution using Deep Internal Learning](https://mlsanity.com/paper/1712.06087/) (2017)
- [Benchmarking Denoising Algorithms with Real Photographs](https://mlsanity.com/paper/1707.01313/) (2017)
- [SRFlow: Learning the Super-Resolution Space with Normalizing Flow](https://mlsanity.com/paper/2006.14200/) (2020)
- [Non-Local Recurrent Network for Image Restoration](https://mlsanity.com/paper/1806.02919/) (2018)
- [Neural Nearest Neighbors Networks](https://mlsanity.com/paper/1810.12575/) (2018)
- [Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser](https://mlsanity.com/paper/2007.13640/) (2020)
- [A Bayesian Perspective on the Deep Image Prior](https://mlsanity.com/paper/1904.07457/) (2019)
- [Stochastic Image Denoising by Sampling from the Posterior Distribution](https://mlsanity.com/paper/2101.09552/) (2021)
- [Explorable Super Resolution](https://mlsanity.com/paper/1912.01839/) (2019)
- [Structured Output Learning with Conditional Generative Flows](https://mlsanity.com/paper/1905.13288/) (2019)
- [Deblurring using Analysis-Synthesis Networks Pair](https://mlsanity.com/paper/2004.02956/) (2020)
- [DeepSEE: Deep Disentangled Semantic Explorative Extreme Super-Resolution](https://mlsanity.com/paper/2004.04433/) (2020)

## Built on by (most cited first, 5 of 5)

- [A Theory of the Distortion-Perception Tradeoff in Wasserstein Space](https://mlsanity.com/paper/2107.02555/) (2021)
- [Towards a Mechanistic Explanation of Diffusion Model Generalization](https://mlsanity.com/paper/2411.19339/) (2024)
- [PatchNR: Learning from Very Few Images by Patch Normalizing Flow Regularization](https://mlsanity.com/paper/2205.12021/) (2022)
- [WPPNets and WPPFlows: The Power of Wasserstein Patch Priors for Superresolution](https://mlsanity.com/paper/2201.08157/) (2022)
- [Learning from small data sets: Patch-based regularizers in inverse problems for image reconstruction](https://mlsanity.com/paper/2312.16611/) (2023)

## Abstract (arXiv)

Almost all existing methods for image restoration are based on optimizing the mean squared error (MSE), even though it is known that the best estimate in terms of MSE may yield a highly atypical image due to the fact that there are many plausible restorations for a given noisy image. In this paper, we show how to combine explicit priors on patches of natural images in order to sample from the posterior probability of a full image given a degraded image. We prove that our algorithm generates correct samples from the distribution p(x|y) propto exp(-E(x|y)) where E(x|y) is the cost function minimized in previous patch-based approaches that compute a single restoration. Unlike previous approaches that computed a single restoration using MAP or MMSE, our method makes explicit the uncertainty in the restored images and guarantees that all patches in the restored images will be typical given the patch prior. Unlike previous approaches that used implicit priors on fixed-size images, our approach can be used with images of any size. Our experimental results show that posterior sampling using patch priors yields images of high perceptual quality and high PSNR on a range of challenging image restoration problems.