# WPPNets and WPPFlows: The Power of Wasserstein Patch Priors for Superresolution

arXiv: https://arxiv.org/abs/2201.08157 · HTML: https://mlsanity.com/paper/2201.08157/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 75 references that are in the map.
- First posted on arXiv 20 Jan 2022.

## Builds on (most cited first, 30 of 43)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Deep Learning for Image Super-resolution: A Survey](https://mlsanity.com/paper/1902.06068/) (2019)
- [The Little Engine that Could: Regularization by Denoising (RED)](https://mlsanity.com/paper/1611.02862/) (2016)
- ["Zero-Shot" Super-Resolution using Deep Internal Learning](https://mlsanity.com/paper/1712.06087/) (2017)
- [Residual Flows for Invertible Generative Modeling](https://mlsanity.com/paper/1906.02735/) (2019)
- [Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applications](https://mlsanity.com/paper/1605.01710/) (2016)
- [Analyzing Inverse Problems with Invertible Neural Networks](https://mlsanity.com/paper/1808.04730/) (2018)
- [SRFlow: Learning the Super-Resolution Space with Normalizing Flow](https://mlsanity.com/paper/2006.14200/) (2020)
- [Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems](https://mlsanity.com/paper/1704.03488/) (2017)
- [CNN-Based Projected Gradient Descent for Consistent Image Reconstruction](https://mlsanity.com/paper/1709.01809/) (2017)
- [Deep Probabilistic Imaging: Uncertainty Quantification and Multi-modal Solution Characterization for Computational Imaging](https://mlsanity.com/paper/2010.14462/) (2020)
- [Computed Tomography Reconstruction Using Deep Image Prior and Learned Reconstruction Methods](https://mlsanity.com/paper/2003.04989/) (2020)
- [Total Deep Variation for Linear Inverse Problems](https://mlsanity.com/paper/2001.05005/) (2020)
- [Convolutional Proximal Neural Networks and Plug-and-Play Algorithms](https://mlsanity.com/paper/2011.02281/) (2020)
- [Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpoint](https://mlsanity.com/paper/2109.11375/) (2021)
- [HINT: Hierarchical Invertible Neural Transport for Density Estimation and Bayesian Inference](https://mlsanity.com/paper/1905.10687/) (2019)
- [RAISR: Rapid and Accurate Image Super Resolution](https://mlsanity.com/paper/1606.01299/) (2016)

## Built on by (most cited first, 10 of 10)

- [Posterior Sampling Based on Gradient Flows of the MMD with Negative Distance Kernel](https://mlsanity.com/paper/2310.03054/) (2023)
- [PatchNR: Learning from Very Few Images by Patch Normalizing Flow Regularization](https://mlsanity.com/paper/2205.12021/) (2022)
- [Equivariant Bootstrapping for Uncertainty Quantification in Imaging Inverse Problems](https://mlsanity.com/paper/2310.11838/) (2023)
- [Conditional Generative Models are Provably Robust: Pointwise Guarantees for Bayesian Inverse Problems](https://mlsanity.com/paper/2303.15845/) (2023)
- [Learning from small data sets: Patch-based regularizers in inverse problems for image reconstruction](https://mlsanity.com/paper/2312.16611/) (2023)
- [Importance Corrected Neural JKO Sampling](https://mlsanity.com/paper/2407.20444/) (2024)
- [T-FAKE: Synthesizing Thermal Images for Facial Landmarking](https://mlsanity.com/paper/2408.15127/) (2024)
- [Mixed Noise and Posterior Estimation with Conditional DeepGEM](https://mlsanity.com/paper/2402.02964/) (2024)
- [Robustness and Exploration of Variational and Machine Learning Approaches to Inverse Problems: An Overview](https://mlsanity.com/paper/2402.12072/) (2024)
- [Self-Supervised Uncertainty Estimation For Super-Resolution of Satellite Images](https://mlsanity.com/paper/2603.14074/) (2026)

## Abstract (arXiv)

Exploiting image patches instead of whole images have proved to be a powerful approach to tackle various problems in image processing. Recently, Wasserstein patch priors (WPP), which are based on the comparison of the patch distributions of the unknown image and a reference image, were successfully used as data-driven regularizers in the variational formulation of superresolution. However, for each input image, this approach requires the solution of a non-convex minimization problem which is computationally costly. In this paper, we propose to learn two kind of neural networks in an unsupervised way based on WPP loss functions. First, we show how convolutional neural networks (CNNs) can be incorporated. Once the network, called WPPNet, is learned, it can be very efficiently applied to any input image. Second, we incorporate conditional normalizing flows to provide a tool for uncertainty quantification. Numerical examples demonstrate the very good performance of WPPNets for superresolution in various image classes even if the forward operator is known only approximately.