# The Perception-Robustness Tradeoff in Deterministic Image Restoration

arXiv: https://arxiv.org/abs/2311.09253 · HTML: https://mlsanity.com/paper/2311.09253/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 71 references that are in the map.
- First posted on arXiv 14 Nov 2023.

## Builds on (most cited first, 30 of 48)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Designing a Practical Degradation Model for Deep Blind Image Super-Resolution](https://mlsanity.com/paper/2103.14006/) (2021)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019)
- [Towards Real-World Blind Face Restoration with Generative Facial Prior](https://mlsanity.com/paper/2101.04061/) (2021)
- [Deep multi-scale video prediction beyond mean square error](https://mlsanity.com/paper/1511.05440/) (2015)
- [Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks](https://mlsanity.com/paper/1506.05751/) (2015)
- [Deblurring by Realistic Blurring](https://mlsanity.com/paper/2004.01860/) (2020)
- [Fast Underwater Image Enhancement for Improved Visual Perception](https://mlsanity.com/paper/1903.09766/) (2019)
- [Unfolding the Alternating Optimization for Blind Super Resolution](https://mlsanity.com/paper/2010.02631/) (2020)
- [Real-Time User-Guided Image Colorization with Learned Deep Priors](https://mlsanity.com/paper/1705.02999/) (2017)
- [Frequency Separation for Real-World Super-Resolution](https://mlsanity.com/paper/1911.07850/) (2019)
- [Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors](https://mlsanity.com/paper/2202.13142/) (2022)

## Built on by (most cited first, 7 of 7)

- [Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Models](https://mlsanity.com/paper/2405.16475/) (2024)
- [FiRe: Fixed-points of Restoration Priors for Solving Inverse Problems](https://mlsanity.com/paper/2411.18970/) (2024)
- [Kernel Density Steering: Inference-Time Scaling via Mode Seeking for Image Restoration](https://mlsanity.com/paper/2507.05604/) (2025)
- [Perceptual Fairness in Image Restoration](https://mlsanity.com/paper/2405.13805/) (2024)
- [Robustness and Exploration of Variational and Machine Learning Approaches to Inverse Problems: An Overview](https://mlsanity.com/paper/2402.12072/) (2024)
- [Imitating the Functionality of Image-to-Image Models Using a Single Example](https://mlsanity.com/paper/2406.00828/) (2024)
- [On Inverse Problems, Parameter Estimation, and Domain Generalization](https://mlsanity.com/paper/2506.06024/) (2025)

## Abstract (arXiv)

We study the behavior of deterministic methods for solving inverse problems in imaging. These methods are commonly designed to achieve two goals: (1) attaining high perceptual quality, and (2) generating reconstructions that are consistent with the measurements. We provide a rigorous proof that the better a predictor satisfies these two requirements, the larger its Lipschitz constant must be, regardless of the nature of the degradation involved. In particular, to approach perfect perceptual quality and perfect consistency, the Lipschitz constant of the model must grow to infinity. This implies that such methods are necessarily more susceptible to adversarial attacks. We demonstrate our theory on single image super-resolution algorithms, addressing both noisy and noiseless settings. We also show how this undesired behavior can be leveraged to explore the posterior distribution, thereby allowing the deterministic model to imitate stochastic methods.