mlsanityData to 14 Oct 2026

The Perception-Robustness Tradeoff in Deterministic Image Restoration

arXiv 2311.09253 ↗First posted 14 Nov 2023eess.IV cs.CV cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

We study the behavior of deterministic methods for solving inverse problems in imaging. These methods are commonly designed to achieve two goals: (1) attaining high perceptual quality, and (2) generating reconstructions that are consistent with the measurements. We provide a rigorous proof that the better a predictor satisfies these two requirements, the larger its Lipschitz constant must be, regardless of the nature of the degradation involved. In particular, to approach perfect perceptual quality and perfect consistency, the Lipschitz constant of the model must grow to infinity. This implies that such methods are necessarily more susceptible to adversarial attacks. We demonstrate our theory on single image super-resolution algorithms, addressing both noisy and noiseless settings. We also show how this undesired behavior can be leveraged to explore the posterior distribution, thereby allowing the deterministic model to imitate stochastic methods.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 48 of its 71 references that are in the map. First posted on arXiv 14 Nov 2023.

Builds on

48 of 71 references.

References · 48

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  5. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  6. 2016Improved Techniques for Training GANscited by 2,970
  7. 2014Conditional Generative Adversarial Netscited by 2,162
  8. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  9. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,887
  10. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  11. 2018Demystifying MMD GANscited by 1,293
  12. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
36 more
  1. 2016Colorful Image Colorizationcited by 1,114
  2. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
  3. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  4. 2016Adversarial Machine Learning at Scalecited by 936
  5. 2018Which Training Methods for GANs do actually Converge?cited by 679
  6. 2017The Perception-Distortion Tradeoffcited by 587
  7. 2021Designing a Practical Degradation Model for Deep Blind Image Super-Resolutioncited by 529
  8. 2017DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networkscited by 367
  9. 2019DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Bettercited by 362
  10. 2021Towards Real-World Blind Face Restoration with Generative Facial Priorcited by 349
  11. 2015Deep multi-scale video prediction beyond mean square errorcited by 300
  12. 2015Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networkscited by 214
  13. 2020Deblurring by Realistic Blurringcited by 188
  14. 2019Fast Underwater Image Enhancement for Improved Visual Perceptioncited by 183
  15. 2020Unfolding the Alternating Optimization for Blind Super Resolutioncited by 129
  16. 2017Real-Time User-Guided Image Colorization with Learned Deep Priorscited by 123
  17. 2019Frequency Separation for Real-World Super-Resolutioncited by 115
  18. 2022Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priorscited by 100
  19. 2020NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Resultscited by 83
  20. 2022Multi-Realism Image Compression with a Conditional Generatorcited by 80
  21. 2019Diversity-Sensitive Conditional Generative Adversarial Networkscited by 77
  22. 2021A Theory of the Distortion-Perception Tradeoff in Wasserstein Spacecited by 52
  23. 2022Learned reconstruction methods with convergence guaranteescited by 46
  24. 2020Spatially-Adaptive Pixelwise Networks for Fast Image Translationcited by 45
  25. 2017Pixel Recursive Super Resolutioncited by 44
  26. 2020The troublesome kernel -- On hallucinations, no free lunches and the accuracy-stability trade-off in inverse problemscited by 40
  27. 2019Evaluating Robustness of Deep Image Super-Resolution against Adversarial Attackscited by 37
  28. 2019AIM 2019 Challenge on Real-World Image Super-Resolution: Methods and Resultscited by 34
  29. 2021High Perceptual Quality Image Denoising with a Posterior Sampling CGANcited by 33
  30. 2017PixColor: Pixel Recursive Colorizationcited by 32
  31. 2017Sharpness-aware Low dose CT denoising using conditional generative adversarial networkcited by 29
  32. 2023Principal Uncertainty Quantification with Spatial Correlation for Image Restoration Problemscited by 28
  33. 2020Deep Generative Adversarial Residual Convolutional Networks for Real-World Super-Resolutioncited by 17
  34. 2022Reasons for the Superiority of Stochastic Estimators over Deterministic Ones: Robustness, Consistency and Perceptual Qualitycited by 13
  35. 2022High-Perceptual Quality JPEG Decoding via Posterior Samplingcited by 8
  36. 2021Generative Models Improve Radiomics Reproducibility in Low Dose CTs: A Simulation Studycited by 3

Built on by

Cited by year

Citing papers per year: 2024: 5; 2025: 2.

7 papers.

  1. 2024Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Modelscited by 12
  2. 2024FiRe: Fixed-points of Restoration Priors for Solving Inverse Problemscited by 6
  3. 2025Kernel Density Steering: Inference-Time Scaling via Mode Seeking for Image Restorationcited by 5
  4. 2024Perceptual Fairness in Image Restorationcited by 2
  5. 2024Robustness and Exploration of Variational and Machine Learning Approaches to Inverse Problems: An Overviewcited by 1
  6. 2024Imitating the Functionality of Image-to-Image Models Using a Single Examplecited by 0
  7. 2025On Inverse Problems, Parameter Estimation, and Domain Generalizationcited by 0