# The troublesome kernel -- On hallucinations, no free lunches and the accuracy-stability trade-off in inverse problems

arXiv: https://arxiv.org/abs/2001.01258 · HTML: https://mlsanity.com/paper/2001.01258/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jan 2020.

## Built on by (most cited first, 30 of 40)

- [The mathematics of adversarial attacks in AI -- Why deep learning is unstable despite the existence of stable neural networks](https://mlsanity.com/paper/2109.06098/) (2021)
- [Learning Regularization Functionals for Inverse Problems: A Comparative Study](https://mlsanity.com/paper/2510.01755/) (2025)
- [Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Models](https://mlsanity.com/paper/2405.16475/) (2024)
- [Limitations of Deep Learning for Inverse Problems on Digital Hardware](https://mlsanity.com/paper/2202.13490/) (2022)
- [Deep Learning for Accelerated and Robust MRI Reconstruction: a Review](https://mlsanity.com/paper/2404.15692/) (2024)
- [Ambiguity in solving imaging inverse problems with deep learning based operators](https://mlsanity.com/paper/2305.19774/) (2023)
- [DRIP: Deep Regularizers for Inverse Problems](https://mlsanity.com/paper/2304.00015/) (2023)
- [The Perception-Robustness Tradeoff in Deterministic Image Restoration](https://mlsanity.com/paper/2311.09253/) (2023)
- [On Logical Extrapolation for Mazes with Recurrent and Implicit Networks](https://mlsanity.com/paper/2410.03020/) (2024)
- [Computability of Optimizers](https://mlsanity.com/paper/2301.06148/) (2023)
- [Convergence and Recovery Guarantees of Unsupervised Neural Networks for Inverse Problems](https://mlsanity.com/paper/2309.12128/) (2023)
- [Uncertainty Estimation and Out-of-Distribution Detection for Deep Learning-Based Image Reconstruction using the Local Lipschitz](https://mlsanity.com/paper/2305.07618/) (2023)
- [When can you trust feature selection? -- I: A condition-based analysis of LASSO and generalised hardness of approximation](https://mlsanity.com/paper/2312.11425/) (2023)
- [Do stable neural networks exist for classification problems? -- A new view on stability in AI](https://mlsanity.com/paper/2401.07874/) (2024)
- [pcaGAN: Improving Posterior-Sampling cGANs via Principal Component Regularization](https://mlsanity.com/paper/2411.00605/) (2024)
- [Learning a Filtered Backprojection Reconstruction Method for Photoacoustic Computed Tomography with Hemispherical Measurement Geometries](https://mlsanity.com/paper/2412.01971/) (2024)
- [Approximation properties of neural ODEs](https://mlsanity.com/paper/2503.15696/) (2025)
- [Conformal Bounds on Full-Reference Image Quality for Imaging Inverse Problems](https://mlsanity.com/paper/2505.09528/) (2025)
- [How many measurements are enough? Bayesian recovery in inverse problems with general distributions](https://mlsanity.com/paper/2505.10630/) (2025)
- [A Stability Benchmark of Generative Regularizers for Inverse Problems](https://mlsanity.com/paper/2605.10076/) (2026)
- [Residual Back Projection With Untrained Neural Networks](https://mlsanity.com/paper/2210.14416/) (2022)
- [Deep Learning Empowered Super-Resolution: A Comprehensive Survey and Future Prospects](https://mlsanity.com/paper/2509.22692/) (2025)
- [Position-Blind Ptychography: Viability of image reconstruction via data-driven variational inference](https://mlsanity.com/paper/2509.25269/) (2025)
- [Minimax Multi-Target Conformal Prediction with Applications to Imaging Inverse Problems](https://mlsanity.com/paper/2511.13533/) (2025)
- [CHEM: Estimating and Understanding Hallucinations in Deep Learning for Image Processing](https://mlsanity.com/paper/2512.09806/) (2025)
- [Principled Confidence Estimation for Deep Computed Tomography](https://mlsanity.com/paper/2602.05812/) (2026)
- [Weakly Convex Ridge Regularization for 3D Non-Cartesian MRI Reconstruction](https://mlsanity.com/paper/2603.27158/) (2026)
- [Composing diffusion priors with explicit physical context via generative Gibbs sampling](https://mlsanity.com/paper/2605.10642/) (2026)
- [Two Approaches to Supervised Image Segmentation](https://mlsanity.com/paper/2307.10123/) (2023)
- [SGD Jittering: A Training Strategy for Robust and Accurate Model-Based Architectures](https://mlsanity.com/paper/2410.14667/) (2024)

## Abstract (arXiv)

Methods inspired by Artificial Intelligence (AI) are starting to fundamentally change computational science and engineering through breakthrough performances on challenging problems. However, reliability and trustworthiness of such techniques is a major concern. In inverse problems in imaging, the focus of this paper, there is increasing empirical evidence that methods may suffer from hallucinations, i.e., false, but realistic-looking artifacts; instability, i.e., sensitivity to perturbations in the data; and unpredictable generalization, i.e., excellent performance on some images, but significant deterioration on others. This paper provides a theoretical foundation for these phenomena. We give mathematical explanations for how and when such effects arise in arbitrary reconstruction methods, with several of our results taking the form of `no free lunch' theorems. Specifically, we show that (i) methods that overperform on a single image can wrongly transfer details from one image to another, creating a hallucination, (ii) methods that overperform on two or more images can hallucinate or be unstable, (iii) optimizing the accuracy-stability trade-off is generally difficult, (iv) hallucinations and instabilities, if they occur, are not rare events, and may be encouraged by standard training, (v) it may be impossible to construct optimal reconstruction maps for certain problems. Our results trace these effects to the kernel of the forward operator whenever it is nontrivial, but also apply to the case when the forward operator is ill-conditioned. Based on these insights, our work aims to spur research into new ways to develop robust and reliable AI-based methods for inverse problems in imaging.