mlsanityData to 7 Oct 2026

Limitations of Deep Learning for Inverse Problems on Digital Hardware

arXiv 2202.13490 ↗First posted 28 Feb 2022cs.LG cs.AI eess.SPcited by 9

Not on the map. Why?

Abstract (arXiv)

Deep neural networks have seen tremendous success over the last years. Since the training is performed on digital hardware, in this paper, we analyze what actually can be computed on current hardware platforms modeled as Turing machines, which would lead to inherent restrictions of deep learning. For this, we focus on the class of inverse problems, which, in particular, encompasses any task to reconstruct data from measurements. We prove that finite-dimensional inverse problems are not Banach-Mazur computable for small relaxation parameters. Even more, our results introduce a lower bound on the accuracy that can be obtained algorithmically.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 32 of its 97 references that are in the map. First posted on arXiv 28 Feb 2022.

Builds on

32 of 97 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  5. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  6. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  7. 2018Robustness May Be at Odds with Accuracycited by 873
  8. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  9. 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 459
  10. 2018Learning to See in the Darkcited by 459
  11. 2016Deep Convolutional Neural Network for Inverse Problems in Imagingcited by 431
  12. 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 411
20 more
  1. 2017Learning a Variational Network for Reconstruction of Accelerated MRI Datacited by 340
  2. 2020Deep Learning Techniques for Inverse Problems in Imagingcited by 290
  3. 2018Audio Adversarial Examples: Targeted Attacks on Speech-to-Textcited by 268
  4. 2017A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstructioncited by 231
  5. 2017Image reconstruction by domain transform manifold learningcited by 231
  6. 2019On instabilities of deep learning in image reconstruction - Does AI come at a cost?cited by 211
  7. 2017Solving ill-posed inverse problems using iterative deep neural networkscited by 145
  8. 2020Explainable Deep Learning: A Field Guide for the Uninitiatedcited by 127
  9. 2017A Survey of Neuromorphic Computing and Neural Networks in Hardwarecited by 82
  10. 2021The Modern Mathematics of Deep Learningcited by 63
  11. 2020The gap between theory and practice in function approximation with deep neural networkscited by 53
  12. 2021Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problemcited by 53
  13. 2015A Deep Learning Approach to Structured Signal Recoverycited by 43
  14. 2020The troublesome kernel -- On hallucinations, no free lunches and the accuracy-stability trade-off in inverse problemscited by 40
  15. 2018Learning The Invisible: A Hybrid Deep Learning-Shearlet Framework for Limited Angle Computed Tomographycited by 29
  16. 2021The mathematics of adversarial attacks in AI -- Why deep learning is unstable despite the existence of stable neural networkscited by 16
  17. 2021A Rate-Distortion Framework for Explaining Black-box Model Decisionscited by 11
  18. 2021The extended Smale's 9th problem -- On computational barriers and paradoxes in estimation, regularisation, computer-assisted proofs and learningcited by 11
  19. 2016Computability and Analysis, a Historical Approachcited by 1
  20. 2021Compressed Ultrasound Imaging:from Sub-Nyquist Rates to Super-Resolutioncited by 1

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 2; 2024: 4.

9 papers.

  1. 2022NESTANets: Stable, accurate and efficient neural networks for analysis-sparse inverse problemscited by 5
  2. 2023Computability of Optimizerscited by 5
  3. 2022Deep neural networks can stably solve high-dimensional, noisy, non-linear inverse problemscited by 3
  4. 2022Limitations of neural network training due to numerical instability of backpropagationcited by 2
  5. 2024Mathematical Algorithm Design for Deep Learning under Societal and Judicial Constraints: The Algorithmic Transparency Requirementcited by 1
  6. 2024Deep-Unrolling Multidimensional Harmonic Retrieval Algorithms on Neuromorphic Hardwarecited by 1
  7. 2023Reliable AI: Does the Next Generation Require Quantum Computing?cited by 0
  8. 2024Tighter Learning Guarantees on Digital Computers via Concentration of Measure on Finite Spacescited by 0
  9. 2024Computability of Classification and Deep Learning: From Theoretical Limits to Practical Feasibility through Quantizationcited by 0