# The Unreasonable Effectiveness of Deep Learning in Artificial Intelligence

arXiv: https://arxiv.org/abs/2002.04806 · HTML: https://mlsanity.com/paper/2002.04806/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2020.

## Built on by (most cited first, 30 of 57)

- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [Generative AI](https://mlsanity.com/paper/2309.07930/) (2023)
- [Hebbian Deep Learning Without Feedback](https://mlsanity.com/paper/2209.11883/) (2022)
- [Large Language Models and the Reverse Turing Test](https://mlsanity.com/paper/2207.14382/) (2022)
- [SoftHebb: Bayesian Inference in Unsupervised Hebbian Soft Winner-Take-All Networks](https://mlsanity.com/paper/2107.05747/) (2021)
- [P2M: A Processing-in-Pixel-in-Memory Paradigm for Resource-Constrained TinyML Applications](https://mlsanity.com/paper/2203.04737/) (2022)
- [On the link between conscious function and general intelligence in humans and machines](https://mlsanity.com/paper/2204.05133/) (2022)
- [Math Agents: Computational Infrastructure, Mathematical Embedding, and Genomics](https://mlsanity.com/paper/2307.02502/) (2023)
- [Towards a theory of out-of-distribution learning](https://mlsanity.com/paper/2109.14501/) (2021)
- [Machine Learning and Deep Learning -- A review for Ecologists](https://mlsanity.com/paper/2204.05023/) (2022)
- [Demystification of Few-shot and One-shot Learning](https://mlsanity.com/paper/2104.12174/) (2021)
- [Deep Learning for Scene Classification: A Survey](https://mlsanity.com/paper/2101.10531/) (2021)
- [Robust PDF Document Conversion Using Recurrent Neural Networks](https://mlsanity.com/paper/2102.09395/) (2021)
- [Few Shot Learning for Medical Imaging: A Comparative Analysis of Methodologies and Formal Mathematical Framework](https://mlsanity.com/paper/2305.04401/) (2023)
- [Physics-informed attention-based neural network for solving non-linear partial differential equations](https://mlsanity.com/paper/2105.07898/) (2021)
- [Optimized ensemble deep learning framework for scalable forecasting of dynamics containing extreme events](https://mlsanity.com/paper/2106.08968/) (2021)
- [Structural Neural Additive Models: Enhanced Interpretable Machine Learning](https://mlsanity.com/paper/2302.09275/) (2023)
- [Gaussian Process Surrogate Models for Neural Networks](https://mlsanity.com/paper/2208.06028/) (2022)
- [Deep Learning of Compositional Targets with Hierarchical Spectral Methods](https://mlsanity.com/paper/2602.10867/) (2026)
- [Should attention be all we need? The epistemic and ethical implications of unification in machine learning](https://mlsanity.com/paper/2205.08377/) (2022)
- [Probabilistic Lipschitzness and the Stable Rank for Comparing Explanation Models](https://mlsanity.com/paper/2402.18863/) (2024)
- [Trustworthy Artificial Intelligence in the Context of Metrology](https://mlsanity.com/paper/2406.10117/) (2024)
- [Algebraic Adversarial Attacks on Integrated Gradients](https://mlsanity.com/paper/2407.16233/) (2024)
- [Novel Saliency Analysis for the Forward Forward Algorithm](https://mlsanity.com/paper/2409.15365/) (2024)
- [Scalable Geometric Deep Learning on Molecular Graphs](https://mlsanity.com/paper/2112.03364/) (2021)
- [Soft ascent-descent as a stable and flexible alternative to flooding](https://mlsanity.com/paper/2310.10006/) (2023)
- [Decoding complexity: how machine learning is redefining scientific discovery](https://mlsanity.com/paper/2405.04161/) (2024)
- [On Space Folds of ReLU Neural Networks](https://mlsanity.com/paper/2502.09954/) (2025)
- [Tangentially Aligned Integrated Gradients for User-Friendly Explanations](https://mlsanity.com/paper/2503.08240/) (2025)
- [A Comprehensive Study of Vision Transformers in Image Classification Tasks](https://mlsanity.com/paper/2312.01232/) (2023)

## Abstract (arXiv)

Deep learning networks have been trained to recognize speech, caption photographs and translate text between languages at high levels of performance. Although applications of deep learning networks to real world problems have become ubiquitous, our understanding of why they are so effective is lacking. These empirical results should not be possible according to sample complexity in statistics and non-convex optimization theory. However, paradoxes in the training and effectiveness of deep learning networks are being investigated and insights are being found in the geometry of high-dimensional spaces. A mathematical theory of deep learning would illuminate how they function, allow us to assess the strengths and weaknesses of different network architectures and lead to major improvements. Deep learning has provided natural ways for humans to communicate with digital devices and is foundational for building artificial general intelligence. Deep learning was inspired by the architecture of the cerebral cortex and insights into autonomy and general intelligence may be found in other brain regions that are essential for planning and survival, but major breakthroughs will be needed to achieve these goals.