Data to 7 Oct 2026

The Modern Mathematics of Deep Learning

arXiv 2105.04026 ↗First posted 9 May 2021cs.LG stat.MLcited by 63

Not on the map. Why?

Abstract (arXiv)

We describe the new field of mathematical analysis of deep learning. This field emerged around a list of research questions that were not answered within the classical framework of learning theory. These questions concern: the outstanding generalization power of overparametrized neural networks, the role of depth in deep architectures, the apparent absence of the curse of dimensionality, the surprisingly successful optimization performance despite the non-convexity of the problem, understanding what features are learned, why deep architectures perform exceptionally well in physical problems, and which fine aspects of an architecture affect the behavior of a learning task in which way. We present an overview of modern approaches that yield partial answers to these questions. For selected approaches, we describe the main ideas in more detail.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 May 2021.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 23; 2023: 13; 2024: 9; 2025: 2; 2026: 8.

63 papers.

Sort:

By area:

2024DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Trainingcited by 137
2021Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionalitycited by 34
2023Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grokcited by 31
2022Neural and spectral operator surrogates: unified construction and expression rate boundscited by 21
2022Problem-Dependent Power of Quantum Neural Networks on Multi-Class Classificationcited by 20
2022Training Fully Connected Neural Networks is existsℝ-Completecited by 15
2022Robust SDE-Based Variational Formulations for Solving Linear PDEs via Deep Learningcited by 13
2022Dynamical systems' based neural networkscited by 10
2022Spiking neural networks for nonlinear regressioncited by 10
2021Optimal learning of high-dimensional classification problems using deep neural networkscited by 9
2022Limitations of Deep Learning for Inverse Problems on Digital Hardwarecited by 9
2022On the Omnipresence of Spurious Local Minima in Certain Neural Network Training Problemscited by 8

38 more

2022To be or not to be stable, that is the question: understanding neural networks for inverse problemscited by 8
2022Approximation results for Gradient Descent trained Shallow Neural Networks in 1dcited by 7
2022ML Interpretability: Simple Isn't Easycited by 7
2023Noise-Free Sampling Algorithms via Regularized Wasserstein Proximalscited by 7
2022On generalization bounds for deep networks based on loss surface implicit regularizationcited by 5
2022Decision boundaries and convex hulls in the feature space that deep learning functions learn from imagescited by 5
2022Learning ReLU networks to high uniform accuracy is intractablecited by 5
2023Computability of Optimizerscited by 5
2023Error convergence and engineering-guided hyperparameter search of PINNs: towards optimized I-FENN performancecited by 5
2023Statistical learning theory and Occam's razor: The core argumentcited by 5
2024Stable Learning Using Spiking Neural Networks Equipped With Affine Encoders and Decoderscited by 5
2024Understanding the Difficulty of Solving Cauchy Problems with PINNscited by 5
2024The Definitive Guide to Policy Gradients in Deep Reinforcement Learning: Theory, Algorithms and Implementationscited by 4
2021SaDe: Learning Models that Provably Satisfy Domain Constraintscited by 3
2021Envisioning Future Deep Learning Theories: Some Basic Concepts and Characteristicscited by 3
2022Deep neural networks can stably solve high-dimensional, noisy, non-linear inverse problemscited by 3
2023On the Generalization and Approximation Capacities of Neural Controlled Differential Equationscited by 3
2021Component Transfer Learning for Deep RL Based on Abstract Representationscited by 2
2022Blind Estimation of a Doubly Selective OFDM Channel: A Deep Learning Algorithm and Theorycited by 2
2022Rosenblatt's first theorem and frugality of deep learningcited by 2
2022Accelerating hypersonic reentry simulations using deep learning-based hybridization (with guarantees)cited by 2
2023Approximation Results for Gradient Descent trained Neural Networkscited by 2
2026Solomonoff inductioncited by 2
2021Sobolev-type embeddings for neural network approximation spacescited by 1
2022Density Regression and Uncertainty Quantification with Bayesian Deep Noise Neural Networkscited by 1
2022Transmission Neural Networks: From Virus Spread Models to Neural Networkscited by 1
2023How to Do Things with Deep Learning Codecited by 1
2023Transgressing the boundaries: towards a rigorous understanding of deep learning and its (non-)robustnesscited by 1
2023Towards a population-informed approach to the definition of data-driven models for structural dynamicscited by 1
2024A topological description of loss surfaces based on Betti Numberscited by 1
2024Mathematical Algorithm Design for Deep Learning under Societal and Judicial Constraints: The Algorithmic Transparency Requirementcited by 1
2024Approximation and Gradient Descent Training with Neural Networkscited by 1
2024On Functional Dimension and Persistent Pseudodimensioncited by 1
2021A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep Neural Networkscited by 0
2021Using Computational Intelligence for solving the Ornstein-Zernike equationcited by 0
2022On Practical Reinforcement Learning: Provable Robustness, Scalability, and Statistical Efficiencycited by 0
2022The Mathematics of Artificial Intelligencecited by 0
2022Global quantitative robustness of regression feed-forward neural networkscited by 0

50 most cited of 63 All 63 (JSON)