# Full error analysis for the training of deep neural networks

arXiv: https://arxiv.org/abs/1910.00121 · HTML: https://mlsanity.com/paper/1910.00121/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2019.

## Built on by (most cited first, 17 of 17)

- [On the approximation of functions by tanh neural networks](https://mlsanity.com/paper/2104.08938/) (2021)
- [Time Series Prediction using Deep Learning Methods in Healthcare](https://mlsanity.com/paper/2108.13461/) (2021)
- [Generic bounds on the approximation error for physics-informed (and) operator learning](https://mlsanity.com/paper/2205.11393/) (2022)
- [Partition of unity networks: deep hp-approximation](https://mlsanity.com/paper/2101.11256/) (2021)
- [Deep Network Approximation in Terms of Intrinsic Parameters](https://mlsanity.com/paper/2111.07964/) (2021)
- [On the existence of global minima and convergence analyses for gradient descent methods in the training of deep neural networks](https://mlsanity.com/paper/2112.09684/) (2021)
- [A proof of convergence for stochastic gradient descent in the training of artificial neural networks with ReLU activation for constant target functions](https://mlsanity.com/paper/2104.00277/) (2021)
- [Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions](https://mlsanity.com/paper/2112.07369/) (2021)
- [Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks](https://mlsanity.com/paper/2402.05155/) (2024)
- [Learning smooth functions in high dimensions: from sparse polynomials to deep neural networks](https://mlsanity.com/paper/2404.03761/) (2024)
- [CAS4DL: Christoffel Adaptive Sampling for function approximation via Deep Learning](https://mlsanity.com/paper/2208.12190/) (2022)
- [Error analysis for deep neural network approximations of parametric hyperbolic conservation laws](https://mlsanity.com/paper/2207.07362/) (2022)
- [Uncertainty quantification for deep learning-based schemes for solving high-dimensional backward stochastic differential equations](https://mlsanity.com/paper/2310.03393/) (2023)
- [Polynomial-Spline Neural Networks with Exact Integrals](https://mlsanity.com/paper/2110.14055/) (2021)
- [On bounds for norms of reparameterized ReLU artificial neural network parameters: sums of fractional powers of the Lipschitz norm control the network parameter vector](https://mlsanity.com/paper/2206.13646/) (2022)
- [Deep neural network approximation of composite functions without the curse of dimensionality](https://mlsanity.com/paper/2304.05790/) (2023)
- [Error analysis for the deep Kolmogorov method](https://mlsanity.com/paper/2508.17167/) (2025)

## Abstract (arXiv)

Deep learning algorithms have been applied very successfully in recent years to a range of problems out of reach for classical solution paradigms. Nevertheless, there is no completely rigorous mathematical error and convergence analysis which explains the success of deep learning algorithms. The error of a deep learning algorithm can in many situations be decomposed into three parts, the approximation error, the generalization error, and the optimization error. In this work we estimate for a certain deep learning algorithm each of these three errors and combine these three error estimates to obtain an overall error analysis for the deep learning algorithm under consideration. In particular, we thereby establish convergence with a suitable convergence speed for the overall error of the deep learning algorithm under consideration. Our convergence speed analysis is far from optimal and the convergence speed that we establish is rather slow, increases exponentially in the dimensions, and, in particular, suffers from the curse of dimensionality. The main contribution of this work is, instead, to provide a full error analysis (i) which covers each of the three different sources of errors usually emerging in deep learning algorithms and (ii) which merges these three sources of errors into one overall error estimate for the considered deep learning algorithm.