# Approximation properties of Residual Neural Networks for Kolmogorov PDEs

arXiv: https://arxiv.org/abs/2111.00215 · HTML: https://mlsanity.com/paper/2111.00215/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 35 references that are in the map.
- First posted on arXiv 30 Oct 2021.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Wider or Deeper: Revisiting the ResNet Model for Visual Recognition](https://mlsanity.com/paper/1611.10080/) (2016)
- [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017)
- [Data Driven Governing Equations Approximation Using Deep Neural Networks](https://mlsanity.com/paper/1811.05537/) (2018)
- [A Theoretical Analysis of Deep Neural Networks and Parametric PDEs](https://mlsanity.com/paper/1904.00377/) (2019)
- [Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations](https://mlsanity.com/paper/1709.05963/) (2017)
- [A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations](https://mlsanity.com/paper/1809.02362/) (2018)
- [Approximation spaces of deep neural networks](https://mlsanity.com/paper/1905.01208/) (2019)
- [Solving the Kolmogorov PDE by means of deep learning](https://mlsanity.com/paper/1806.00421/) (2018)
- [A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients](https://mlsanity.com/paper/1809.07321/) (2018)
- [Deep neural network approximation for high-dimensional elliptic PDEs with boundary conditions](https://mlsanity.com/paper/2007.05384/) (2020)
- [A Priori Estimates of the Population Risk for Residual Networks](https://mlsanity.com/paper/1903.02154/) (2019)
- [Residual Networks of Residual Networks: Multilevel Residual Networks](https://mlsanity.com/paper/1608.02908/) (2016)
- [Neural ODEs as the Deep Limit of ResNets with constant weights](https://mlsanity.com/paper/1906.12183/) (2019)
- [Space-time deep neural network approximations for high-dimensional partial differential equations](https://mlsanity.com/paper/2006.02199/) (2020)
- [Space-time error estimates for deep neural network approximations for differential equations](https://mlsanity.com/paper/1908.03833/) (2019)
- [Deep neural network approximations for Monte Carlo algorithms](https://mlsanity.com/paper/1908.10828/) (2019)
- [On the space-time expressivity of ResNets](https://mlsanity.com/paper/1910.09599/) (2019)

## Built on by (most cited first, 2 of 2)

- [ADAPT: Lightweight, Long-Range Machine Learning Force Fields Without Graphs](https://mlsanity.com/paper/2509.24115/) (2025)
- [Residual neural networks overcome the curse of dimensionality for semilinear heat equations](https://mlsanity.com/paper/2609.03626/) (2026)

## Abstract (arXiv)

In recent years residual neural networks (ResNets) as introduced by [He, K., Zhang, X., Ren, S., and Sun, J., Proceedings of the IEEE conference on computer vision and pattern recognition (2016), 770-778] have become very popular in a large number of applications, including in image classification and segmentation. They provide a new perspective in training very deep neural networks without suffering the vanishing gradient problem. In this article we show that ResNets are able to approximate solutions of Kolmogorov partial differential equations (PDEs) with constant diffusion and possibly nonlinear drift coefficients without suffering the curse of dimensionality, which is to say the number of parameters of the approximating ResNets grows at most polynomially in the reciprocal of the approximation accuracy varepsilon > 0 and the dimension of the considered PDE d∈mathbbN. We adapt a proof in [Jentzen, A., Salimova, D., and Welti, T., Commun. Math. Sci. 19, 5 (2021), 1167-1205] - who showed a similar result for feedforward neural networks (FNNs) - to ResNets. In contrast to FNNs, the Euler-Maruyama approximation structure of ResNets simplifies the construction of the approximating ResNets substantially. Moreover, contrary to the above work, in our proof using ResNets does not require the existence of an FNN (or a ResNet) representing the identity map, which enlarges the set of applicable activation functions.