# Error bounds for approximations with deep ReLU neural networks in Ws,p norms

arXiv: https://arxiv.org/abs/1902.07896 · HTML: https://mlsanity.com/paper/1902.07896/

## Key facts

- Cited by 95 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2019.

## Built on by (most cited first, 30 of 95)

- [Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data](https://mlsanity.com/paper/2302.07194/) (2023)
- [On the approximation of functions by tanh neural networks](https://mlsanity.com/paper/2104.08938/) (2021)
- [An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization](https://mlsanity.com/paper/2404.07771/) (2024)
- [Generic bounds on the approximation error for physics-informed (and) operator learning](https://mlsanity.com/paper/2205.11393/) (2022)
- [Error analysis for physics informed neural networks (PINNs) approximating Kolmogorov PDEs](https://mlsanity.com/paper/2106.14473/) (2021)
- [Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces](https://mlsanity.com/paper/2211.14400/) (2022)
- [Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networks](https://mlsanity.com/paper/2107.00940/) (2021)
- [Universal Approximation Theorems for Differentiable Geometric Deep Learning](https://mlsanity.com/paper/2101.05390/) (2021)
- [Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems](https://mlsanity.com/paper/2212.03130/) (2022)
- [Deep Network Approximation: Beyond ReLU to Diverse Activation Functions](https://mlsanity.com/paper/2307.06555/) (2023)
- [Universal Approximation Under Constraints is Possible with Transformers](https://mlsanity.com/paper/2110.03303/) (2021)
- [Nearly Optimal VC-Dimension and Pseudo-Dimension Bounds for Deep Neural Network Derivatives](https://mlsanity.com/paper/2305.08466/) (2023)
- [Sumformer: Universal Approximation for Efficient Transformers](https://mlsanity.com/paper/2307.02301/) (2023)
- [Learning elliptic partial differential equations with randomized linear algebra](https://mlsanity.com/paper/2102.00491/) (2021)
- [Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations](https://mlsanity.com/paper/2206.09527/) (2022)
- [A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equations](https://mlsanity.com/paper/2103.06183/) (2021)
- [Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithms](https://mlsanity.com/paper/2310.10810/) (2023)
- [Variational Physics Informed Neural Networks: the role of quadratures and test functions](https://mlsanity.com/paper/2109.02035/) (2021)
- [Neural Network Architecture Beyond Width and Depth](https://mlsanity.com/paper/2205.09459/) (2022)
- [Lyapunov-Net: A Deep Neural Network Architecture for Lyapunov Function Approximation](https://mlsanity.com/paper/2109.13359/) (2021)
- [Universal Approximation Theorem and error bounds for quantum neural networks and quantum reservoirs](https://mlsanity.com/paper/2307.12904/) (2023)
- [Approximation Properties of Deep ReLU CNNs](https://mlsanity.com/paper/2109.00190/) (2021)
- [Error Estimates for the Deep Ritz Method with Boundary Penalty](https://mlsanity.com/paper/2103.01007/) (2021)
- [Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss](https://mlsanity.com/paper/2402.00152/) (2024)
- [On the Optimal Expressive Power of ReLU DNNs and Its Application in Approximation with Kolmogorov Superposition Theorem](https://mlsanity.com/paper/2308.05509/) (2023)
- [On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks](https://mlsanity.com/paper/2409.00901/) (2024)
- [ReLU Deep Neural Networks from the Hierarchical Basis Perspective](https://mlsanity.com/paper/2105.04156/) (2021)
- [Solving PDEs on Unknown Manifolds with Machine Learning](https://mlsanity.com/paper/2106.06682/) (2021)
- [Deep Network Approximation in Terms of Intrinsic Parameters](https://mlsanity.com/paper/2111.07964/) (2021)
- [Rates of Approximation by ReLU Shallow Neural Networks](https://mlsanity.com/paper/2307.12461/) (2023)

## Abstract (arXiv)

We analyze approximation rates of deep ReLU neural networks for Sobolev-regular functions with respect to weaker Sobolev norms. First, we construct, based on a calculus of ReLU networks, artificial neural networks with ReLU activation functions that achieve certain approximation rates. Second, we establish lower bounds for the approximation by ReLU neural networks for classes of Sobolev-regular functions. Our results extend recent advances in the approximation theory of ReLU networks to the regime that is most relevant for applications in the numerical analysis of partial differential equations.