# On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks arXiv: https://arxiv.org/abs/2409.00901 · HTML: https://mlsanity.com/paper/2409.00901/ ## Key facts - Cited by 13 papers in the map (data through 14 Oct 2026). - Builds on 24 of its 51 references that are in the map. - First posted on arXiv 2 Sep 2024. ## Builds on (most cited first, 24 of 24) - [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016) - [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017) - [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017) - [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017) - [Optimal approximation of continuous functions by very deep ReLU networks](https://mlsanity.com/paper/1802.03620/) (2018) - [Diffusion Models are Minimax Optimal Distribution Estimators](https://mlsanity.com/paper/2303.01861/) (2023) - [Deep Network Approximation for Smooth Functions](https://mlsanity.com/paper/2001.03040/) (2020) - [Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality](https://mlsanity.com/paper/1810.08033/) (2018) - [Approximating Continuous Functions by ReLU Nets of Minimal Width](https://mlsanity.com/paper/1710.11278/) (2017) - [Deep Network Approximation Characterized by Number of Neurons](https://mlsanity.com/paper/1906.05497/) (2019) - [Optimal Approximation Rate of ReLU Networks in terms of Width and Depth](https://mlsanity.com/paper/2103.00502/) (2021) - [Error bounds for approximations with deep ReLU neural networks in Ws,p norms](https://mlsanity.com/paper/1902.07896/) (2019) - [The phase diagram of approximation rates for deep neural networks](https://mlsanity.com/paper/1906.09477/) (2019) - [Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recovery](https://mlsanity.com/paper/1908.01842/) (2019) - [Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks](https://mlsanity.com/paper/1610.09887/) (2016) - [Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces](https://mlsanity.com/paper/2211.14400/) (2022) - [Fast convergence rates of deep neural networks for classification](https://mlsanity.com/paper/1812.03599/) (2018) - [Nonlinear Approximation and (Deep) ReLU Networks](https://mlsanity.com/paper/1905.02199/) (2019) - [Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimality](https://mlsanity.com/paper/2110.06897/) (2021) - [Nonlinear Approximation via Compositions](https://mlsanity.com/paper/1902.10170/) (2019) - [Approximation bounds for norm constrained neural networks with applications to regression and GANs](https://mlsanity.com/paper/2201.09418/) (2022) - [Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regression](https://mlsanity.com/paper/2304.01561/) (2023) - [Approximation in shift-invariant spaces with deep ReLU neural networks](https://mlsanity.com/paper/2005.11949/) (2020) - [On the rates of convergence for learning with convolutional neural networks](https://mlsanity.com/paper/2403.16459/) (2024) ## Built on by (most cited first, 13 of 13) - [Transformers Can Overcome the Curse of Dimensionality: A Theoretical Study from an Approximation Perspective](https://mlsanity.com/paper/2504.13558/) (2025) - [Higher Order Approximation Rates for ReLU CNNs in Korobov Spaces](https://mlsanity.com/paper/2501.11275/) (2025) - [Some Super-approximation Rates of ReLU Neural Networks for Korobov Functions](https://mlsanity.com/paper/2507.10345/) (2025) - [Sharper Generalization Bounds for Transformer](https://mlsanity.com/paper/2603.21541/) (2026) - [Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networks](https://mlsanity.com/paper/2605.31152/) (2026) - [CHEM: Estimating and Understanding Hallucinations in Deep Learning for Image Processing](https://mlsanity.com/paper/2512.09806/) (2025) - [SAQNN: Spectral Adaptive Quantum Neural Network as a Universal Approximator](https://mlsanity.com/paper/2602.09718/) (2026) - [Memorization capacity of deep ReLU neural networks characterized by width and depth](https://mlsanity.com/paper/2603.09589/) (2026) - [Efficient Approximation to Analytic and Lp functions by Height-Augmented ReLU Networks](https://mlsanity.com/paper/2603.11128/) (2026) - [Black-Box Assisted Regression: Phase Transitions and Minimax Optimality](https://mlsanity.com/paper/2606.25743/) (2026) - [Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Width](https://mlsanity.com/paper/2607.10589/) (2026) - [Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks](https://mlsanity.com/paper/2609.19937/) (2026) - [Minimax rates for learning spectral Barron functions by deep ReLU neural networks](https://mlsanity.com/paper/2609.39020/) (2026) ## Abstract (arXiv) This paper studies the problem of how efficiently functions in the Sobolev spaces Ws,q([0,1]d) and Besov spaces Bsq,r([0,1]d) can be approximated by deep ReLU neural networks with width W and depth L, when the error is measured in the Lp([0,1]d) norm. This problem has been studied by several recent works, which obtained the approximation rate O((WL)-2s/d) up to logarithmic factors when p=q=∞, and the rate O(L-2s/d) for networks with fixed width when the Sobolev embedding condition 1/q -1/p