mlsanityData to 7 Oct 2026

On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks

arXiv 2409.00901 ↗First posted 2 Sep 2024stat.ML cs.LG cs.NAcited by 13

Not on the map. Why?

Abstract (arXiv)

This paper studies the problem of how efficiently functions in the Sobolev spaces Ws,q([0,1]d) and Besov spaces Bsq,r([0,1]d) can be approximated by deep ReLU neural networks with width W and depth L, when the error is measured in the Lp([0,1]d) norm. This problem has been studied by several recent works, which obtained the approximation rate O((WL)-2s/d) up to logarithmic factors when p=q=∞, and the rate O(L-2s/d) for networks with fixed width when the Sobolev embedding condition 1/q -1/p<s/d holds. We generalize these results by showing that the rate O((WL)-2s/d) indeed holds under the Sobolev embedding condition. It is known that this rate is optimal up to logarithmic factors. The key tool in our proof is a novel encoding of sparse vectors by using deep ReLU neural networks with varied width and depth, which may be of independent interest.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 24 of its 51 references that are in the map. First posted on arXiv 2 Sep 2024.

Builds on

24 of 51 references.

References · 24

  1. 2016Error bounds for approximations with deep ReLU networkscited by 567
  2. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  3. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 304
  4. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 176
  5. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 165
  6. 2020Deep Network Approximation for Smooth Functionscited by 163
  7. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
  8. 2023Diffusion Models are Minimax Optimal Distribution Estimatorscited by 159
  9. 2017Approximating Continuous Functions by ReLU Nets of Minimal Widthcited by 120
  10. 2019Deep Network Approximation Characterized by Number of Neuronscited by 120
  11. 2019Error bounds for approximations with deep ReLU neural networks in Ws,p normscited by 95
  12. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
12 more
  1. 2019The phase diagram of approximation rates for deep neural networkscited by 86
  2. 2019Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recoverycited by 73
  3. 2016Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networkscited by 58
  4. 2022Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spacescited by 50
  5. 2018Fast convergence rates of deep neural networks for classificationcited by 46
  6. 2019Nonlinear Approximation and (Deep) ReLU Networkscited by 45
  7. 2021Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimalitycited by 40
  8. 2019Nonlinear Approximation via Compositionscited by 37
  9. 2022Approximation bounds for norm constrained neural networks with applications to regression and GANscited by 21
  10. 2023Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regressioncited by 15
  11. 2020Approximation in shift-invariant spaces with deep ReLU neural networkscited by 12
  12. 2024On the rates of convergence for learning with convolutional neural networkscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 9.

13 papers.

Sort:

  1. 2025Transformers Can Overcome the Curse of Dimensionality: A Theoretical Study from an Approximation Perspectivecited by 5
  2. 2025Higher Order Approximation Rates for ReLU CNNs in Korobov Spacescited by 4
  3. 2025Some Super-approximation Rates of ReLU Neural Networks for Korobov Functionscited by 3
  4. 2026Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networkscited by 3
  5. 2026Sharper Generalization Bounds for Transformercited by 2
  6. 2025CHEM: Estimating and Understanding Hallucinations in Deep Learning for Image Processingcited by 1
  7. 2026SAQNN: Spectral Adaptive Quantum Neural Network as a Universal Approximatorcited by 0
  8. 2026Memorization capacity of deep ReLU neural networks characterized by width and depthcited by 0
  9. 2026Efficient Approximation to Analytic and Lp functions by Height-Augmented ReLU Networkscited by 0
  10. 2026Black-Box Assisted Regression: Phase Transitions and Minimax Optimalitycited by 0
  11. 2026Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Widthcited by 0
  12. 2026Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networkscited by 0
1 more
  1. 2026Minimax rates for learning spectral Barron functions by deep ReLU neural networkscited by 0