mlsanityData to 7 Oct 2026

Higher Order Approximation Rates for ReLU CNNs in Korobov Spaces

arXiv 2501.11275 ↗First posted 20 Jan 2025cs.LG cs.NA math.NAcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper investigates the Lp approximation error for higher order Korobov functions using deep convolutional neural networks (CNNs) with ReLU activation. For target functions having a mixed derivative of order m+1 in each direction, we improve classical approximation rate of second order to (m+1)-th order (modulo a logarithmic factor) in terms of the depth of CNNs. The key ingredient in our analysis is approximate representation of high-order sparse grid basis functions by CNNs. The results suggest that higher order expressivity of CNNs does not severely suffer from the curse of dimensionality.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 32 of its 60 references that are in the map. First posted on arXiv 20 Jan 2025.

Builds on

32 of 60 references.

References · 32

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  4. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  5. 2016Error bounds for approximations with deep ReLU networkscited by 567
  6. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 416
  7. 2020A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospectscited by 396
  8. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  9. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 304
  10. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 176
  11. 2018Universality of Deep Convolutional Neural Networkscited by 176
  12. 2020Deep Network Approximation for Smooth Functionscited by 163
20 more
  1. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
  2. 2019Deep Network Approximation Characterized by Number of Neuronscited by 120
  3. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
  4. 2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
  5. 2022Error estimates for physics informed neural networks approximating the Navier-Stokes equationscited by 63
  6. 2022Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spacescited by 50
  7. 2023MgNO: Efficient Parameterization of Linear Operators via Multigridcited by 40
  8. 2018Equivalence of approximation by convolutional neural networks and fully-connected networkscited by 38
  9. 2019Nonlinear Approximation via Compositionscited by 37
  10. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  11. 2020Theory of Deep Convolutional Neural Networks II: Spherical Analysiscited by 22
  12. 2021Universal Consistency of Deep Convolutional Neural Networkscited by 16
  13. 2021Approximation Properties of Deep ReLU CNNscited by 15
  14. 2023Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regressioncited by 15
  15. 2024On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networkscited by 13
  16. 2023Rates of Approximation by ReLU Shallow Neural Networkscited by 12
  17. 2022Learning Ability of Interpolating Deep Convolutional Neural Networkscited by 9
  18. 2023Expressivity and Approximation Properties of Deep Neural Networks with ReLUk Activationcited by 8
  19. 2022Approximation analysis of CNNs from a feature extraction viewcited by 6
  20. 2024Approximation Rates for Shallow ReLUk Neural Networks on Sobolev Spaces via the Radon Transformcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Some Super-approximation Rates of ReLU Neural Networks for Korobov Functionscited by 3
  2. 2026Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networkscited by 3
  3. 2025On the Dimension-Free Approximation of Deep Neural Networks for Symmetric Korobov Functionscited by 0
  4. 2026Shallow neural network approximation in mixed Sobolev spacescited by 0