mlsanityData to 7 Oct 2026

On the rates of convergence for learning with convolutional neural networks

arXiv 2403.16459 ↗First posted 25 Mar 2024cs.LG math.ST stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

We study approximation and learning capacities of convolutional neural networks (CNNs) with one-side zero-padding and multiple channels. Our first result proves a new approximation bound for CNNs with certain constraint on the weights. Our second result gives new analysis on the covering number of feed-forward neural networks with CNNs as special cases. The analysis carefully takes into account the size of the weights and hence gives better bounds than the existing literature in some situations. Using these two results, we are able to derive rates of convergence for estimators based on CNNs in many learning problems. In particular, we establish minimax optimal convergence rates of the least squares based on CNNs for learning smooth functions in the nonparametric regression setting. For binary classification, we derive convergence rates for CNN classifiers with hinge loss and logistic loss. It is also shown that the obtained rates for classification are minimax optimal in some common settings.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 23 of its 49 references that are in the map. First posted on arXiv 25 Mar 2024.

Builds on

23 of 49 references.

References · 23

  1. 2016Error bounds for approximations with deep ReLU networkscited by 567
  2. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  3. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  4. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 304
  5. 2018Universality of Deep Convolutional Neural Networkscited by 176
  6. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 165
  7. 2020Deep Network Approximation for Smooth Functionscited by 163
  8. 2019Deep Network Approximation Characterized by Number of Neuronscited by 120
  9. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
  10. 2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
  11. 2021Characterization of the Variation Spaces Corresponding to Shallow Neural Networkscited by 48
  12. 2018Fast convergence rates of deep neural networks for classificationcited by 46
11 more
  1. 2019Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networkscited by 40
  2. 2021Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networkscited by 32
  3. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  4. 2020Theory of Deep Convolutional Neural Networks II: Spherical Analysiscited by 22
  5. 2021Theory of Deep Convolutional Neural Networks III: Approximating Radial Functionscited by 22
  6. 2021Convergence rates of deep ReLU networks for multiclass classificationcited by 21
  7. 2022Approximation bounds for norm constrained neural networks with applications to regression and GANscited by 21
  8. 2021Universal Consistency of Deep Convolutional Neural Networkscited by 16
  9. 2023Classification with Deep Neural Networks and Logistic Losscited by 16
  10. 2023Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networkscited by 11
  11. 2022Learning Ability of Interpolating Deep Convolutional Neural Networkscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 6.

10 papers.

  1. 2024On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networkscited by 13
  2. 2024Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Studycited by 13
  3. 2025Nonparametric estimation of a factorizable density using diffusion modelscited by 7
  4. 2026Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networkscited by 3
  5. 2024Dynamic Universal Approximation Theory: The Basic Theory for Deep Learning-Based Computer Vision Modelscited by 0
  6. 2026Kantorovich-Type Stochastic Neural Network Operators for the Mean-Square Approximation of Certain Second-Order Stochastic Processescited by 0
  7. 2026Simultaneous CNN Approximation on Manifolds with Applications to Boundary Value Problemscited by 0
  8. 2026Approximation Error Upper and Lower Bounds for H"older Class with Transformerscited by 0
  9. 2026On the Regularity and Generalization of One-Step Wasserstein-guided Generative Models for PDE-Induced Measurescited by 0
  10. 2026New Fractional Ambiguity Function Integrated with CNN-Based Machine Learning for Signal Classificationcited by 0