mlsanityData to 14 Oct 2026

Learning Ability of Interpolating Deep Convolutional Neural Networks

arXiv 2210.14184 ↗First posted 25 Oct 2022stat.ML cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

It is frequently observed that overparameterized neural networks generalize well. Regarding such phenomena, existing theoretical work mainly devotes to linear settings or fully-connected neural networks. This paper studies the learning ability of an important family of deep neural networks, deep convolutional neural networks (DCNNs), under both underparameterized and overparameterized settings. We establish the first learning rates of underparameterized DCNNs without parameter or function variable structure restrictions presented in the literature. We also show that by adding well-defined layers to a non-interpolating DCNN, we can obtain some interpolating DCNNs that maintain the good learning rates of the non-interpolating DCNN. This result is achieved by a novel network deepening scheme designed for DCNNs. Our work provides theoretical verification of how overfitted DCNNs generalize well.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 18 of its 35 references that are in the map. First posted on arXiv 25 Oct 2022.

Builds on

18 of 35 references.

References · 18

  1. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  2. 2016Error bounds for approximations with deep ReLU networkscited by 574
  3. 2019Benign Overfitting in Linear Regressioncited by 549
  4. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 433
  5. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 311
  6. 2018To understand deep learning we need to understand kernel learningcited by 187
  7. 2018Universality of Deep Convolutional Neural Networkscited by 176
  8. 2017Optimal Approximation with Sparsely Connected Deep Neural Networkscited by 91
  9. 2022Benign Overfitting in Two-layer Convolutional Neural Networkscited by 91
  10. 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
  11. 2021Benign Overfitting of Constant-Stepsize SGD for Linear Regressioncited by 71
  12. 2019Approximation spaces of deep neural networkscited by 51
6 more
  1. 2019Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networkscited by 40
  2. 2018Equivalence of approximation by convolutional neural networks and fully-connected networkscited by 38
  3. 2020Neural network approximation and estimation of classifiers with classification boundary in a Barron classcited by 28
  4. 2021Theory of Deep Convolutional Neural Networks III: Approximating Radial Functionscited by 22
  5. 2021The Interplay Between Implicit Bias and Benign Overfitting in Two-Layer Linear Networkscited by 18
  6. 2021Universal Consistency of Deep Convolutional Neural Networkscited by 16

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 3; 2025: 2.

9 papers.

  1. 2023Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regressioncited by 15
  2. 2024On the rates of convergence for learning with convolutional neural networkscited by 10
  3. 2023Classification of Data Generated by Gaussian Mixture Models Using Deep ReLU Networkscited by 4
  4. 2025Higher Order Approximation Rates for ReLU CNNs in Korobov Spacescited by 4
  5. 2023Solving PDEs on Spheres with Physics-Informed Convolutional Neural Networkscited by 3
  6. 2024Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learningcited by 2
  7. 2025Two-Dimensional Deep ReLU CNN Approximation for Korobov Functions: A Constructive Approachcited by 2
  8. 2023Enhancing Kernel Flexibility via Learning Asymmetric Locally-Adaptive Kernelscited by 1
  9. 2024Component-based Sketching for Deep ReLU Netscited by 1