# Learning Ability of Interpolating Deep Convolutional Neural Networks

arXiv: https://arxiv.org/abs/2210.14184 · HTML: https://mlsanity.com/paper/2210.14184/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 25 Oct 2022.

## Builds on (most cited first, 18 of 18)

- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017)
- [To understand deep learning we need to understand kernel learning](https://mlsanity.com/paper/1802.01396/) (2018)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [Optimal Approximation with Sparsely Connected Deep Neural Networks](https://mlsanity.com/paper/1705.01714/) (2017)
- [Benign Overfitting in Two-layer Convolutional Neural Networks](https://mlsanity.com/paper/2202.06526/) (2022)
- [Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with  ℓ1  and  ℓ0  Controls](https://mlsanity.com/paper/1607.07819/) (2016)
- [Benign Overfitting of Constant-Stepsize SGD for Linear Regression](https://mlsanity.com/paper/2103.12692/) (2021)
- [Approximation spaces of deep neural networks](https://mlsanity.com/paper/1905.01208/) (2019)
- [Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks](https://mlsanity.com/paper/1903.10047/) (2019)
- [Equivalence of approximation by convolutional neural networks and fully-connected networks](https://mlsanity.com/paper/1809.00973/) (2018)
- [Neural network approximation and estimation of classifiers with classification boundary in a Barron class](https://mlsanity.com/paper/2011.09363/) (2020)
- [Theory of Deep Convolutional Neural Networks III: Approximating Radial Functions](https://mlsanity.com/paper/2107.00896/) (2021)
- [The Interplay Between Implicit Bias and Benign Overfitting in Two-Layer Linear Networks](https://mlsanity.com/paper/2108.11489/) (2021)
- [Universal Consistency of Deep Convolutional Neural Networks](https://mlsanity.com/paper/2106.12498/) (2021)

## Built on by (most cited first, 9 of 9)

- [Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regression](https://mlsanity.com/paper/2304.01561/) (2023)
- [On the rates of convergence for learning with convolutional neural networks](https://mlsanity.com/paper/2403.16459/) (2024)
- [Classification of Data Generated by Gaussian Mixture Models Using Deep ReLU Networks](https://mlsanity.com/paper/2308.08030/) (2023)
- [Higher Order Approximation Rates for ReLU CNNs in Korobov Spaces](https://mlsanity.com/paper/2501.11275/) (2025)
- [Solving PDEs on Spheres with Physics-Informed Convolutional Neural Networks](https://mlsanity.com/paper/2308.09605/) (2023)
- [Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learning](https://mlsanity.com/paper/2406.01435/) (2024)
- [Two-Dimensional Deep ReLU CNN Approximation for Korobov Functions: A Constructive Approach](https://mlsanity.com/paper/2503.07976/) (2025)
- [Enhancing Kernel Flexibility via Learning Asymmetric Locally-Adaptive Kernels](https://mlsanity.com/paper/2310.05236/) (2023)
- [Component-based Sketching for Deep ReLU Nets](https://mlsanity.com/paper/2409.14174/) (2024)

## Abstract (arXiv)

It is frequently observed that overparameterized neural networks generalize well. Regarding such phenomena, existing theoretical work mainly devotes to linear settings or fully-connected neural networks. This paper studies the learning ability of an important family of deep neural networks, deep convolutional neural networks (DCNNs), under both underparameterized and overparameterized settings. We establish the first learning rates of underparameterized DCNNs without parameter or function variable structure restrictions presented in the literature. We also show that by adding well-defined layers to a non-interpolating DCNN, we can obtain some interpolating DCNNs that maintain the good learning rates of the non-interpolating DCNN. This result is achieved by a novel network deepening scheme designed for DCNNs. Our work provides theoretical verification of how overfitted DCNNs generalize well.