# Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1903.10047 · HTML: https://mlsanity.com/paper/1903.10047/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Mar 2019.

## Built on by (most cited first, 30 of 40)

- [Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks](https://mlsanity.com/paper/2109.02832/) (2021)
- [Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data](https://mlsanity.com/paper/2411.06646/) (2024)
- [Theory of Deep Convolutional Neural Networks III: Approximating Radial Functions](https://mlsanity.com/paper/2107.00896/) (2021)
- [Universal Consistency of Deep Convolutional Neural Networks](https://mlsanity.com/paper/2106.12498/) (2021)
- [Approximation Properties of Deep ReLU CNNs](https://mlsanity.com/paper/2109.00190/) (2021)
- [A Brief Survey on the Approximation Theory for Sequence Modelling](https://mlsanity.com/paper/2302.13752/) (2023)
- [Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study](https://mlsanity.com/paper/2410.00357/) (2024)
- [Approximation Theory of Convolutional Architectures for Time Series Modelling](https://mlsanity.com/paper/2107.09355/) (2021)
- [Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?](https://mlsanity.com/paper/2204.09664/) (2022)
- [On the rates of convergence for learning with convolutional neural networks](https://mlsanity.com/paper/2403.16459/) (2024)
- [Deep Neural Network Approximation of Invariant Functions through Dynamical Systems](https://mlsanity.com/paper/2208.08707/) (2022)
- [Learning Ability of Interpolating Deep Convolutional Neural Networks](https://mlsanity.com/paper/2210.14184/) (2022)
- [On the Universal Approximation Property of Deep Fully Convolutional Neural Networks](https://mlsanity.com/paper/2211.14047/) (2022)
- [Double-well Net for Image Segmentation](https://mlsanity.com/paper/2401.00456/) (2023)
- [Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraint](https://mlsanity.com/paper/2206.04569/) (2022)
- [Approximation analysis of CNNs from a feature extraction view](https://mlsanity.com/paper/2210.09041/) (2022)
- [Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights](https://mlsanity.com/paper/2505.03205/) (2025)
- [Non-asymptotic Excess Risk Bounds for Classification with Deep Convolutional Neural Networks](https://mlsanity.com/paper/2105.00292/) (2021)
- [Analysis of convolutional neural network image classifiers in a rotationally symmetric model](https://mlsanity.com/paper/2205.05500/) (2022)
- [Theoretical guarantees for the advantage of GNNs over NNs in generalizing bandlimited functions on Euclidean cubes](https://mlsanity.com/paper/2206.05904/) (2022)
- [Deep Convolutional Neural Networks with Zero-Padding: Feature Extraction and Learning](https://mlsanity.com/paper/2307.16203/) (2023)
- [Connections between Operator-splitting Methods and Deep Neural Networks with Applications in Image Segmentation](https://mlsanity.com/paper/2307.09052/) (2023)
- [Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation](https://mlsanity.com/paper/2406.05320/) (2024)
- [Some Super-approximation Rates of ReLU Neural Networks for Korobov Functions](https://mlsanity.com/paper/2507.10345/) (2025)
- [Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networks](https://mlsanity.com/paper/2605.31152/) (2026)
- [Analysis of convolutional neural network image classifiers in a hierarchical max-pooling model with additional local pooling](https://mlsanity.com/paper/2106.05233/) (2021)
- [Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networks](https://mlsanity.com/paper/2307.01649/) (2023)
- [Statistical Spatially Inhomogeneous Diffusion Inference](https://mlsanity.com/paper/2312.05793/) (2023)
- [Asymptotic Risk of Overparameterized Likelihood Models: Double Descent Theory for Deep Neural Networks](https://mlsanity.com/paper/2103.00500/) (2021)
- [Optimal Convergence Rates of Deep Convolutional Neural Networks: Additive Ridge Functions](https://mlsanity.com/paper/2202.12119/) (2022)

## Abstract (arXiv)

Convolutional neural networks (CNNs) have been shown to achieve optimal approximation and estimation error rates (in minimax sense) in several function classes. However, previous analyzed optimal CNNs are unrealistically wide and difficult to obtain via optimization due to sparse constraints in important function classes, including the H"older class. We show a ResNet-type CNN can attain the minimax optimal error rates in these classes in more plausible situations -- it can be dense, and its width, channel size, and filter size are constant with respect to sample size. The key idea is that we can replicate the learning ability of Fully-connected neural networks (FNNs) by tailored CNNs, as long as the FNNs have textitblock-sparse structures. Our theory is general in a sense that we can automatically translate any approximation rate achieved by block-sparse FNNs into that by CNNs. As an application, we derive approximation and estimation error rates of the aformentioned type of CNNs for the Barron and H"older classes with the same strategy.