mlsanityData to 7 Oct 2026

Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks

arXiv 1903.10047 ↗First posted 24 Mar 2019stat.ML cs.LG math.STcited by 40

Not on the map. Why?

Abstract (arXiv)

Convolutional neural networks (CNNs) have been shown to achieve optimal approximation and estimation error rates (in minimax sense) in several function classes. However, previous analyzed optimal CNNs are unrealistically wide and difficult to obtain via optimization due to sparse constraints in important function classes, including the H"older class. We show a ResNet-type CNN can attain the minimax optimal error rates in these classes in more plausible situations -- it can be dense, and its width, channel size, and filter size are constant with respect to sample size. The key idea is that we can replicate the learning ability of Fully-connected neural networks (FNNs) by tailored CNNs, as long as the FNNs have textitblock-sparse structures. Our theory is general in a sense that we can automatically translate any approximation rate achieved by block-sparse FNNs into that by CNNs. As an application, we derive approximation and estimation error rates of the aformentioned type of CNNs for the Barron and H"older classes with the same strategy.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 10; 2023: 6; 2024: 4; 2025: 6; 2026: 6.

40 papers.

Sort:

  1. 2021Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networkscited by 32
  2. 2024Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Datacited by 30
  3. 2021Theory of Deep Convolutional Neural Networks III: Approximating Radial Functionscited by 22
  4. 2021Universal Consistency of Deep Convolutional Neural Networkscited by 16
  5. 2021Approximation Properties of Deep ReLU CNNscited by 15
  6. 2023A Brief Survey on the Approximation Theory for Sequence Modellingcited by 15
  7. 2024Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Studycited by 13
  8. 2021Approximation Theory of Convolutional Architectures for Time Series Modellingcited by 10
  9. 2022Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?cited by 10
  10. 2024On the rates of convergence for learning with convolutional neural networkscited by 10
  11. 2022Learning Ability of Interpolating Deep Convolutional Neural Networkscited by 9
  12. 2022Deep Neural Network Approximation of Invariant Functions through Dynamical Systemscited by 8
28 more
  1. 2023Double-well Net for Image Segmentationcited by 7
  2. 2022Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraintcited by 6
  3. 2022Approximation analysis of CNNs from a feature extraction viewcited by 6
  4. 2022On the Universal Approximation Property of Deep Fully Convolutional Neural Networkscited by 6
  5. 2021Non-asymptotic Excess Risk Bounds for Classification with Deep Convolutional Neural Networkscited by 4
  6. 2022Analysis of convolutional neural network image classifiers in a rotationally symmetric modelcited by 4
  7. 2022Theoretical guarantees for the advantage of GNNs over NNs in generalizing bandlimited functions on Euclidean cubescited by 4
  8. 2023Deep Convolutional Neural Networks with Zero-Padding: Feature Extraction and Learningcited by 4
  9. 2023Connections between Operator-splitting Methods and Deep Neural Networks with Applications in Image Segmentationcited by 3
  10. 2024Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimationcited by 3
  11. 2025Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insightscited by 3
  12. 2025Some Super-approximation Rates of ReLU Neural Networks for Korobov Functionscited by 3
  13. 2026Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networkscited by 3
  14. 2021Analysis of convolutional neural network image classifiers in a hierarchical max-pooling model with additional local poolingcited by 2
  15. 2023Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networkscited by 2
  16. 2023Statistical Spatially Inhomogeneous Diffusion Inferencecited by 2
  17. 2021Asymptotic Risk of Overparameterized Likelihood Models: Double Descent Theory for Deep Neural Networkscited by 1
  18. 2022Optimal Convergence Rates of Deep Convolutional Neural Networks: Additive Ridge Functionscited by 1
  19. 2022Neural Network Approximation of Continuous Functions in High Dimensions with Applications to Inverse Problemscited by 1
  20. 2025Fr'echet Cumulative Covariance Net for Deep Nonlinear Sufficient Dimension Reduction with Random Objectscited by 1
  21. 2026Does Sparse Connectivity Improve Generalization? Convolutional Networks Below the Edge of Stabilitycited by 1
  22. 2025Parallel Layer Normalization for Universal Approximationcited by 0
  23. 2025A General Method for Proving Networks Universal Approximation Propertycited by 0
  24. 2025Data-Driven Model Reduction using WeldNet: Windowed Encoders for Learning Dynamicscited by 0
  25. 2026Progressive Approximation in Deep Residual Networks: Theory and Validationcited by 0
  26. 2026Neural Flow Operators can Approximate any Operator: Abstract Frameworks and Universal Approximationscited by 0
  27. 2026Provable Data Scaling Law for Meta Learning via Complexity Minimizationcited by 0
  28. 2026Mitigating the Curse of Dimensionality in Uniform Convergence of Deep Neural Networks via Smooth Activationscited by 0