mlsanityData to 7 Oct 2026

Theory of Deep Convolutional Neural Networks III: Approximating Radial Functions

arXiv 2107.00896 ↗First posted 2 Jul 2021cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

We consider a family of deep neural networks consisting of two groups of convolutional layers, a downsampling operator, and a fully connected layer. The network structure depends on two structural parameters which determine the numbers of convolutional layers and the width of the fully connected layer. We establish an approximation theory with explicit approximation rates when the approximated function takes a composite form fcirc Q with a feature polynomial Q and a univariate function f. In particular, we prove that such a network can outperform fully connected shallow networks in approximating radial functions with Q(x) =|x|2, when the dimension d of data from ℝd is large. This gives the first rigorous proof for the superiority of deep convolutional neural networks in approximating functions with special structures. Then we carry out generalization analysis for empirical risk minimization with such a deep network in a regression framework with the regression function of the form fcirc Q. Our network structure which does not use any composite information or the functions Q and f can automatically extract features and make use of the composite nature of the regression function via tuning the structural parameters. Our analysis provides an error bound which decreases with the network depth to a minimum and then increases, verifying theoretically a trade-off phenomenon observed for network depths in many practical applications.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 22 of its 41 references that are in the map. First posted on arXiv 2 Jul 2021.

Builds on

22 of 41 references.

References · 22

  1. 2016Error bounds for approximations with deep ReLU networkscited by 567
  2. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  3. 2016Benefits of depth in neural networkscited by 248
  4. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 176
  5. 2018Universality of Deep Convolutional Neural Networkscited by 176
  6. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
  7. 2017Optimal Approximation with Sparsely Connected Deep Neural Networkscited by 90
  8. 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
  9. 2016Understanding Deep Convolutional Networkscited by 76
  10. 2019Theoretical Issues in Deep Networks: Approximation, Optimization and Generalizationcited by 72
  11. 2015Neural Network with Unbounded Activation Functions is Universal Approximatorcited by 63
  12. 2016Deep vs. shallow networks : An approximation theory perspectivecited by 63
10 more
  1. 2015Provable approximation properties for deep neural networkscited by 60
  2. 2016Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networkscited by 58
  3. 2018Deep Neural Networks Learn Non-Smooth Functions Effectivelycited by 57
  4. 2019Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networkscited by 40
  5. 2018Equivalence of approximation by convolutional neural networks and fully-connected networkscited by 38
  6. 2017Stochastic Configuration Networks: Fundamentals and Algorithmscited by 34
  7. 2020Theory of Deep Convolutional Neural Networks II: Spherical Analysiscited by 22
  8. 2019Deep Neural Networks for Rotation-Invariance Approximation and Learningcited by 10
  9. 2019Adaptive Approximation and Generalization of Deep Neural Network with Intrinsic Dimensionalitycited by 9
  10. 2014Consistency Analysis of an Empirical Minimum Error Entropy Algorithmcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 8; 2024: 5; 2025: 2; 2026: 3.

22 papers.

Sort:

  1. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  2. 2023Rates of Approximation by ReLU Shallow Neural Networkscited by 12
  3. 2023The Hessian perspective into the Nature of Convolutional Neural Networkscited by 10
  4. 2024On the rates of convergence for learning with convolutional neural networkscited by 10
  5. 2022Learning Ability of Interpolating Deep Convolutional Neural Networkscited by 9
  6. 2023Expressivity and Approximation Properties of Deep Neural Networks with ReLUk Activationcited by 8
  7. 2024Nonlinear functional regression by functional deep neural network with kernel embeddingcited by 7
  8. 2022Approximation analysis of CNNs from a feature extraction viewcited by 6
  9. 2022Attention Enables Zero Approximation Errorcited by 4
  10. 2023Exploring the Complexity of Deep Neural Networks through Functional Equivalencecited by 4
  11. 2023Learning Theory of Distribution Regression with Neural Networkscited by 3
  12. 2023Solving PDEs on Spheres with Physics-Informed Convolutional Neural Networkscited by 3
10 more
  1. 2024Spherical Analysis of Learning Nonlinear Functionalscited by 3
  2. 2024Convergence Analysis for Deep Sparse Coding via Convolutional Neural Networkscited by 2
  3. 2022Optimal Convergence Rates of Deep Convolutional Neural Networks: Additive Ridge Functionscited by 1
  4. 2023Optimal Approximation and Learning Rates for Deep Convolutional Neural Networkscited by 1
  5. 2024Performance of computer vision algorithms for fine-grained classification using crowdsourced insect imagescited by 1
  6. 2025Adaptive Classification of Interval-Valued Time Seriescited by 1
  7. 2026Does Sparse Connectivity Improve Generalization? Convolutional Networks Below the Edge of Stabilitycited by 1
  8. 2026Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unitycited by 1
  9. 2025Generalization Analysis for Classification on Korobov Spacecited by 0
  10. 2026Sparse-Aware Neural Networks for Nonlinear Functionals: Mitigating the Exponential Dependence on Dimensioncited by 0