# Theory of Deep Convolutional Neural Networks III: Approximating Radial Functions

arXiv: https://arxiv.org/abs/2107.00896 · HTML: https://mlsanity.com/paper/2107.00896/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 41 references that are in the map.
- First posted on arXiv 2 Jul 2021.

## Builds on (most cited first, 22 of 22)

- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Benefits of depth in neural networks](https://mlsanity.com/paper/1602.04485/) (2016)
- [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality](https://mlsanity.com/paper/1810.08033/) (2018)
- [Optimal Approximation with Sparsely Connected Deep Neural Networks](https://mlsanity.com/paper/1705.01714/) (2017)
- [Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with  ℓ1  and  ℓ0  Controls](https://mlsanity.com/paper/1607.07819/) (2016)
- [Understanding Deep Convolutional Networks](https://mlsanity.com/paper/1601.04920/) (2016)
- [Theoretical Issues in Deep Networks: Approximation, Optimization and Generalization](https://mlsanity.com/paper/1908.09375/) (2019)
- [Neural Network with Unbounded Activation Functions is Universal Approximator](https://mlsanity.com/paper/1505.03654/) (2015)
- [Deep vs. shallow networks : An approximation theory perspective](https://mlsanity.com/paper/1608.03287/) (2016)
- [Provable approximation properties for deep neural networks](https://mlsanity.com/paper/1509.07385/) (2015)
- [Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks](https://mlsanity.com/paper/1610.09887/) (2016)
- [Deep Neural Networks Learn Non-Smooth Functions Effectively](https://mlsanity.com/paper/1802.04474/) (2018)
- [Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks](https://mlsanity.com/paper/1903.10047/) (2019)
- [Equivalence of approximation by convolutional neural networks and fully-connected networks](https://mlsanity.com/paper/1809.00973/) (2018)
- [Stochastic Configuration Networks: Fundamentals and Algorithms](https://mlsanity.com/paper/1702.03180/) (2017)
- [Theory of Deep Convolutional Neural Networks II: Spherical Analysis](https://mlsanity.com/paper/2007.14285/) (2020)
- [Deep Neural Networks for Rotation-Invariance Approximation and Learning](https://mlsanity.com/paper/1904.01814/) (2019)
- [Adaptive Approximation and Generalization of Deep Neural Network with Intrinsic Dimensionality](https://mlsanity.com/paper/1907.02177/) (2019)
- [Consistency Analysis of an Empirical Minimum Error Entropy Algorithm](https://mlsanity.com/paper/1412.5272/) (2014)

## Built on by (most cited first, 22 of 22)

- [Nonparametric regression using over-parameterized shallow ReLU neural networks](https://mlsanity.com/paper/2306.08321/) (2023)
- [Rates of Approximation by ReLU Shallow Neural Networks](https://mlsanity.com/paper/2307.12461/) (2023)
- [The Hessian perspective into the Nature of Convolutional Neural Networks](https://mlsanity.com/paper/2305.09088/) (2023)
- [On the rates of convergence for learning with convolutional neural networks](https://mlsanity.com/paper/2403.16459/) (2024)
- [Learning Ability of Interpolating Deep Convolutional Neural Networks](https://mlsanity.com/paper/2210.14184/) (2022)
- [Expressivity and Approximation Properties of Deep Neural Networks with ReLUk Activation](https://mlsanity.com/paper/2312.16483/) (2023)
- [Nonlinear functional regression by functional deep neural network with kernel embedding](https://mlsanity.com/paper/2401.02890/) (2024)
- [Approximation analysis of CNNs from a feature extraction view](https://mlsanity.com/paper/2210.09041/) (2022)
- [Attention Enables Zero Approximation Error](https://mlsanity.com/paper/2202.12166/) (2022)
- [Exploring the Complexity of Deep Neural Networks through Functional Equivalence](https://mlsanity.com/paper/2305.11417/) (2023)
- [Spherical Analysis of Learning Nonlinear Functionals](https://mlsanity.com/paper/2410.01047/) (2024)
- [Learning Theory of Distribution Regression with Neural Networks](https://mlsanity.com/paper/2307.03487/) (2023)
- [Solving PDEs on Spheres with Physics-Informed Convolutional Neural Networks](https://mlsanity.com/paper/2308.09605/) (2023)
- [Convergence Analysis for Deep Sparse Coding via Convolutional Neural Networks](https://mlsanity.com/paper/2408.05540/) (2024)
- [Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unity](https://mlsanity.com/paper/2605.08811/) (2026)
- [Optimal Convergence Rates of Deep Convolutional Neural Networks: Additive Ridge Functions](https://mlsanity.com/paper/2202.12119/) (2022)
- [Optimal Approximation and Learning Rates for Deep Convolutional Neural Networks](https://mlsanity.com/paper/2308.03259/) (2023)
- [Performance of computer vision algorithms for fine-grained classification using crowdsourced insect images](https://mlsanity.com/paper/2404.03474/) (2024)
- [Adaptive Classification of Interval-Valued Time Series](https://mlsanity.com/paper/2504.03318/) (2025)
- [Does Sparse Connectivity Improve Generalization? Convolutional Networks Below the Edge of Stability](https://mlsanity.com/paper/2603.04807/) (2026)
- [Sparse-Aware Neural Networks for Nonlinear Functionals: Mitigating the Exponential Dependence on Dimension](https://mlsanity.com/paper/2604.06774/) (2026)
- [Generalization Analysis for Classification on Korobov Space](https://mlsanity.com/paper/2509.22748/) (2025)

## Abstract (arXiv)

We consider a family of deep neural networks consisting of two groups of convolutional layers, a downsampling operator, and a fully connected layer. The network structure depends on two structural parameters which determine the numbers of convolutional layers and the width of the fully connected layer. We establish an approximation theory with explicit approximation rates when the approximated function takes a composite form fcirc Q with a feature polynomial Q and a univariate function f. In particular, we prove that such a network can outperform fully connected shallow networks in approximating radial functions with Q(x) =|x|2, when the dimension d of data from ℝd is large. This gives the first rigorous proof for the superiority of deep convolutional neural networks in approximating functions with special structures. Then we carry out generalization analysis for empirical risk minimization with such a deep network in a regression framework with the regression function of the form fcirc Q. Our network structure which does not use any composite information or the functions Q and f can automatically extract features and make use of the composite nature of the regression function via tuning the structural parameters. Our analysis provides an error bound which decreases with the network depth to a minimum and then increases, verifying theoretically a trade-off phenomenon observed for network depths in many practical applications.