# Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks

arXiv: https://arxiv.org/abs/2109.02832 · HTML: https://mlsanity.com/paper/2109.02832/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 79 references that are in the map.
- First posted on arXiv 7 Sep 2021.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [ResNeSt: Split-Attention Networks](https://mlsanity.com/paper/2004.08955/) (2020)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [Recent Trends in Deep Learning Based Natural Language Processing](https://mlsanity.com/paper/1708.02709/) (2017)
- [OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks](https://mlsanity.com/paper/1312.6229/) (2013)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality](https://mlsanity.com/paper/1810.08033/) (2018)
- [Deep ReLU network approximation of functions on a manifold](https://mlsanity.com/paper/1908.00695/) (2019)
- [Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recovery](https://mlsanity.com/paper/1908.01842/) (2019)
- [Provable approximation properties for deep neural networks](https://mlsanity.com/paper/1509.07385/) (2015)
- [On the Intrinsic Dimensionality of Image Representations](https://mlsanity.com/paper/1803.09672/) (2018)
- [Error bounds for deep ReLU networks using the Kolmogorov--Arnold superposition theorem](https://mlsanity.com/paper/1906.11945/) (2019)
- [Fast convergence rates of deep neural networks for classification](https://mlsanity.com/paper/1812.03599/) (2018)
- [Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space](https://mlsanity.com/paper/1910.12799/) (2019)
- [Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks](https://mlsanity.com/paper/1903.10047/) (2019)
- [Equivalence of approximation by convolutional neural networks and fully-connected networks](https://mlsanity.com/paper/1809.00973/) (2018)
- [Learning Deep ResNet Blocks Sequentially using Boosting Theory](https://mlsanity.com/paper/1706.04964/) (2017)
- [Smooth function approximation by deep neural networks with general activation functions](https://mlsanity.com/paper/1906.06903/) (2019)
- [Deep nets for local manifold learning](https://mlsanity.com/paper/1607.07110/) (2016)

## Built on by (most cited first, 30 of 32)

- [Deep Nonparametric Estimation of Operators between Infinite Dimensional Spaces](https://mlsanity.com/paper/2201.00217/) (2022)
- [Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data](https://mlsanity.com/paper/2411.06646/) (2024)
- [Deep Operator Learning Lessens the Curse of Dimensionality for PDEs](https://mlsanity.com/paper/2301.12227/) (2023)
- [Hardness of Learning Neural Networks under the Manifold Hypothesis](https://mlsanity.com/paper/2406.01461/) (2024)
- [Understanding In-Context Learning on Structured Manifolds: Bridging Attention to Kernel Methods](https://mlsanity.com/paper/2506.10959/) (2025)
- [Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study](https://mlsanity.com/paper/2410.00357/) (2024)
- [Sample Complexity of Nonparametric Off-Policy Evaluation on Low-Dimensional Manifolds using Deep Networks](https://mlsanity.com/paper/2206.02887/) (2022)
- [Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model Reduction for Operator Learning](https://mlsanity.com/paper/2401.10490/) (2024)
- [Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes](https://mlsanity.com/paper/2406.06838/) (2024)
- [Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?](https://mlsanity.com/paper/2204.09664/) (2022)
- [Deep Nonparametric Estimation of Intrinsic Data Structures by Chart Autoencoders: Generalization Error and Robustness](https://mlsanity.com/paper/2303.09863/) (2023)
- [On the rates of convergence for learning with convolutional neural networks](https://mlsanity.com/paper/2403.16459/) (2024)
- [Double-well Net for Image Segmentation](https://mlsanity.com/paper/2401.00456/) (2023)
- [Side Effects of Learning from Low-dimensional Data Embedded in a Euclidean Space](https://mlsanity.com/paper/2203.00614/) (2022)
- [Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraint](https://mlsanity.com/paper/2206.04569/) (2022)
- [Approximation analysis of CNNs from a feature extraction view](https://mlsanity.com/paper/2210.09041/) (2022)
- [Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights](https://mlsanity.com/paper/2505.03205/) (2025)
- [Analysis of convolutional neural network image classifiers in a rotationally symmetric model](https://mlsanity.com/paper/2205.05500/) (2022)
- [Linear Regression on Manifold Structured Data: the Impact of Extrinsic Geometry on Solutions](https://mlsanity.com/paper/2307.02478/) (2023)
- [Connections between Operator-splitting Methods and Deep Neural Networks with Applications in Image Segmentation](https://mlsanity.com/paper/2307.09052/) (2023)
- [Solving PDEs on Spheres with Physics-Informed Convolutional Neural Networks](https://mlsanity.com/paper/2308.09605/) (2023)
- [Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation](https://mlsanity.com/paper/2406.05320/) (2024)
- [Coefficient-to-Basis Network: A Fine-Tunable Operator Learning Framework for Inverse Problems with Adaptive Discretizations and Theoretical Guarantees](https://mlsanity.com/paper/2503.08642/) (2025)
- [Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networks](https://mlsanity.com/paper/2307.01649/) (2023)
- [Causal inference through multi-stage learning and doubly robust deep neural networks](https://mlsanity.com/paper/2407.08560/) (2024)
- [Convergence Analysis for Deep Sparse Coding via Convolutional Neural Networks](https://mlsanity.com/paper/2408.05540/) (2024)
- [Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unity](https://mlsanity.com/paper/2605.08811/) (2026)
- [Effective Minkowski Dimension of Deep Nonparametric Regression: Function Approximation and Statistical Theories](https://mlsanity.com/paper/2306.14859/) (2023)
- [Local EGOP for Continuous Index Learning](https://mlsanity.com/paper/2601.07061/) (2026)
- [Does Sparse Connectivity Improve Generalization? Convolutional Networks Below the Edge of Stability](https://mlsanity.com/paper/2603.04807/) (2026)

## Abstract (arXiv)

Most of existing statistical theories on deep neural networks have sample complexities cursed by the data dimension and therefore cannot well explain the empirical success of deep learning on high-dimensional data. To bridge this gap, we propose to exploit low-dimensional geometric structures of the real world data sets. We establish theoretical guarantees of convolutional residual networks (ConvResNet) in terms of function approximation and statistical estimation for binary classification. Specifically, given the data lying on a d-dimensional manifold isometrically embedded in ℝD, we prove that if the network architecture is properly chosen, ConvResNets can (1) approximate Besov functions on manifolds with arbitrary accuracy, and (2) learn a classifier by minimizing the empirical logistic risk, which gives an excess risk in the order of n^-fracs2s+2(svee d), where s is a smoothness parameter. This implies that the sample complexity depends on the intrinsic dimension d, instead of the data dimension D. Our results demonstrate that ConvResNets are adaptive to low-dimensional structures of data sets.