mlsanityData to 7 Oct 2026

Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks

arXiv 2109.02832 ↗First posted 7 Sep 2021stat.ML cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Most of existing statistical theories on deep neural networks have sample complexities cursed by the data dimension and therefore cannot well explain the empirical success of deep learning on high-dimensional data. To bridge this gap, we propose to exploit low-dimensional geometric structures of the real world data sets. We establish theoretical guarantees of convolutional residual networks (ConvResNet) in terms of function approximation and statistical estimation for binary classification. Specifically, given the data lying on a d-dimensional manifold isometrically embedded in ℝD, we prove that if the network architecture is properly chosen, ConvResNets can (1) approximate Besov functions on manifolds with arbitrary accuracy, and (2) learn a classifier by minimizing the empirical logistic risk, which gives an excess risk in the order of n^-fracs2s+2(svee d), where s is a smoothness parameter. This implies that the sample complexity depends on the intrinsic dimension d, instead of the data dimension D. Our results demonstrate that ConvResNets are adaptive to low-dimensional structures of data sets.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 37 of its 79 references that are in the map. First posted on arXiv 7 Sep 2021.

Builds on

37 of 79 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  4. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  5. 2016Wide Residual Networkscited by 3,057
  6. 2015Fast R-CNNcited by 2,930
  7. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  8. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  9. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  10. 2020What is the State of Neural Network Pruning?cited by 578
  11. 2016Error bounds for approximations with deep ReLU networkscited by 567
  12. 2020ResNeSt: Split-Attention Networkscited by 445
25 more
  1. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  2. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 318
  3. 2017Recent Trends in Deep Learning Based Natural Language Processingcited by 262
  4. 2013OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networkscited by 224
  5. 2018Universality of Deep Convolutional Neural Networkscited by 176
  6. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
  7. 2019Deep ReLU network approximation of functions on a manifoldcited by 73
  8. 2019Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recoverycited by 73
  9. 2015Provable approximation properties for deep neural networkscited by 60
  10. 2018On the Intrinsic Dimensionality of Image Representationscited by 57
  11. 2019Error bounds for deep ReLU networks using the Kolmogorov--Arnold superposition theoremcited by 49
  12. 2018Fast convergence rates of deep neural networks for classificationcited by 46
  13. 2019Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov spacecited by 46
  14. 2019Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networkscited by 40
  15. 2018Equivalence of approximation by convolutional neural networks and fully-connected networkscited by 38
  16. 2017Learning Deep ResNet Blocks Sequentially using Boosting Theorycited by 32
  17. 2019Smooth function approximation by deep neural networks with general activation functionscited by 29
  18. 2016Deep nets for local manifold learningcited by 26
  19. 2017On the ability of neural nets to express distributionscited by 26
  20. 2020Theory of Deep Convolutional Neural Networks II: Spherical Analysiscited by 22
  21. 2018Functional Gradient Boosting based on Residual Network Perceptioncited by 14
  22. 2020Sharp Rate of Convergence for Deep Neural Network Classifiers under the Teacher-Student Settingcited by 11
  23. 2020A deep network construction that adapts to intrinsic dimensionality beyond the domaincited by 10
  24. 2019Adaptive Approximation and Generalization of Deep Neural Network with Intrinsic Dimensionalitycited by 9
  25. 2020Doubly Robust Off-Policy Learning on Low-Dimensional Manifolds by Deep Neural Networkscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 8; 2024: 9; 2025: 3; 2026: 5.

32 papers.

Sort:

  1. 2022Deep Nonparametric Estimation of Operators between Infinite Dimensional Spacescited by 45
  2. 2024Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Datacited by 30
  3. 2023Deep Operator Learning Lessens the Curse of Dimensionality for PDEscited by 18
  4. 2024Hardness of Learning Neural Networks under the Manifold Hypothesiscited by 18
  5. 2024Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Studycited by 13
  6. 2025Understanding In-Context Learning on Structured Manifolds: Bridging Attention to Kernel Methodscited by 13
  7. 2022Sample Complexity of Nonparametric Off-Policy Evaluation on Low-Dimensional Manifolds using Deep Networkscited by 12
  8. 2024Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model Reduction for Operator Learningcited by 12
  9. 2024Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizescited by 12
  10. 2022Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?cited by 10
  11. 2023Deep Nonparametric Estimation of Intrinsic Data Structures by Chart Autoencoders: Generalization Error and Robustnesscited by 10
  12. 2024On the rates of convergence for learning with convolutional neural networkscited by 10
20 more
  1. 2023Double-well Net for Image Segmentationcited by 7
  2. 2022Side Effects of Learning from Low-dimensional Data Embedded in a Euclidean Spacecited by 6
  3. 2022Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraintcited by 6
  4. 2022Approximation analysis of CNNs from a feature extraction viewcited by 6
  5. 2022Analysis of convolutional neural network image classifiers in a rotationally symmetric modelcited by 4
  6. 2023Linear Regression on Manifold Structured Data: the Impact of Extrinsic Geometry on Solutionscited by 3
  7. 2023Connections between Operator-splitting Methods and Deep Neural Networks with Applications in Image Segmentationcited by 3
  8. 2023Solving PDEs on Spheres with Physics-Informed Convolutional Neural Networkscited by 3
  9. 2024Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimationcited by 3
  10. 2025Coefficient-to-Basis Network: A Fine-Tunable Operator Learning Framework for Inverse Problems with Adaptive Discretizations and Theoretical Guaranteescited by 3
  11. 2025Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insightscited by 3
  12. 2023Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networkscited by 2
  13. 2024Causal inference through multi-stage learning and doubly robust deep neural networkscited by 2
  14. 2024Convergence Analysis for Deep Sparse Coding via Convolutional Neural Networkscited by 2
  15. 2023Effective Minkowski Dimension of Deep Nonparametric Regression: Function Approximation and Statistical Theoriescited by 1
  16. 2026Does Sparse Connectivity Improve Generalization? Convolutional Networks Below the Edge of Stabilitycited by 1
  17. 2026Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unitycited by 1
  18. 2026Local EGOP for Continuous Index Learningcited by 0
  19. 2026Sparse-Aware Neural Networks for Nonlinear Functionals: Mitigating the Exponential Dependence on Dimensioncited by 0
  20. 2026Mitigating the Curse of Dimensionality in Uniform Convergence of Deep Neural Networks via Smooth Activationscited by 0