mlsanityData to 7 Oct 2026

Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space

arXiv 1910.12799 ↗First posted 28 Oct 2019stat.ML cs.LGcited by 46

Not on the map. Why?

Abstract (arXiv)

Deep learning has exhibited superior performance for various tasks, especially for high-dimensional datasets, such as images. To understand this property, we investigate the approximation and estimation ability of deep learning on anisotropic Besov spaces. The anisotropic Besov space is characterized by direction-dependent smoothness and includes several function classes that have been investigated thus far. We demonstrate that the approximation error and estimation error of deep learning only depend on the average value of the smoothness parameters in all directions. Consequently, the curse of dimensionality can be avoided if the smoothness of the target function is highly anisotropic. Unlike existing studies, our analysis does not require a low-dimensional structure of the input data. We also investigate the minimax optimality of deep learning and compare its performance with that of the kernel method (more generally, linear estimators). The results show that deep learning has better dependence on the input dimensionality if the target function possesses anisotropic smoothness, and it achieves an adaptive rate for functions with spatially inhomogeneous smoothness.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 8; 2023: 9; 2024: 7; 2025: 9; 2026: 9.

46 papers.

Sort:

By area:

  1. 2024Transformers are Minimax Optimal Nonparametric In-Context Learnerscited by 36
  2. 2021Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networkscited by 32
  3. 2023Approximation and Estimation Ability of Transformers for Sequence-to-Sequence Functions with Infinite Dimensional Inputcited by 32
  4. 2023Generalization Ability of Wide Neural Networks on ℝcited by 28
  5. 2024A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Modelscited by 20
  6. 2021On the rate of convergence of a classifier based on a Transformer encodercited by 16
  7. 2022Adaptive deep learning for nonlinear time series modelscited by 11
  8. 2022Understanding Deep Neural Function Approximation in Reinforcement Learning via ε-Greedy Explorationcited by 11
  9. 2022Excess Risk of Two-Layer ReLU Neural Networks in Teacher-Student Settings and its Superiority to Kernel Methodscited by 10
  10. 2024Posterior and variational inference for deep neural networks with heavy-tailed weightscited by 9
  11. 2024Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networkscited by 8
  12. 2026Stable and Explainable Personality Trait Evaluation in Large Language Models with Internal Activationscited by 8
34 more
  1. 2022On the inability of Gaussian process regression to optimally learn compositional functionscited by 7
  2. 2022Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraintcited by 6
  3. 2023Sup-Norm Convergence of Deep Neural Network Estimator for Nonparametric Regression by Adversarial Trainingcited by 6
  4. 2021Estimation of a regression function on a manifold by fully connected deep neural networkscited by 5
  5. 2024State Space Models are Provably Comparable to Transformers in Dynamic Token Selectioncited by 5
  6. 2025Nested Expectations with Kernel Quadraturecited by 5
  7. 2022Analysis of convolutional neural network image classifiers in a rotationally symmetric modelcited by 4
  8. 2022Predicting Out-of-Domain Generalization with Neighborhood Invariancecited by 4
  9. 2022Smooth Sailing: Improving Active Learning for Pre-trained Language Models with Representation Smoothness Analysiscited by 4
  10. 2023Exploring the Approximation Capabilities of Multiplicative Neural Networks for Smooth Functionscited by 4
  11. 2023Analysis of the expected L2 error of an over-parametrized deep neural network estimate learned by gradient descent without regularizationcited by 4
  12. 2024Generalization Error of Graph Neural Networks in the Mean-field Regimecited by 4
  13. 2025Provable Benefit of Curriculum in Transformer Tree-Reasoning Post-Trainingcited by 4
  14. 2023Solving PDEs on Spheres with Physics-Informed Convolutional Neural Networkscited by 3
  15. 2025Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regressioncited by 3
  16. 2026Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networkscited by 3
  17. 2021Analysis of convolutional neural network image classifiers in a hierarchical max-pooling model with additional local poolingcited by 2
  18. 2023Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networkscited by 2
  19. 2023On the rate of convergence of an over-parametrized Transformer classifier learned by gradient descentcited by 2
  20. 2024Approximation Bounds for Recurrent Neural Networks with Application to Regressioncited by 2
  21. 2025Meta Optimality for Demographic Parity Constrained Regression via Post-Processingcited by 2
  22. 2025Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionalitycited by 2
  23. 2025Expressive Power of Deep Networks on Manifolds: Simultaneous Approximationcited by 2
  24. 2025On the Rate of Convergence of Kolmogorov-Arnold Network Regression Estimatorscited by 2
  25. 2023Neural Network Approximation for Pessimistic Offline Reinforcement Learningcited by 1
  26. 2026On Interpolation Formulas Describing Neural Network Generalizationcited by 1
  27. 2025Transfer Learning through Enhanced Sufficient Representation: Enriching Source Domain Knowledge with Target Datacited by 0
  28. 2025Distributional Evaluation of Generative Models via Relative Density Ratiocited by 0
  29. 2026Provable Target Sample Complexity Improvements as Pre-Trained Models Scalecited by 0
  30. 2026A Theory of Nonparametric Covariance Function Estimation for Discretely Observed Datacited by 0
  31. 2026Posterior Contraction Rates for Sparse Kolmogorov-Arnold Networks in Anisotropic Besov Spacescited by 0
  32. 2026Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Widthcited by 0
  33. 2026Vector Search As Nearest Neighbor Matching: RAG-based Policy Learning in Causal Inferencecited by 0
  34. 2026Minimax rates for learning spectral Barron functions by deep ReLU neural networkscited by 0