# Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space

arXiv: https://arxiv.org/abs/1910.12799 · HTML: https://mlsanity.com/paper/1910.12799/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2019.

## Built on by (most cited first, 30 of 46)

- [Transformers are Minimax Optimal Nonparametric In-Context Learners](https://mlsanity.com/paper/2408.12186/) (2024)
- [Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks](https://mlsanity.com/paper/2109.02832/) (2021)
- [Approximation and Estimation Ability of Transformers for Sequence-to-Sequence Functions with Infinite Dimensional Input](https://mlsanity.com/paper/2305.18699/) (2023)
- [Generalization Ability of Wide Neural Networks on ℝ](https://mlsanity.com/paper/2302.05933/) (2023)
- [A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models](https://mlsanity.com/paper/2401.07187/) (2024)
- [On the rate of convergence of a classifier based on a Transformer encoder](https://mlsanity.com/paper/2111.14574/) (2021)
- [Adaptive deep learning for nonlinear time series models](https://mlsanity.com/paper/2207.02546/) (2022)
- [Understanding Deep Neural Function Approximation in Reinforcement Learning via ε-Greedy Exploration](https://mlsanity.com/paper/2209.07376/) (2022)
- [Excess Risk of Two-Layer ReLU Neural Networks in Teacher-Student Settings and its Superiority to Kernel Methods](https://mlsanity.com/paper/2205.14818/) (2022)
- [Posterior and variational inference for deep neural networks with heavy-tailed weights](https://mlsanity.com/paper/2406.03369/) (2024)
- [Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks](https://mlsanity.com/paper/2404.18769/) (2024)
- [Stable and Explainable Personality Trait Evaluation in Large Language Models with Internal Activations](https://mlsanity.com/paper/2601.09833/) (2026)
- [On the inability of Gaussian process regression to optimally learn compositional functions](https://mlsanity.com/paper/2205.07764/) (2022)
- [Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraint](https://mlsanity.com/paper/2206.04569/) (2022)
- [Sup-Norm Convergence of Deep Neural Network Estimator for Nonparametric Regression by Adversarial Training](https://mlsanity.com/paper/2307.04042/) (2023)
- [State Space Models are Provably Comparable to Transformers in Dynamic Token Selection](https://mlsanity.com/paper/2405.19036/) (2024)
- [Estimation of a regression function on a manifold by fully connected deep neural networks](https://mlsanity.com/paper/2107.09532/) (2021)
- [Nested Expectations with Kernel Quadrature](https://mlsanity.com/paper/2502.18284/) (2025)
- [Analysis of convolutional neural network image classifiers in a rotationally symmetric model](https://mlsanity.com/paper/2205.05500/) (2022)
- [Predicting Out-of-Domain Generalization with Neighborhood Invariance](https://mlsanity.com/paper/2207.02093/) (2022)
- [Smooth Sailing: Improving Active Learning for Pre-trained Language Models with Representation Smoothness Analysis](https://mlsanity.com/paper/2212.11680/) (2022)
- [Exploring the Approximation Capabilities of Multiplicative Neural Networks for Smooth Functions](https://mlsanity.com/paper/2301.04605/) (2023)
- [Analysis of the expected L2 error of an over-parametrized deep neural network estimate learned by gradient descent without regularization](https://mlsanity.com/paper/2311.14609/) (2023)
- [Generalization Error of Graph Neural Networks in the Mean-field Regime](https://mlsanity.com/paper/2402.07025/) (2024)
- [Approximation Bounds for Recurrent Neural Networks with Application to Regression](https://mlsanity.com/paper/2409.05577/) (2024)
- [On the Rate of Convergence of Kolmogorov-Arnold Network Regression Estimators](https://mlsanity.com/paper/2509.19830/) (2025)
- [Provable Benefit of Curriculum in Transformer Tree-Reasoning Post-Training](https://mlsanity.com/paper/2511.07372/) (2025)
- [Solving PDEs on Spheres with Physics-Informed Convolutional Neural Networks](https://mlsanity.com/paper/2308.09605/) (2023)
- [Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regression](https://mlsanity.com/paper/2501.04898/) (2025)
- [Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networks](https://mlsanity.com/paper/2605.31152/) (2026)

## Abstract (arXiv)

Deep learning has exhibited superior performance for various tasks, especially for high-dimensional datasets, such as images. To understand this property, we investigate the approximation and estimation ability of deep learning on anisotropic Besov spaces. The anisotropic Besov space is characterized by direction-dependent smoothness and includes several function classes that have been investigated thus far. We demonstrate that the approximation error and estimation error of deep learning only depend on the average value of the smoothness parameters in all directions. Consequently, the curse of dimensionality can be avoided if the smoothness of the target function is highly anisotropic. Unlike existing studies, our analysis does not require a low-dimensional structure of the input data. We also investigate the minimax optimality of deep learning and compare its performance with that of the kernel method (more generally, linear estimators). The results show that deep learning has better dependence on the input dimensionality if the target function possesses anisotropic smoothness, and it achieves an adaptive rate for functions with spatially inhomogeneous smoothness.