mlsanityData to 7 Oct 2026

Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality

arXiv 1810.08033 ↗First posted 18 Oct 2018stat.ML cs.LGcited by 160

Not on the map. Why?

Abstract (arXiv)

Deep learning has shown high performances in various types of tasks from visual recognition to natural language processing, which indicates superior flexibility and adaptivity of deep learning. To understand this phenomenon theoretically, we develop a new approximation and estimation error analysis of deep learning with the ReLU activation for functions in a Besov space and its variant with mixed smoothness. The Besov space is a considerably general function space including the Holder space and Sobolev space, and especially can capture spatial inhomogeneity of smoothness. Through the analysis in the Besov space, it is shown that deep learning can achieve the minimax optimal rate and outperform any non-adaptive (linear) estimator such as kernel ridge regression, which shows that deep learning has higher adaptivity to the spatial inhomogeneity of the target function than other estimators such as linear ones. In addition to this, it is shown that deep learning can avoid the curse of dimensionality if the target function is in a mixed smooth Besov space. We also show that the dependency of the convergence rate on the dimensionality is tight due to its minimax optimality. These results support high adaptivity of deep learning and its superior ability as a feature extractor.

Key facts. Cited by 160 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 28; 2023: 30; 2024: 34; 2025: 31; 2026: 24.

160 papers.

Sort:

By area:

  1. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Datacited by 171
  2. 2022High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representationcited by 146
  3. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 127
  4. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 92
  5. 2024Model Collapse Demystified: The Case of Regressioncited by 80
  6. 2022Convex Analysis of the Mean Field Langevin Dynamicscited by 53
  7. 2022Deep Nonparametric Estimation of Operators between Infinite Dimensional Spacescited by 45
  8. 2021Universal Approximation Theorems for Differentiable Geometric Deep Learningcited by 41
  9. 2021Near-Minimax Optimal Estimation With Shallow ReLU Neural Networkscited by 40
  10. 2024Transformers are Minimax Optimal Nonparametric In-Context Learnerscited by 36
  11. 2023Deep Network Approximation: Beyond ReLU to Diverse Activation Functionscited by 33
  12. 2021Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networkscited by 32
38 more
  1. 2023Approximation and Estimation Ability of Transformers for Sequence-to-Sequence Functions with Infinite Dimensional Inputcited by 32
  2. 2023Generalization Ability of Wide Neural Networks on ℝcited by 28
  3. 2024Flow matching achieves almost minimax optimal convergencecited by 27
  4. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 26
  5. 2022How do noise tails impact on deep ReLU networks?cited by 25
  6. 2021Wasserstein Generative Learning of Conditional Distributioncited by 24
  7. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  8. 2021Theory of Deep Convolutional Neural Networks III: Approximating Radial Functionscited by 22
  9. 2023Distribution learning via neural differential equations: a nonparametric statistical perspectivecited by 22
  10. 2024A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Modelscited by 20
  11. 2023Deep Operator Learning Lessens the Curse of Dimensionality for PDEscited by 18
  12. 2022Neural Network Architecture Beyond Width and Depthcited by 17
  13. 2023On the Convergence and Sample Complexity Analysis of Deep Q-Networks with ε-Greedy Explorationcited by 17
  14. 2021On the rate of convergence of a classifier based on a Transformer encodercited by 16
  15. 2023Classification with Deep Neural Networks and Logistic Losscited by 16
  16. 2024Convergence Analysis of Flow Matching in Latent Space with Transformerscited by 16
  17. 2024Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Losscited by 14
  18. 2024Learning sum of diverse features: computational hardness and efficient gradient-based training for ridge combinationscited by 13
  19. 2024On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networkscited by 13
  20. 2021Deep Network Approximation in Terms of Intrinsic Parameterscited by 12
  21. 2022Sample Complexity of Nonparametric Off-Policy Evaluation on Low-Dimensional Manifolds using Deep Networkscited by 12
  22. 2022Small Transformers Compute Universal Metric Embeddingscited by 12
  23. 2024Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model Reduction for Operator Learningcited by 12
  24. 2024Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizescited by 12
  25. 2022Adaptive deep learning for nonlinear time series modelscited by 11
  26. 2022Understanding Deep Neural Function Approximation in Reinforcement Learning via ε-Greedy Explorationcited by 11
  27. 2022Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?cited by 10
  28. 2022Excess Risk of Two-Layer ReLU Neural Networks in Teacher-Student Settings and its Superiority to Kernel Methodscited by 10
  29. 2022DeepMed: Semiparametric Causal Mediation Analysis with Debiased Deep Learningcited by 10
  30. 2023Excess risk bound for deep learning under weak dependencecited by 9
  31. 2024Posterior and variational inference for deep neural networks with heavy-tailed weightscited by 9
  32. 2025On the minimax optimality of Flow Matching through the connection to kernel density estimationcited by 9
  33. 2024Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networkscited by 8
  34. 2021Deep Neural Networks with ReLU-Sine-Exponential Activations Break Curse of Dimensionality in Approximation on H"older Classcited by 7
  35. 2021Sample Complexity of Offline Reinforcement Learning with Deep ReLU Networkscited by 7
  36. 2021A spectral-based analysis of the separation between two-layer neural networks and linear methodscited by 7
  37. 2022On the inability of Gaussian process regression to optimally learn compositional functionscited by 7
  38. 2022Asymptotic Properties for Bayesian Neural Network in Besov Spacecited by 7

50 most cited of 160 All 160 (JSON)