# Effective Minkowski Dimension of Deep Nonparametric Regression: Function Approximation and Statistical Theories

arXiv: https://arxiv.org/abs/2306.14859 · HTML: https://mlsanity.com/paper/2306.14859/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 44 references that are in the map.
- First posted on arXiv 26 Jun 2023.

## Builds on (most cited first, 14 of 14)

- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Recent Trends in Deep Learning Based Natural Language Processing](https://mlsanity.com/paper/1708.02709/) (2017)
- [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017)
- [Deep ReLU network approximation of functions on a manifold](https://mlsanity.com/paper/1908.00695/) (2019)
- [Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recovery](https://mlsanity.com/paper/1908.01842/) (2019)
- [Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks](https://mlsanity.com/paper/2109.02832/) (2021)
- [A deep network construction that adapts to intrinsic dimensionality beyond the domain](https://mlsanity.com/paper/2008.02545/) (2020)
- [Multiscale regression on unknown manifolds](https://mlsanity.com/paper/2101.05119/) (2021)

## Built on by (most cited first, 1 of 1)

- [Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation](https://mlsanity.com/paper/2406.05320/) (2024)

## Abstract (arXiv)

Existing theories on deep nonparametric regression have shown that when the input data lie on a low-dimensional manifold, deep neural networks can adapt to the intrinsic data structures. In real world applications, such an assumption of data lying exactly on a low dimensional manifold is stringent. This paper introduces a relaxed assumption that the input data are concentrated around a subset of ℝd denoted by S, and the intrinsic dimension of S can be characterized by a new complexity notation -- effective Minkowski dimension. We prove that, the sample complexity of deep nonparametric regression only depends on the effective Minkowski dimension of S denoted by p. We further illustrate our theoretical findings by considering nonparametric regression with an anisotropic Gaussian random design N(0,Sigma), where Sigma is full rank. When the eigenvalues of Sigma have an exponential or polynomial decay, the effective Minkowski dimension of such an Gaussian random design is p=O(√log n) or p=O(nγ), respectively, where n is the sample size and γ∈(0,1) is a small constant depending on the polynomial decay rate. Our theory shows that, when the manifold assumption does not hold, deep neural networks can still adapt to the effective Minkowski dimension of the data, and circumvent the curse of the ambient dimensionality for moderate sample sizes.