# A Statistical Analysis for Supervised Deep Learning with Exponential Families for Intrinsically Low-dimensional Data

arXiv: https://arxiv.org/abs/2412.09779 · HTML: https://mlsanity.com/paper/2412.09779/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 38 references that are in the map.
- First posted on arXiv 13 Dec 2024.

## Builds on (most cited first, 2 of 2)

- [Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality](https://mlsanity.com/paper/1810.08033/) (2018)
- [On the Statistical Properties of Generative Adversarial Models for Low Intrinsic Data Dimension](https://mlsanity.com/paper/2401.15801/) (2024)

## Abstract (arXiv)

Recent advances have revealed that the rate of convergence of the expected test error in deep supervised learning decays as a function of the intrinsic dimension and not the dimension d of the input space. Existing literature defines this intrinsic dimension as the Minkowski dimension or the manifold dimension of the support of the underlying probability measures, which often results in sub-optimal rates and unrealistic assumptions. In this paper, we consider supervised deep learning when the response given the explanatory variable is distributed according to an exponential family with a β-H"older smooth mean function. We consider an entropic notion of the intrinsic data-dimension and demonstrate that with n independent and identically distributed samples, the test error scales as tildeOleft(n^-frac2β2β + bard2β(λ)right), where bard2β(λ) is the 2β-entropic dimension of λ, the distribution of the explanatory variables. This improves on the best-known rates. Furthermore, under the assumption of an upper-bounded density of the explanatory variables, we characterize the rate of convergence as tildeOleft( d^frac2lfloorβrfloor(β + d)2β + dn^-frac2β2β + dright), establishing that the dependence on d is not exponential but at most polynomial. We also demonstrate that when the explanatory variable has a lower bounded density, this rate in terms of the number of data samples, is nearly optimal for learning the dependence structure for exponential families.