# A deep network construction that adapts to intrinsic dimensionality beyond the domain

arXiv: https://arxiv.org/abs/2008.02545 · HTML: https://mlsanity.com/paper/2008.02545/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2020.

## Built on by (most cited first, 10 of 10)

- [Deep Nonparametric Estimation of Operators between Infinite Dimensional Spaces](https://mlsanity.com/paper/2201.00217/) (2022)
- [Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks](https://mlsanity.com/paper/2109.02832/) (2021)
- [Deep Operator Learning Lessens the Curse of Dimensionality for PDEs](https://mlsanity.com/paper/2301.12227/) (2023)
- [Effects of Data Geometry in Early Deep Learning](https://mlsanity.com/paper/2301.00008/) (2022)
- [Stable Recovery of Entangled Weights: Towards Robust Identification of Deep Neural Networks from Minimal Samples](https://mlsanity.com/paper/2101.07150/) (2021)
- [Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraint](https://mlsanity.com/paper/2206.04569/) (2022)
- [Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation](https://mlsanity.com/paper/2406.05320/) (2024)
- [Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces](https://mlsanity.com/paper/2507.20853/) (2025)
- [Effective Minkowski Dimension of Deep Nonparametric Regression: Function Approximation and Statistical Theories](https://mlsanity.com/paper/2306.14859/) (2023)
- [On the Geometry of Reinforcement Learning in Continuous State and Action Spaces](https://mlsanity.com/paper/2301.00009/) (2022)

## Abstract (arXiv)

We study the approximation of two-layer compositions f(x) = g(φ(x)) via deep networks with ReLU activation, where φ is a geometrically intuitive, dimensionality reducing feature map. We focus on two intuitive and practically relevant choices for φ: the projection onto a low-dimensional embedded submanifold and a distance to a collection of low-dimensional sets. We achieve near optimal approximation rates, which depend only on the complexity of the dimensionality reducing map φ rather than the ambient dimension. Since φ encapsulates all nonlinear features that are material to the function f, this suggests that deep nets are faithful to an intrinsic dimension governed by f rather than the complexity of the domain of f. In particular, the prevalent assumption of approximating functions on low-dimensional manifolds can be significantly relaxed using functions of type f(x) = g(φ(x)) with φ representing an orthogonal projection onto the same manifold.