# A Theoretical Study of Neural Network Expressive Power via Manifold Topology

arXiv: https://arxiv.org/abs/2410.16542 · HTML: https://mlsanity.com/paper/2410.16542/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 21 Oct 2024.

## Builds on (most cited first, 18 of 18)

- [Understanding Deep Neural Networks with Rectified Linear Units](https://mlsanity.com/paper/1611.01491/) (2016)
- [The Power of Depth for Feedforward Neural Networks](https://mlsanity.com/paper/1512.03965/) (2015)
- [Complexity of Linear Regions in Deep Networks](https://mlsanity.com/paper/1901.09021/) (2019)
- [Approximating Continuous Functions by ReLU Nets of Minimal Width](https://mlsanity.com/paper/1710.11278/) (2017)
- [Topology of deep neural networks](https://mlsanity.com/paper/2004.06093/) (2020)
- [Deep ReLU network approximation of functions on a manifold](https://mlsanity.com/paper/1908.00695/) (2019)
- [Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology](https://mlsanity.com/paper/1812.09764/) (2018)
- [Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks](https://mlsanity.com/paper/2111.13171/) (2021)
- [Classification and Geometry of General Perceptual Manifolds](https://mlsanity.com/paper/1710.06487/) (2017)
- [Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks](https://mlsanity.com/paper/1610.09887/) (2016)
- [On Characterizing the Capacity of Neural Networks using Algebraic Topology](https://mlsanity.com/paper/1802.04443/) (2018)
- [On transversality of bent hyperplane arrangements and the topological expressiveness of ReLU neural networks](https://mlsanity.com/paper/2008.09052/) (2020)
- [Deep Networks and the Multiple Manifold Problem](https://mlsanity.com/paper/2008.11245/) (2020)
- [Chart Auto-Encoders for Manifold Structured Data](https://mlsanity.com/paper/1912.10094/) (2019)
- [Topological Data Analysis of Decision Boundaries with Application to Model Selection](https://mlsanity.com/paper/1805.09949/) (2018)
- [Topological Generalization Bounds for Discrete-Time Stochastic Optimization Algorithms](https://mlsanity.com/paper/2407.08723/) (2024)
- [For Manifold Learning, Deep Neural Networks can be Locality Sensitive Hash Functions](https://mlsanity.com/paper/2103.06875/) (2021)
- [Two-hidden-layer Feedforward Neural Networks are Universal Approximators: A Constructive Approach](https://mlsanity.com/paper/1907.11457/) (2019)

## Built on by (most cited first, 2 of 2)

- [Representation Gap: Explaining the Unreasonable Effectiveness of Neural Networks from a Geometric Perspective](https://mlsanity.com/paper/2605.21692/) (2026)
- [The Data Manifold under the Microscope](https://mlsanity.com/paper/2606.15760/) (2026)

## Abstract (arXiv)

A prevalent assumption regarding real-world data is that it lies on or close to a low-dimensional manifold. When deploying a neural network on data manifolds, the required size, i.e., the number of neurons of the network, heavily depends on the intricacy of the underlying latent manifold. While significant advancements have been made in understanding the geometric attributes of manifolds, it's essential to recognize that topology, too, is a fundamental characteristic of manifolds. In this study, we investigate network expressive power in terms of the latent data manifold. Integrating both topological and geometric facets of the data manifold, we present a size upper bound of ReLU neural networks.