A Theoretical Study of Neural Network Expressive Power via Manifold Topology
Not on the map. Why?
Abstract (arXiv)
A prevalent assumption regarding real-world data is that it lies on or close to a low-dimensional manifold. When deploying a neural network on data manifolds, the required size, i.e., the number of neurons of the network, heavily depends on the intricacy of the underlying latent manifold. While significant advancements have been made in understanding the geometric attributes of manifolds, it's essential to recognize that topology, too, is a fundamental characteristic of manifolds. In this study, we investigate network expressive power in terms of the latent data manifold. Integrating both topological and geometric facets of the data manifold, we present a size upper bound of ReLU neural networks.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 18 of its 36 references that are in the map. First posted on arXiv 21 Oct 2024.
Builds on
18 of 36 references.
References · 18
- 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 257
- 2015The Power of Depth for Feedforward Neural Networkscited by 183
- 2019Complexity of Linear Regions in Deep Networkscited by 167
- 2017Approximating Continuous Functions by ReLU Nets of Minimal Widthcited by 122
- 2020Topology of deep neural networkscited by 112
- 2019Deep ReLU network approximation of functions on a manifoldcited by 74
- 2018Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topologycited by 73
- 2021Intrinsic Dimension, Persistent Homology and Generalization in Neural Networkscited by 70
- 2017Classification and Geometry of General Perceptual Manifoldscited by 62
- 2016Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networkscited by 59
- 2018On Characterizing the Capacity of Neural Networks using Algebraic Topologycited by 38
- 2020On transversality of bent hyperplane arrangements and the topological expressiveness of ReLU neural networkscited by 35
6 more
- 2020Deep Networks and the Multiple Manifold Problemcited by 35
- 2019Chart Auto-Encoders for Manifold Structured Datacited by 33
- 2018Topological Data Analysis of Decision Boundaries with Application to Model Selectioncited by 19
- 2024Topological Generalization Bounds for Discrete-Time Stochastic Optimization Algorithmscited by 12
- 2021For Manifold Learning, Deep Neural Networks can be Locality Sensitive Hash Functionscited by 6
- 2019Two-hidden-layer Feedforward Neural Networks are Universal Approximators: A Constructive Approachcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.