Data Topology-Dependent Upper Bounds of Neural Network Widths
Not on the map. Why?
Abstract (arXiv)
This paper investigates the relationship between the universal approximation property of deep neural networks and topological characteristics of datasets. Our primary contribution is to introduce data topology-dependent upper bounds on the network width. Specifically, we first show that a three-layer neural network, applying a ReLU activation function and max pooling, can be designed to approximate an indicator function over a compact set, one that is encompassed by a tight convex polytope. This is then extended to a simplicial complex, deriving width upper bounds based on its topological structure. Further, we calculate upper bounds in relation to the Betti numbers of select topological spaces. Finally, we prove the universal approximation property of three-layer ReLU networks using our topological approach. We also verify that gradient descent converges to the network structure proposed in our study.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 30 of its 47 references that are in the map. First posted on arXiv 25 May 2023.
Builds on
30 of 47 references.
References · 30
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
- 2016Benefits of depth in neural networkscited by 248
- 2015The Power of Depth for Feedforward Neural Networkscited by 181
- 2020Topology of deep neural networkscited by 111
- 2017Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activationscited by 104
- 2020Minimum Width for Universal Approximationcited by 92
- 2016Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networkscited by 58
- 2021Towards Lower Bounds on the Depth of ReLU Neural Networkscited by 55
- 2018On Characterizing the Capacity of Neural Networks using Algebraic Topologycited by 37
- 2018Collapse of Deep and Narrow Neural Netscited by 37
- 2023Topological Deep Learning: A Review of an Emerging Paradigmcited by 34
- 2023Learning Topology-Preserving Data Representationscited by 30
18 more
- 2020Constructive Universal High-Dimensional Distribution Generation through Deep ReLU Networkscited by 13
- 2022Topology and geometry of data manifold in deep learningcited by 12
- 2022Minimal Width for Universal Property of Deep RNNcited by 12
- 2022Effects of Data Geometry in Early Deep Learningcited by 12
- 2018The universal approximation power of finite-width deep ReLU networkscited by 11
- 2019Universal Lipschitz Approximation in Bounded Depth Neural Networkscited by 11
- 2020A three layer neural network can represent any multivariate functioncited by 10
- 2022Width is Less Important than Depth in ReLU Neural Networkscited by 10
- 2023Approximation of Nonlinear Functionals Using Deep ReLU Networkscited by 8
- 2019Approximation capabilities of neural networks on unbounded domainscited by 7
- 2020Depth-Width Trade-offs for Neural Networks via Topological Entropycited by 6
- 2022Do ReLU Networks Have An Edge When Approximating Compactly-Supported Functions?cited by 6
- 2020A Topological Framework for Deep Learningcited by 5
- 2021Topological Deep Learning: Classification Neural Networkscited by 5
- 2022What does a deep neural network confidently perceive? The effective dimension of high certainty class manifolds and their low confidence boundariescited by 5
- 2022Theoretical Exploration of Solutions of Feedforward ReLU Networkscited by 4
- 2021The Compact Support Neural Networkcited by 2
- 2021Dive into Layers: Neural Network Capacity Bounding using Algebraic Geometrycited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.