# Data Topology-Dependent Upper Bounds of Neural Network Widths

arXiv: https://arxiv.org/abs/2305.16375 · HTML: https://mlsanity.com/paper/2305.16375/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 30 of 30)

- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Benefits of depth in neural networks](https://mlsanity.com/paper/1602.04485/) (2016)
- [The Power of Depth for Feedforward Neural Networks](https://mlsanity.com/paper/1512.03965/) (2015)
- [Topology of deep neural networks](https://mlsanity.com/paper/2004.06093/) (2020)
- [Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations](https://mlsanity.com/paper/1708.02691/) (2017)
- [Minimum Width for Universal Approximation](https://mlsanity.com/paper/2006.08859/) (2020)
- [Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks](https://mlsanity.com/paper/1610.09887/) (2016)
- [Towards Lower Bounds on the Depth of ReLU Neural Networks](https://mlsanity.com/paper/2105.14835/) (2021)
- [On Characterizing the Capacity of Neural Networks using Algebraic Topology](https://mlsanity.com/paper/1802.04443/) (2018)
- [Collapse of Deep and Narrow Neural Nets](https://mlsanity.com/paper/1808.04947/) (2018)
- [Topological Deep Learning: A Review of an Emerging Paradigm](https://mlsanity.com/paper/2302.03836/) (2023)
- [Learning Topology-Preserving Data Representations](https://mlsanity.com/paper/2302.00136/) (2023)
- [Constructive Universal High-Dimensional Distribution Generation through Deep ReLU Networks](https://mlsanity.com/paper/2006.16664/) (2020)
- [The universal approximation power of finite-width deep ReLU networks](https://mlsanity.com/paper/1806.01528/) (2018)
- [Topology and geometry of data manifold in deep learning](https://mlsanity.com/paper/2204.08624/) (2022)
- [Minimal Width for Universal Property of Deep RNN](https://mlsanity.com/paper/2211.13866/) (2022)
- [Effects of Data Geometry in Early Deep Learning](https://mlsanity.com/paper/2301.00008/) (2022)
- [Universal Lipschitz Approximation in Bounded Depth Neural Networks](https://mlsanity.com/paper/1904.04861/) (2019)
- [A three layer neural network can represent any multivariate function](https://mlsanity.com/paper/2012.03016/) (2020)
- [Width is Less Important than Depth in ReLU Neural Networks](https://mlsanity.com/paper/2202.03841/) (2022)
- [Approximation of Nonlinear Functionals Using Deep ReLU Networks](https://mlsanity.com/paper/2304.04443/) (2023)
- [Approximation capabilities of neural networks on unbounded domains](https://mlsanity.com/paper/1910.09293/) (2019)
- [Depth-Width Trade-offs for Neural Networks via Topological Entropy](https://mlsanity.com/paper/2010.07587/) (2020)
- [Do ReLU Networks Have An Edge When Approximating Compactly-Supported Functions?](https://mlsanity.com/paper/2204.11231/) (2022)
- [A Topological Framework for Deep Learning](https://mlsanity.com/paper/2008.13697/) (2020)
- [Topological Deep Learning: Classification Neural Networks](https://mlsanity.com/paper/2102.08354/) (2021)
- [What does a deep neural network confidently perceive? The effective dimension of high certainty class manifolds and their low confidence boundaries](https://mlsanity.com/paper/2210.05546/) (2022)
- [Theoretical Exploration of Solutions of Feedforward ReLU Networks](https://mlsanity.com/paper/2202.01919/) (2022)
- [The Compact Support Neural Network](https://mlsanity.com/paper/2104.00269/) (2021)
- [Dive into Layers: Neural Network Capacity Bounding using Algebraic Geometry](https://mlsanity.com/paper/2109.01461/) (2021)

## Built on by (most cited first, 1 of 1)

- [Latent Space Topology Evolution in Multilayer Perceptrons](https://mlsanity.com/paper/2506.01569/) (2025)

## Abstract (arXiv)

This paper investigates the relationship between the universal approximation property of deep neural networks and topological characteristics of datasets. Our primary contribution is to introduce data topology-dependent upper bounds on the network width. Specifically, we first show that a three-layer neural network, applying a ReLU activation function and max pooling, can be designed to approximate an indicator function over a compact set, one that is encompassed by a tight convex polytope. This is then extended to a simplicial complex, deriving width upper bounds based on its topological structure. Further, we calculate upper bounds in relation to the Betti numbers of select topological spaces. Finally, we prove the universal approximation property of three-layer ReLU networks using our topological approach. We also verify that gradient descent converges to the network structure proposed in our study.