Complexity of One-Dimensional ReLU DNNs
Not on the map. Why?
Abstract (arXiv)
We study the expressivity of one-dimensional (1D) ReLU deep neural networks through the lens of their linear regions. For randomly initialized, fully connected 1D ReLU networks (He scaling with nonzero bias) in the infinite-width limit, we prove that the expected number of linear regions grows as sumi = 1L ni + mathopoleft(sumi = 1Lniright) + 1, where nℓ denotes the number of neurons in the ℓ-th hidden layer. We also propose a function-adaptive notion of sparsity that compares the expected regions used by the network to the minimal number needed to approximate a target within a fixed tolerance.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 11 of its 18 references that are in the map. First posted on arXiv 8 Dec 2025.
Builds on
11 of 18 references.
References · 11
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
- 2017Deep Neural Networks as Gaussian Processescited by 622
- 2016Error bounds for approximations with deep ReLU networkscited by 574
- 2014On the Number of Linear Regions of Deep Neural Networkscited by 381
- 2016On the Expressive Power of Deep Neural Networkscited by 306
- 2019Complexity of Linear Regions in Deep Networkscited by 167
- 2017Bounding and Counting Linear Regions of Deep Neural Networkscited by 161
- 2019Deep ReLU Networks Have Surprisingly Few Activation Patternscited by 161
- 2019RadiX-Net: Structured Sparse Matrices for Deep Neural Networkscited by 12
- 2023Testing RadiX-Nets: Advances in Viable Sparse Topologiescited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.