Local large deviations for linear-region growth in random piecewise-linear networks
Not on the map. Why?
Abstract (arXiv)
We study a random compositional model for the growth of affine regions in deep piecewise-linear networks. The model is generated by i.i.d. perturbations of the symmetric height-one tent map, and the main observable is the number (Nn) of affine pieces after (n) layers. We prove the existence of a submultiplicative pressure for (Nn), yielding exponential upper bounds for both tails of (n-1log Nn). The same argument applies to abstract submultiplicative complexity observables and gives higher-dimensional extensions for convex-polytopal affine-cover counts and worst-line affine-piece counts. Since the true branch count has no matching supermultiplicative inequality, lower bounds require a separate certified construction. We introduce a finite-state defect process that records branches whose future splitting can be guaranteed, and use bridge words to obtain constructive upper-tail lower bounds. In a uniformly favorable small-noise regime, this process is governed by a companion matrix whose Perron root tends to (2), implying eventual exclusion of lower tails below (log 2-xi).
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 10 of its 24 references that are in the map. First posted on arXiv 8 Jul 2026.
Builds on
10 of 24 references.
References · 10
- 2014On the Number of Linear Regions of Deep Neural Networkscited by 378
- 2016On the Expressive Power of Deep Neural Networkscited by 302
- 2016Benefits of depth in neural networkscited by 248
- 2018Sensitivity and Generalization in Neural Networks: an Empirical Studycited by 187
- 2019Complexity of Linear Regions in Deep Networkscited by 166
- 2017Bounding and Counting Linear Regions of Deep Neural Networkscited by 161
- 2013On the number of response regions of deep feed forward networks with piece-wise linear activationscited by 94
- 2015Representation Benefits of Deep Feedforward Networkscited by 84
- 2024Mathematical theory of deep learningcited by 20
- 2024Large Deviations of Gaussian Neural Networks with ReLU activationcited by 3
Built on by
Not cited yet.