# Local large deviations for linear-region growth in random piecewise-linear networks

arXiv: https://arxiv.org/abs/2607.07014 · HTML: https://mlsanity.com/paper/2607.07014/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 8 Jul 2026.

## Builds on (most cited first, 10 of 10)

- [On the Number of Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1402.1869/) (2014)
- [On the Expressive Power of Deep Neural Networks](https://mlsanity.com/paper/1606.05336/) (2016)
- [Benefits of depth in neural networks](https://mlsanity.com/paper/1602.04485/) (2016)
- [Sensitivity and Generalization in Neural Networks: an Empirical Study](https://mlsanity.com/paper/1802.08760/) (2018)
- [Complexity of Linear Regions in Deep Networks](https://mlsanity.com/paper/1901.09021/) (2019)
- [Bounding and Counting Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1711.02114/) (2017)
- [On the number of response regions of deep feed forward networks with piece-wise linear activations](https://mlsanity.com/paper/1312.6098/) (2013)
- [Representation Benefits of Deep Feedforward Networks](https://mlsanity.com/paper/1509.08101/) (2015)
- [Mathematical theory of deep learning](https://mlsanity.com/paper/2407.18384/) (2024)
- [Large Deviations of Gaussian Neural Networks with ReLU activation](https://mlsanity.com/paper/2405.16958/) (2024)

## Abstract (arXiv)

We study a random compositional model for the growth of affine regions in deep piecewise-linear networks. The model is generated by i.i.d. perturbations of the symmetric height-one tent map, and the main observable is the number (Nn) of affine pieces after (n) layers. We prove the existence of a submultiplicative pressure for (Nn), yielding exponential upper bounds for both tails of (n-1log Nn). The same argument applies to abstract submultiplicative complexity observables and gives higher-dimensional extensions for convex-polytopal affine-cover counts and worst-line affine-piece counts. Since the true branch count has no matching supermultiplicative inequality, lower bounds require a separate certified construction. We introduce a finite-state defect process that records branches whose future splitting can be guaranteed, and use bridge words to obtain constructive upper-tail lower bounds. In a uniformly favorable small-noise regime, this process is governed by a companion matrix whose Perron root tends to (2), implying eventual exclusion of lower tails below (log 2-xi).