mlsanityData to 7 Oct 2026

Local large deviations for linear-region growth in random piecewise-linear networks

arXiv 2607.07014 ↗First posted 8 Jul 2026math.PR stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

We study a random compositional model for the growth of affine regions in deep piecewise-linear networks. The model is generated by i.i.d. perturbations of the symmetric height-one tent map, and the main observable is the number (Nn) of affine pieces after (n) layers. We prove the existence of a submultiplicative pressure for (Nn), yielding exponential upper bounds for both tails of (n-1log Nn). The same argument applies to abstract submultiplicative complexity observables and gives higher-dimensional extensions for convex-polytopal affine-cover counts and worst-line affine-piece counts. Since the true branch count has no matching supermultiplicative inequality, lower bounds require a separate certified construction. We introduce a finite-state defect process that records branches whose future splitting can be guaranteed, and use bridge words to obtain constructive upper-tail lower bounds. In a uniformly favorable small-noise regime, this process is governed by a companion matrix whose Perron root tends to (2), implying eventual exclusion of lower tails below (log 2-xi).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 10 of its 24 references that are in the map. First posted on arXiv 8 Jul 2026.

Builds on

10 of 24 references.

References · 10

  1. 2014On the Number of Linear Regions of Deep Neural Networkscited by 378
  2. 2016On the Expressive Power of Deep Neural Networkscited by 302
  3. 2016Benefits of depth in neural networkscited by 248
  4. 2018Sensitivity and Generalization in Neural Networks: an Empirical Studycited by 187
  5. 2019Complexity of Linear Regions in Deep Networkscited by 166
  6. 2017Bounding and Counting Linear Regions of Deep Neural Networkscited by 161
  7. 2013On the number of response regions of deep feed forward networks with piece-wise linear activationscited by 94
  8. 2015Representation Benefits of Deep Feedforward Networkscited by 84
  9. 2024Mathematical theory of deep learningcited by 20
  10. 2024Large Deviations of Gaussian Neural Networks with ReLU activationcited by 3

Built on by

Not cited yet.