mlsanityData to 14 Oct 2026

Characterizing the Discrete Geometry of ReLU Networks

arXiv 2606.07728 ↗First posted 5 Jun 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

It is well established that ReLU networks define continuous piecewise-linear functions, and that their linear regions are polyhedra in the input space. These regions form a complex that fully partitions the input space. The way these regions fit together is fundamental to the behavior of the network, as nonlinearities occur only at the boundaries where these regions connect. However, relatively little is known about the geometry of these complexes beyond bounds on the total number of regions, and calculating the complex exactly is intractable for most networks. In this work, we prove new theoretical results about these complexes that hold for all fully-connected ReLU networks, specifically about their connectivity graphs in which nodes correspond to regions and edges exist between each pair of regions connected by a face. We find that the average degree of this graph is upper bounded by twice the input dimension regardless of the width and depth of the network, and that the diameter of this graph has an upper bound that does not depend on input dimension, despite the number of regions increasing exponentially with input dimension. We corroborate our findings through experiments with networks trained on both synthetic and real-world data, which provide additional insight into the geometry of ReLU networks. Code to reproduce our results can be found at https://github.com/bl-ake/ICLR-2026.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 33 of its 57 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

33 of 57 references.

References · 33

  1. 2014On the Number of Linear Regions of Deep Neural Networkscited by 381
  2. 2016On the Expressive Power of Deep Neural Networkscited by 306
  3. 2019Complexity of Linear Regions in Deep Networkscited by 167
  4. 2017Bounding and Counting Linear Regions of Deep Neural Networkscited by 161
  5. 2019Deep ReLU Networks Have Surprisingly Few Activation Patternscited by 161
  6. 2018Strong mixed-integer programming formulations for trained neural networkscited by 115
  7. 2017Maximum Resilience of Artificial Neural Networkscited by 97
  8. 2018Tropical Geometry of Deep Neural Networkscited by 95
  9. 2013On the number of response regions of deep feed forward networks with piece-wise linear activationscited by 94
  10. 2019Reverse-Engineering Deep ReLU Networkscited by 83
  11. 2018Mad Max: Affine Spline Insights into Deep Learningcited by 55
  12. 2019The Geometry of Deep Networks: Power Diagram Subdivisioncited by 53
21 more
  1. 2021Partition-based formulations for mixed-integer optimization of trained ReLU neural networkscited by 39
  2. 2021Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sumscited by 37
  3. 2020Empirical Studies on the Properties of Linear Regions in Deep Neural Networkscited by 36
  4. 2020On transversality of bent hyperplane arrangements and the topological expressiveness of ReLU neural networkscited by 35
  5. 2018A Tropical Approach to Neural Networks with Piecewise Linear Activationscited by 26
  6. 2022Algorithmic Determination of the Combinatorial Structure of the Linear Regions of ReLU Neural Networkscited by 22
  7. 2022On the Number of Regions of Piecewise Linear Neural Networkscited by 21
  8. 2023Polyhedral Complex Extraction from ReLU Networks using Edge Subdivisioncited by 14
  9. 2023Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generationcited by 12
  10. 2020Reachability Analysis for Feed-Forward Neural Networks using Face Latticescited by 11
  11. 2023ReLU Neural Networks, Polyhedral Decompositions, and Persistent Homologcited by 10
  12. 2021Test Sample Accuracy Scales with Training Sample Density in Neural Networkscited by 9
  13. 2022Local and global topological complexity measures OF ReLU neural network functionscited by 8
  14. 2017Nonconvex piecewise linear functions: Advanced formulations and simple modeling toolscited by 7
  15. 2023Deep ReLU Networks Have Surprisingly Simple Polytopescited by 5
  16. 2023Hamming Similarity and Graph Laplacians for Class Partitioning and Adversarial Image Detectioncited by 4
  17. 2025The Geometry of ReLU Networks through the ReLU Transition Graphcited by 4
  18. 2021Neural networks behave as hash encoders: An empirical studycited by 3
  19. 2021Mixed Integer Neural Inverse Designcited by 3
  20. 2022Dual Graphs of Polyhedral Decompositions for the Detection of Adversarial Attackscited by 2
  21. 2020Tangent Space Sensitivity and Distribution of Linear Regions in ReLU Networkscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Laguerre Geometry for Interpreting Large Language Modelscited by 0