mlsanityData to 7 Oct 2026

A Quotient Homology Theory of Representation in Neural Networks

arXiv 2502.01360 ↗First posted 3 Feb 2025cs.LG math.AT q-bio.NCcited by 0

Not on the map. Why?

Abstract (arXiv)

Previous research has proven that the set of maps implemented by neural networks with a ReLU activation function is identical to the set of piecewise linear continuous maps. Furthermore, such networks induce a hyperplane arrangement splitting the input domain of the network into convex polyhedra GJ over which a network Phi operates in an affine manner. In this work, we leverage these properties to define an equivalence relation ~_Phi on top of an input dataset, which defines a quotient space that can be split into two sets related to the local rank of PhiJ and the intersections cap ImPhiJi. We refer to the latter as the textitoverlap decomposition O_Phi and prove that if the intersections between each polyhedron and an input manifold are convex, the homology groups of neural representations are isomorphic to quotient homology groups Hk(Phi(M)) simeq Hk(M/O_Phi). This lets us intrinsically calculate the Betti numbers of neural representations without the choice of an external metric. We develop methods to numerically compute the overlap decomposition through linear programming and a union-find algorithm. Using this framework, we perform several experiments on toy datasets showing that, compared to standard persistent homology, our overlap homology-based computation of Betti numbers tracks purely topological rather than geometric features. Finally, we study the evolution of the overlap decomposition during training on several classification problems and discuss some shortcomings of our method.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 57 references that are in the map. First posted on arXiv 3 Feb 2025.

Builds on

34 of 57 references.

References · 34

  1. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  2. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  3. 2014On the Number of Linear Regions of Deep Neural Networkscited by 378
  4. 2016On the Expressive Power of Deep Neural Networkscited by 302
  5. 2016Exponential expressivity in deep neural networks through transient chaoscited by 260
  6. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 253
  7. 2015Persistence Images: A Stable Vector Representation of Persistent Homologycited by 235
  8. 2017Bounding and Counting Linear Regions of Deep Neural Networkscited by 161
  9. 2019Deep ReLU Networks Have Surprisingly Few Activation Patternscited by 160
  10. 2019Ripser: efficient computation of Vietoris-Rips persistence barcodescited by 115
  11. 2020Topology of deep neural networkscited by 111
  12. 2013On the number of response regions of deep feed forward networks with piece-wise linear activationscited by 94
22 more
  1. 2015Parallelizing the dual revised simplex methodcited by 83
  2. 2018Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topologycited by 73
  3. 2021On the geometry of generalization and memorization in deep neural networkscited by 71
  4. 2021Intrinsic Dimension, Persistent Homology and Generalization in Neural Networkscited by 70
  5. 2024Position: Topological Deep Learning is the New Frontier for Relational Learningcited by 65
  6. 2019The Geometry of Deep Networks: Power Diagram Subdivisioncited by 53
  7. 2018On Characterizing the Capacity of Neural Networks using Algebraic Topologycited by 37
  8. 2020Empirical Studies on the Properties of Linear Regions in Deep Neural Networkscited by 36
  9. 2020On transversality of bent hyperplane arrangements and the topological expressiveness of ReLU neural networkscited by 34
  10. 2020Persistent Laplacians: properties, algorithms and implicationscited by 28
  11. 2019Persistent homology detects curvaturecited by 25
  12. 2022On the Number of Regions of Piecewise Linear Neural Networkscited by 21
  13. 2022Algorithmic Determination of the Combinatorial Structure of the Linear Regions of ReLU Neural Networkscited by 21
  14. 2018Exposition and Interpretation of the Topology of Neural Networkscited by 20
  15. 2022On the effectiveness of persistent homologycited by 20
  16. 2021Activation Landscapes as a Topological Summary of Neural Network Performancecited by 17
  17. 2023Generalization Bounds with Data-dependent Fractal Dimensionscited by 15
  18. 2023Metric Space Magnitude and Generalisation in Neural Networkscited by 15
  19. 2023ReLU Neural Networks, Polyhedral Decompositions, and Persistent Homologcited by 10
  20. 2023Deep ReLU Networks Have Surprisingly Simple Polytopescited by 5
  21. 2021Persistent Homology Captures the Generalization of Neural Networks Without A Validation Setcited by 4
  22. 2024A rank decomposition for the topological classification of neural representationscited by 1

Built on by

Not cited yet.