mlsanityData to 14 Oct 2026

The Real Tropical Geometry of Neural Networks

arXiv 2403.11871 ↗First posted 18 Mar 2024math.CO cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

We consider a binary classifier defined as the sign of a tropical rational function, that is, as the difference of two convex piecewise linear functions. The parameter space of ReLU neural networks is contained as a semialgebraic set inside the parameter space of tropical rational functions. We initiate the study of two different subdivisions of this parameter space: a subdivision into semialgebraic sets, on which the combinatorial type of the decision boundary is fixed, and a subdivision into a polyhedral fan, capturing the combinatorics of the partitions of the dataset. The sublevel sets of the 0/1-loss function arise as subfans of this classification fan, and we show that the level-sets are not necessarily connected. We describe the classification fan i) geometrically, as normal fan of the activation polytope, and ii) combinatorially through a list of properties of associated bipartite graphs, in analogy to covector axioms of oriented matroids and tropical oriented matroids. Our findings extend and refine the connection between neural networks and tropical geometry by observing structures established in real tropical geometry, such as positive tropicalizations of hypersurfaces and tropical semialgebraic sets.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 39 of its 76 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

39 of 76 references.

References · 39

  1. 2014On the Number of Linear Regions of Deep Neural Networkscited by 381
  2. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 311
  3. 2016On the Expressive Power of Deep Neural Networkscited by 306
  4. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 257
  5. 2016Benefits of depth in neural networkscited by 250
  6. 2017Bounding and Counting Linear Regions of Deep Neural Networkscited by 161
  7. 2021Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariancescited by 127
  8. 2017Spurious Local Minima are Common in Two-Layer ReLU Neural Networkscited by 104
  9. 2016Topology and Geometry of Half-Rectified Network Optimizationcited by 102
  10. 2018Tropical Geometry of Deep Neural Networkscited by 95
  11. 2013On the number of response regions of deep feed forward networks with piece-wise linear activationscited by 94
  12. 2019Reverse-Engineering Deep ReLU Networkscited by 83
27 more
  1. 2019Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Netscited by 63
  2. 2021Towards Lower Bounds on the Depth of ReLU Neural Networkscited by 56
  3. 2019The Geometry of Deep Networks: Power Diagram Subdivisioncited by 53
  4. 2023When Deep Learning Meets Polyhedral Theory: A Surveycited by 39
  5. 2023Hidden symmetries of ReLU networkscited by 39
  6. 2018Deep, Skinny Neural Networks are not Universal Approximatorscited by 38
  7. 2021Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sumscited by 37
  8. 2020On transversality of bent hyperplane arrangements and the topological expressiveness of ReLU neural networkscited by 35
  9. 2022Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositionscited by 31
  10. 2023SplineCam: Exact Visualization and Characterization of Deep Network Geometry and Decision Boundariescited by 31
  11. 2020The Hidden Convex Optimization Landscape of Two-Layer ReLU Neural Networks: an Exact Characterization of the Optimal Solutionscited by 30
  12. 2018A Tropical Approach to Neural Networks with Piecewise Linear Activationscited by 26
  13. 2020On the Decision Boundaries of Neural Networks: A Tropical Geometry Perspectivecited by 25
  14. 2021Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programscited by 25
  15. 2017Exponentially vanishing sub-optimal local minima in multilayer neural networkscited by 19
  16. 2018Neural Networks Should Be Wide Enough to Learn Disconnected Decision Regionscited by 17
  17. 2023Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscapecited by 17
  18. 2018A Framework for the construction of upper bounds on the number of affine linear regions of ReLU feed-forward neural networkscited by 15
  19. 2020Piecewise Linear Regression via a Difference of Convex Functionscited by 14
  20. 2021When Are Solutions Connected in Deep Networks?cited by 11
  21. 2023Exact Count of Boundary Pieces of ReLU Classifiers: Towards the Proper Complexity Measure for Classificationcited by 11
  22. 2023Optimal Sets and Solution Paths of ReLU Networkscited by 10
  23. 2021Spurious Local Minima Are Common for Deep Neural Networks with Piecewise Linear Activationscited by 7
  24. 2018On decision regions of narrow deep neural networkscited by 5
  25. 2021Novel convex decomposition of piecewise affine functionscited by 4
  26. 2022A Combinatorial Perspective on the Optimization of Shallow ReLU Networkscited by 3
  27. 2016Tropical spectrahedracited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 9; 2026: 9.

24 papers.

Sort:

By area:

  1. 2025Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometrycited by 21
  2. 2024Activation degree thresholds and expressiveness of polynomial neural networkscited by 14
  3. 2025Tropical Attention: Neural Algorithmic Reasoning for Combinatorial Algorithmscited by 12
  4. 2025Depth-Bounds for Neural Networks via the Braid Arrangementcited by 11
  5. 2024Decomposition Polyhedra of Piecewise Linear Functionscited by 9
  6. 2024Neural Networks and (Virtual) Extended Formulationscited by 9
  7. 2024On the Local Complexity of Linear Regions in Deep ReLU Networkscited by 9
  8. 2026Most ReLU Networks Admit Identifiable Parameterscited by 6
  9. 2025Constraining the outputs of ReLU neural networkscited by 3
  10. 2026Sparsity is Combinatorial Depth: Quantifying MoE Expressivity via Tropical Geometrycited by 3
  11. 2026Robustness Verification of Polynomial Neural Networkscited by 3
  12. 2024Representing Piecewise-Linear Functions by Functions with Minimal Aritycited by 2
12 more
  1. 2024A Computational Tropical Geometry Framework for Neural Networkscited by 1
  2. 2025From Universal Approximation Theorem to Tropical Geometry of Multi-Layer Perceptronscited by 1
  3. 2026Algebraic Invariants of Lightning Self-Attentioncited by 1
  4. 2026Minimal Filling Architectures of Polynomial Neural Networks: Counterexamples, Frontier Search, and Defectscited by 1
  5. 2026TwistedMerge: Certified Higher-Order Diagnostics and Abstention for Model Mergingcited by 1
  6. 2025How to Learn a Star: Binary Classification with Starshaped Polyhedral Setscited by 0
  7. 2025Algebraic Approach to Ridge-Regularized Mean Squared Error Minimization in Minimal ReLU Neural Networkcited by 0
  8. 2025Maxout Polytopescited by 0
  9. 2025On the expressivity of sparse maxout networkscited by 0
  10. 2026Hidden Monotonicity: Explaining Deep Neural Networks via their DC Decompositioncited by 0
  11. 2026Patchwork: A compact representation for 3D polygonal shapescited by 0
  12. 2026Constructive Bounds for the Topological Expressivity of Neural Networkscited by 0