# A Tropical Approach to Neural Networks with Piecewise Linear Activations

arXiv: https://arxiv.org/abs/1805.08749 · HTML: https://mlsanity.com/paper/1805.08749/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2018.

## Built on by (most cited first, 26 of 26)

- [Towards Lower Bounds on the Depth of ReLU Neural Networks](https://mlsanity.com/paper/2105.14835/) (2021)
- [When Deep Learning Meets Polyhedral Theory: A Survey](https://mlsanity.com/paper/2305.00241/) (2023)
- [Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums](https://mlsanity.com/paper/2104.08135/) (2021)
- [Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes](https://mlsanity.com/paper/2302.12553/) (2023)
- [The Real Tropical Geometry of Neural Networks](https://mlsanity.com/paper/2403.11871/) (2024)
- [An enriched category theory of language: from syntax to semantics](https://mlsanity.com/paper/2106.07890/) (2021)
- [On the Expected Complexity of Maxout Networks](https://mlsanity.com/paper/2107.00379/) (2021)
- [Activation degree thresholds and expressiveness of polynomial neural networks](https://mlsanity.com/paper/2408.04569/) (2024)
- [Neurons on Amoebae](https://mlsanity.com/paper/2106.03695/) (2021)
- [Exact Count of Boundary Pieces of ReLU Classifiers: Towards the Proper Complexity Measure for Classification](https://mlsanity.com/paper/2306.08805/) (2023)
- [Min-Max-Plus Neural Networks](https://mlsanity.com/paper/2102.06358/) (2021)
- [Most ReLU Networks Admit Identifiable Parameters](https://mlsanity.com/paper/2605.03601/) (2026)
- [TropNNC: Structured Neural Network Compression Using Tropical Geometry](https://mlsanity.com/paper/2409.03945/) (2024)
- [Constraining the outputs of ReLU neural networks](https://mlsanity.com/paper/2508.03867/) (2025)
- [Sparsity is Combinatorial Depth: Quantifying MoE Expressivity via Tropical Geometry](https://mlsanity.com/paper/2602.03204/) (2026)
- [Revisiting Tropical Polynomial Division: Theory, Algorithms and Application to Neural Networks](https://mlsanity.com/paper/2306.15157/) (2023)
- [Tight Certified Robustness via Min-Max Representations of ReLU Neural Networks](https://mlsanity.com/paper/2310.04916/) (2023)
- [Arithmetic Circuits and Neural Networks for Regular Matroids](https://mlsanity.com/paper/2511.02406/) (2025)
- [Tropical Decision Boundaries for Neural Networks Are Robust Against Adversarial Attacks](https://mlsanity.com/paper/2402.00576/) (2024)
- [A Computational Tropical Geometry Framework for Neural Networks](https://mlsanity.com/paper/2405.20174/) (2024)
- [Relu and softplus neural nets as zero-sum turn-based games](https://mlsanity.com/paper/2512.20582/) (2025)
- [Characterizing the Discrete Geometry of ReLU Networks](https://mlsanity.com/paper/2606.07728/) (2026)
- [Enumeration of max-pooling responses with generalized permutohedra](https://mlsanity.com/paper/2209.14978/) (2022)
- [Alternating Minimization for Regression with Tropical Rational Functions](https://mlsanity.com/paper/2305.20072/) (2023)
- [Algebraic Approach to Ridge-Regularized Mean Squared Error Minimization in Minimal ReLU Neural Network](https://mlsanity.com/paper/2508.17783/) (2025)
- [A Geometric Approach to Soft Actor-Critic with Zonotopes for Locomotion Learning](https://mlsanity.com/paper/2610.12113/) (2026)

## Abstract (arXiv)

We present a new, unifying approach following some recent developments on the complexity of neural networks with piecewise linear activations. We treat neural network layers with piecewise linear activations as tropical polynomials, which generalize polynomials in the so-called (max, +) or tropical algebra, with possibly real-valued exponents. Motivated by the discussion in (arXiv:1402.1869), this approach enables us to refine their upper bounds on linear regions of layers with ReLU or leaky ReLU activations to minleft 2m, sumj=0n binommj right, where n, m are the number of inputs and outputs, respectively. Additionally, we recover their upper bounds on maxout layers. Our work follows a novel path, exclusively under the lens of tropical geometry, which is independent of the improvements reported in (arXiv:1611.01491, arXiv:1711.02114). Finally, we present a geometric approach for effective counting of linear regions using random sampling in order to avoid the computational overhead of exact counting approaches