# On the Decision Boundaries of Neural Networks: A Tropical Geometry Perspective

arXiv: https://arxiv.org/abs/2002.08838 · HTML: https://mlsanity.com/paper/2002.08838/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2020.

## Built on by (most cited first, 25 of 25)

- [Towards Lower Bounds on the Depth of ReLU Neural Networks](https://mlsanity.com/paper/2105.14835/) (2021)
- [Advancing Model Pruning via Bi-level Optimization](https://mlsanity.com/paper/2210.04092/) (2022)
- [The Real Tropical Geometry of Neural Networks](https://mlsanity.com/paper/2403.11871/) (2024)
- [Latent Imitator: Generating Natural Individual Discriminatory Instances for Black-Box Fairness Testing](https://mlsanity.com/paper/2305.11602/) (2023)
- [Activation degree thresholds and expressiveness of polynomial neural networks](https://mlsanity.com/paper/2408.04569/) (2024)
- [Exact Count of Boundary Pieces of ReLU Classifiers: Towards the Proper Complexity Measure for Classification](https://mlsanity.com/paper/2306.08805/) (2023)
- [On Minimal Depth in Neural Networks](https://mlsanity.com/paper/2402.15315/) (2024)
- [Verification of Neural Networks' Global Robustness](https://mlsanity.com/paper/2402.19322/) (2024)
- [On the Local Complexity of Linear Regions in Deep ReLU Networks](https://mlsanity.com/paper/2412.18283/) (2024)
- [The Geometric Structure of Fully-Connected ReLU Layers](https://mlsanity.com/paper/2310.03482/) (2023)
- [TropNNC: Structured Neural Network Compression Using Tropical Geometry](https://mlsanity.com/paper/2409.03945/) (2024)
- [Sparsity is Combinatorial Depth: Quantifying MoE Expressivity via Tropical Geometry](https://mlsanity.com/paper/2602.03204/) (2026)
- [Revisiting Tropical Polynomial Division: Theory, Algorithms and Application to Neural Networks](https://mlsanity.com/paper/2306.15157/) (2023)
- [Doubly Robust Instance-Reweighted Adversarial Training](https://mlsanity.com/paper/2308.00311/) (2023)
- [Defining Neural Network Architecture through Polytope Structures of Dataset](https://mlsanity.com/paper/2402.02407/) (2024)
- [Concentration inequalities and optimal number of layers for stochastic deep neural networks](https://mlsanity.com/paper/2206.11241/) (2022)
- [Invisible Watermarks, Visible Gains: Steering Machine Unlearning with Bi-Level Watermarking Design](https://mlsanity.com/paper/2508.10065/) (2025)
- [From Universal Approximation Theorem to Tropical Geometry of Multi-Layer Perceptrons](https://mlsanity.com/paper/2510.15012/) (2025)
- [Geometric Capacity of Transformers: A Tropical Geometry Perspective](https://mlsanity.com/paper/2604.14727/) (2026)
- [The Vanishing Decision Boundary Complexity and the Strong First Component](https://mlsanity.com/paper/2211.16209/) (2022)
- [Adapter Pruning using Tropical Characterization](https://mlsanity.com/paper/2310.19232/) (2023)
- [Compound and Parallel Modes of Tropical Convolutional Neural Networks](https://mlsanity.com/paper/2504.06881/) (2025)
- [Algebraic Approach to Ridge-Regularized Mean Squared Error Minimization in Minimal ReLU Neural Network](https://mlsanity.com/paper/2508.17783/) (2025)
- [ShapDBM: Exploring Decision Boundary Maps in Shapley Space](https://mlsanity.com/paper/2603.22235/) (2026)
- [Patchwork: A compact representation for 3D polygonal shapes](https://mlsanity.com/paper/2605.16266/) (2026)

## Abstract (arXiv)

This work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piecewise linear non-linearity activations. We use tropical geometry, a new development in the area of algebraic geometry, to characterize the decision boundaries of a simple network of the form (Affine, ReLU, Affine). Our main finding is that the decision boundaries are a subset of a tropical hypersurface, which is intimately related to a polytope formed by the convex hull of two zonotopes. The generators of these zonotopes are functions of the network parameters. This geometric characterization provides new perspectives to three tasks. (i) We propose a new tropical perspective to the lottery ticket hypothesis, where we view the effect of different initializations on the tropical geometric representation of a network's decision boundaries. (ii) Moreover, we propose new tropical based optimization reformulations that directly influence the decision boundaries of the network for the task of network pruning. (iii) At last, we discuss the reformulation of the generation of adversarial attacks in a tropical sense. We demonstrate that one can construct adversaries in a new tropical setting by perturbing a specific set of decision boundaries by perturbing a set of parameters in the network.