# Min-Max-Plus Neural Networks

arXiv: https://arxiv.org/abs/2102.06358 · HTML: https://mlsanity.com/paper/2102.06358/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 12 Feb 2021.

## Builds on (most cited first, 14 of 14)

- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [On the Number of Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1402.1869/) (2014)
- [Maxout Networks](https://mlsanity.com/paper/1302.4389/) (2013)
- [The Power of Depth for Feedforward Neural Networks](https://mlsanity.com/paper/1512.03965/) (2015)
- [Tropical Geometry of Deep Neural Networks](https://mlsanity.com/paper/1805.07091/) (2018)
- [AdderNet: Do We Really Need Multiplications in Deep Learning?](https://mlsanity.com/paper/1912.13200/) (2019)
- [A Tropical Approach to Neural Networks with Piecewise Linear Activations](https://mlsanity.com/paper/1805.08749/) (2018)
- [A Universal Approximation Result for Difference of log-sum-exp Neural Networks](https://mlsanity.com/paper/1905.08503/) (2019)
- [Tropical Polynomial Division and Neural Networks](https://mlsanity.com/paper/1911.12922/) (2019)
- [Tropical Geometry and Piecewise-Linear Approximation of Curves and Surfaces on Weighted Lattices](https://mlsanity.com/paper/1912.03891/) (2019)

## Built on by (most cited first, 8 of 8)

- [Multiplication-Free Transformer Training via Piecewise Affine Operations](https://mlsanity.com/paper/2305.17190/) (2023)
- [An Alternative Practice of Tropical Convolution to Traditional Convolutional Neural Networks](https://mlsanity.com/paper/2103.02096/) (2021)
- [Training Deep Morphological Neural Networks as Universal Approximators](https://mlsanity.com/paper/2505.09710/) (2025)
- [Exploiting Subgradient Sparsity in Max-Plus Neural Networks](https://mlsanity.com/paper/2603.04133/) (2026)
- [Compound and Parallel Modes of Tropical Convolutional Neural Networks](https://mlsanity.com/paper/2504.06881/) (2025)
- [On the explainability of max-plus neural networks](https://mlsanity.com/paper/2605.00889/) (2026)
- [Beyond Affine Transformations: A Soft Dominance Layer for Coordinate-Wise Neural Computation](https://mlsanity.com/paper/2610.00563/) (2026)
- [Correlational Training of Morphological Neural Networks](https://mlsanity.com/paper/2610.11740/) (2026)

## Abstract (arXiv)

We present a new model of neural networks called Min-Max-Plus Neural Networks (MMP-NNs) based on operations in tropical arithmetic. In general, an MMP-NN is composed of three types of alternately stacked layers, namely linear layers, min-plus layers and max-plus layers. Specifically, the latter two types of layers constitute the nonlinear part of the network which is trainable and more sophisticated compared to the nonlinear part of conventional neural networks. In addition, we show that with higher capability of nonlinearity expression, MMP-NNs are universal approximators of continuous functions, even when the number of multiplication operations is tremendously reduced (possibly to none in certain extreme cases). Furthermore, we formulate the backpropagation algorithm in the training process of MMP-NNs and introduce an algorithm of normalization to improve the rate of convergence in training.