# Expressive Monotonic Neural Networks

arXiv: https://arxiv.org/abs/2307.07512 · HTML: https://mlsanity.com/paper/2307.07512/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 14 Jul 2023.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [Regularisation of Neural Networks by Enforcing Lipschitz Continuity](https://mlsanity.com/paper/1804.04368/) (2018)
- [Sorting out Lipschitz function approximation](https://mlsanity.com/paper/1811.05381/) (2018)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Unconstrained Monotonic Neural Networks](https://mlsanity.com/paper/1908.05164/) (2019)
- [Deep Lattice Networks and Partial Monotonic Functions](https://mlsanity.com/paper/1709.06680/) (2017)
- [Block Neural Autoregressive Flow](https://mlsanity.com/paper/1904.04676/) (2019)
- [Certified Monotonic Neural Networks](https://mlsanity.com/paper/2011.10219/) (2020)
- [Limitations of the Lipschitz constant as a defense against adversarial examples](https://mlsanity.com/paper/1807.09705/) (2018)
- [Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100](https://mlsanity.com/paper/2108.04062/) (2021)
- [How to Initialize your Network? Robust Initialization for WeightNorm & ResNets](https://mlsanity.com/paper/1906.02341/) (2019)
- [Counterexample-Guided Learning of Monotonic Neural Networks](https://mlsanity.com/paper/2006.08852/) (2020)
- [How to Incorporate Monotonicity in Deep Networks While Preserving Flexibility?](https://mlsanity.com/paper/1909.10662/) (2019)
- [MonoNet: Towards Interpretable Models by Learning Monotonic Features](https://mlsanity.com/paper/1909.13611/) (2019)
- [Robust and Provably Monotonic Networks](https://mlsanity.com/paper/2112.00038/) (2021)
- [A better method to enforce monotonic constraints in regression and classification trees](https://mlsanity.com/paper/2011.00986/) (2020)

## Built on by (most cited first, 7 of 7)

- [Advancing Constrained Monotonic Neural Networks: Achieving Universal Approximation Beyond Bounded Activations](https://mlsanity.com/paper/2505.02537/) (2025)
- [Cross-Model Semantics in Representation Learning](https://mlsanity.com/paper/2508.03649/) (2025)
- [Latent Linear Quadratic Regulator for Robotic Control Tasks](https://mlsanity.com/paper/2407.11107/) (2024)
- [Positional Encoder Graph Quantile Neural Networks for Geographic Data](https://mlsanity.com/paper/2409.18865/) (2024)
- [LucidAtlas: Learning Uncertainty-Aware, Covariate-Disentangled, Individualized Atlas Representations](https://mlsanity.com/paper/2502.08445/) (2025)
- [Fast Gaussian Processes under Monotonicity Constraints](https://mlsanity.com/paper/2507.06677/) (2025)
- [MonoCon: A general framework for learning ultra-compact high-fidelity representations using monotonicity constraints](https://mlsanity.com/paper/2509.22931/) (2025)

## Abstract (arXiv)

The monotonic dependence of the outputs of a neural network on some of its inputs is a crucial inductive bias in many scenarios where domain knowledge dictates such behavior. This is especially important for interpretability and fairness considerations. In a broader context, scenarios in which monotonicity is important can be found in finance, medicine, physics, and other disciplines. It is thus desirable to build neural network architectures that implement this inductive bias provably. In this work, we propose a weight-constrained architecture with a single residual connection to achieve exact monotonic dependence in any subset of the inputs. The weight constraint scheme directly controls the Lipschitz constant of the neural network and thus provides the additional benefit of robustness. Compared to currently existing techniques used for monotonicity, our method is simpler in implementation and in theory foundations, has negligible computational overhead, is guaranteed to produce monotonic dependence, and is highly expressive. We show how the algorithm is used to train powerful, robust, and interpretable discriminators that achieve competitive performance compared to current state-of-the-art methods across various benchmarks, from social applications to the classification of the decays of subatomic particles produced at the CERN Large Hadron Collider.