# Constrained Monotonic Neural Networks

arXiv: https://arxiv.org/abs/2205.11775 · HTML: https://mlsanity.com/paper/2205.11775/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 56 references that are in the map.
- First posted on arXiv 24 May 2022.

## Builds on (most cited first, 20 of 20)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Input Convex Neural Networks](https://mlsanity.com/paper/1609.07152/) (2016)
- [A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay](https://mlsanity.com/paper/1803.09820/) (2018)
- [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019)
- [Searching for Activation Functions](https://mlsanity.com/paper/1710.05941/) (2017)
- [Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units](https://mlsanity.com/paper/1603.05201/) (2016)
- [Deep Lattice Networks and Partial Monotonic Functions](https://mlsanity.com/paper/1709.06680/) (2017)
- [The age of secrecy and unfairness in recidivism prediction](https://mlsanity.com/paper/1811.00731/) (2018)
- [Certified Monotonic Neural Networks](https://mlsanity.com/paper/2011.10219/) (2020)
- [DiracNets: Training Very Deep Neural Networks Without Skip-Connections](https://mlsanity.com/paper/1706.00388/) (2017)
- [Monotonic Calibrated Interpolated Look-Up Tables](https://mlsanity.com/paper/1505.06378/) (2015)
- [Counterexample-Guided Learning of Monotonic Neural Networks](https://mlsanity.com/paper/2006.08852/) (2020)
- [How to Incorporate Monotonicity in Deep Networks While Preserving Flexibility?](https://mlsanity.com/paper/1909.10662/) (2019)
- [Shifting Mean Activation Towards Zero with Bipolar Activation Functions](https://mlsanity.com/paper/1709.04054/) (2017)

## Built on by (most cited first, 30 of 32)

- [Structure-Preserving Digital Twins via Conditional Neural Whitney Forms](https://mlsanity.com/paper/2508.06981/) (2025)
- [Data-driven particle dynamics: Structure-preserving coarse-graining for emergent behavior in non-equilibrium systems](https://mlsanity.com/paper/2508.12569/) (2025)
- [Monodense Deep Neural Model for Determining Item Price Elasticity](https://mlsanity.com/paper/2603.29261/) (2026)
- [A Deep Learning Analysis of Climate Change, Innovation, and Uncertainty](https://mlsanity.com/paper/2310.13200/) (2023)
- [Learning truly monotone operators with applications to nonlinear inverse problems](https://mlsanity.com/paper/2404.00390/) (2024)
- [A mathematical certification for positivity conditions in Neural Networks with applications to partial monotonicity and Trustworthy AI](https://mlsanity.com/paper/2406.08525/) (2024)
- [Aligning the Evaluation of Probabilistic Predictions with Downstream Value](https://mlsanity.com/paper/2508.18251/) (2025)
- [On Aligning Prediction Models with Clinical Experiential Learning: A Prostate Cancer Case Study](https://mlsanity.com/paper/2509.04053/) (2025)
- [Learning to Recorrupt: Noise Distribution Agnostic Self-Supervised Image Denoising](https://mlsanity.com/paper/2603.25869/) (2026)
- [Monotonic Kolmogorov-Arnold Networks: A Theoretical and Empirical Study of Monotonicity as an Inductive Bias](https://mlsanity.com/paper/2606.17886/) (2026)
- [Optimization- and AI-based approaches to academic quality quantification for transparent academic recruitment: part 1-model development](https://mlsanity.com/paper/2305.05460/) (2023)
- [Introducing Flexible Monotone Multiple Choice Item Response Theory Models and Bit Scales](https://mlsanity.com/paper/2410.01480/) (2024)
- [LucidAtlas: Learning Uncertainty-Aware, Covariate-Disentangled, Individualized Atlas Representations](https://mlsanity.com/paper/2502.08445/) (2025)
- [A Monotone Single-Index Modal Regression Powered by Deep Neural Networks for Non-Gaussian Periodontal Data](https://mlsanity.com/paper/2505.02153/) (2025)
- [Fast Gaussian Processes under Monotonicity Constraints](https://mlsanity.com/paper/2507.06677/) (2025)
- [MonoCon: A general framework for learning ultra-compact high-fidelity representations using monotonicity constraints](https://mlsanity.com/paper/2509.22931/) (2025)
- [Computing frustration and near-monotonicity in deep neural networks](https://mlsanity.com/paper/2510.05286/) (2025)
- [WTNN: Weibull-Tailored Neural Networks for survival analysis](https://mlsanity.com/paper/2512.09163/) (2025)
- [Efficient reformulations of ReLU deep neural networks for surrogate modelling in power system optimisation](https://mlsanity.com/paper/2601.14673/) (2026)
- [Monotone Optimisation with Learned Projections](https://mlsanity.com/paper/2601.20983/) (2026)
- [Directed Ordinal Diffusion Regularization for Progression-Aware Diabetic Retinopathy Grading](https://mlsanity.com/paper/2602.21942/) (2026)
- [Learning What Evaluators Value: A Reliable Approach to Modeling Evaluator Preferences](https://mlsanity.com/paper/2605.16615/) (2026)
- [Neural Slack Variables for Shape Constraints](https://mlsanity.com/paper/2606.13803/) (2026)
- [Profit-Based Counterfactual Explanations for Product Improvement: A Case Study of Manga Sales in Japan](https://mlsanity.com/paper/2607.01610/) (2026)
- [LRX-PINN: A Layer-Resolving XNet Physics-Informed Neural Network with Integrated Cauchy Activations for Convection-Dominated Problems](https://mlsanity.com/paper/2607.03682/) (2026)
- [Physically Constrained Federated Additive Models for O-RAN SLA-Risk Prediction](https://mlsanity.com/paper/2607.21665/) (2026)
- [CC-AOS: Cost- and Horizon-Conditioned Amortized Backward Induction for Finite-Horizon Optimal Stopping](https://mlsanity.com/paper/2607.22774/) (2026)
- [Diagnosing Under-Development of Irreversible Processes in Video Generation](https://mlsanity.com/paper/2608.00617/) (2026)
- [Differentiable Fuzzy Inference Layer: A Monotone, Compositional Ordinal Reasoning Head for Large Language Models](https://mlsanity.com/paper/2609.26113/) (2026)
- [SLP-ProbHard: Probabilistic Hard-Constrained Learning via Structural Latent Parameterization](https://mlsanity.com/paper/2609.33515/) (2026)

## Abstract (arXiv)

Wider adoption of neural networks in many critical domains such as finance and healthcare is being hindered by the need to explain their predictions and to impose additional constraints on them. Monotonicity constraint is one of the most requested properties in real-world scenarios and is the focus of this paper. One of the oldest ways to construct a monotonic fully connected neural network is to constrain signs on its weights. Unfortunately, this construction does not work with popular non-saturated activation functions as it can only approximate convex functions. We show this shortcoming can be fixed by constructing two additional activation functions from a typical unsaturated monotonic activation function and employing each of them on the part of neurons. Our experiments show this approach of building monotonic neural networks has better accuracy when compared to other state-of-the-art methods, while being the simplest one in the sense of having the least number of parameters, and not requiring any modifications to the learning procedure or post-learning steps. Finally, we prove it can approximate any continuous monotone function on a compact subset of ℝn.