# How to Incorporate Monotonicity in Deep Networks While Preserving Flexibility?

arXiv: https://arxiv.org/abs/1909.10662 · HTML: https://mlsanity.com/paper/1909.10662/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2019.

## Built on by (most cited first, 21 of 21)

- [Constrained Monotonic Neural Networks](https://mlsanity.com/paper/2205.11775/) (2022)
- [Traversing the Local Polytopes of ReLU Neural Networks: A Unified Approach for Network Verification](https://mlsanity.com/paper/2111.08922/) (2021)
- [Matching Learned Causal Effects of Neural Networks with Domain Priors](https://mlsanity.com/paper/2111.12490/) (2021)
- [Advancing Constrained Monotonic Neural Networks: Achieving Universal Approximation Beyond Bounded Activations](https://mlsanity.com/paper/2505.02537/) (2025)
- [Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions](https://mlsanity.com/paper/2107.09546/) (2021)
- [Expressive Monotonic Neural Networks](https://mlsanity.com/paper/2307.07512/) (2023)
- [Monotonicity Regularization: Improved Penalties and Novel Applications to Disentangled Representation Learning and Robust Classification](https://mlsanity.com/paper/2205.08247/) (2022)
- [Smooth Min-Max Monotonic Networks](https://mlsanity.com/paper/2306.01147/) (2023)
- [Deep neural networks for choice analysis: Enhancing behavioral regularity with gradient regularization](https://mlsanity.com/paper/2404.14701/) (2024)
- [MonoKAN: Certified Monotonic Kolmogorov-Arnold Network](https://mlsanity.com/paper/2409.11078/) (2024)
- [OIL-AD: An Anomaly Detection Framework for Sequential Decision Sequences](https://mlsanity.com/paper/2402.04567/) (2024)
- [A mathematical certification for positivity conditions in Neural Networks with applications to partial monotonicity and Trustworthy AI](https://mlsanity.com/paper/2406.08525/) (2024)
- [Enhancing Monotonic Modeling with Spatio-Temporal Adaptive Awareness in Diverse Marketing](https://mlsanity.com/paper/2406.14132/) (2024)
- [MMCE: A Framework for Deep Monotonic Modeling of Multiple Causal Effects](https://mlsanity.com/paper/2504.03753/) (2025)
- [Learning Monotonic Probabilities with a Generative Cost Model](https://mlsanity.com/paper/2506.03542/) (2025)
- [DIM: Enforcing Domain-Informed Monotonicity in Deep Neural Networks](https://mlsanity.com/paper/2509.21666/) (2025)
- [Negotiating Networks in Oligopoly Markets for Price-Sensitive Products](https://mlsanity.com/paper/2110.13303/) (2021)
- [Multi-Horizon Time Series Forecasting of non-parametric CDFs with Deep Lattice Networks](https://mlsanity.com/paper/2511.13756/) (2025)
- [WTNN: Weibull-Tailored Neural Networks for survival analysis](https://mlsanity.com/paper/2512.09163/) (2025)
- [MLPlatt: Simple Calibration Framework for Ranking Models](https://mlsanity.com/paper/2601.08345/) (2026)
- [R&F-Inventory: A Large-Scale Dataset for Monotonic Inventory Estimation in Reach and Frequency Advertising](https://mlsanity.com/paper/2604.16821/) (2026)

## Abstract (arXiv)

The importance of domain knowledge in enhancing model performance and making reliable predictions in the real-world is critical. This has led to an increased focus on specific model properties for interpretability. We focus on incorporating monotonic trends, and propose a novel gradient-based point-wise loss function for enforcing partial monotonicity with deep neural networks. While recent developments have relied on structural changes to the model, our approach aims at enhancing the learning process. Our model-agnostic point-wise loss function acts as a plug-in to the standard loss and penalizes non-monotonic gradients. We demonstrate that the point-wise loss produces comparable (and sometimes better) results on both AUC and monotonicity measure, as opposed to state-of-the-art deep lattice networks that guarantee monotonicity. Moreover, it is able to learn differentiated individual trends and produces smoother conditional curves which are important for personalized decisions, while preserving the flexibility of deep networks.