# Unconstrained Monotonic Neural Networks

arXiv: https://arxiv.org/abs/1908.05164 · HTML: https://mlsanity.com/paper/1908.05164/

## Key facts

- Cited by 92 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Aug 2019.

## Built on by (most cited first, 30 of 92)

- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Better Uncertainty Calibration via Proper Scores for Classification and Beyond](https://mlsanity.com/paper/2203.07835/) (2022)
- [Smooth Normalizing Flows](https://mlsanity.com/paper/2110.00351/) (2021)
- [MALI: A memory efficient and reverse accurate integrator for Neural ODEs](https://mlsanity.com/paper/2102.04668/) (2021)
- [Indeterminacy in Generative Models: Characterization and Strong Identifiability](https://mlsanity.com/paper/2206.00801/) (2022)
- [Addressing Misspecification in Simulation-based Inference through Data-driven Calibration](https://mlsanity.com/paper/2405.08719/) (2024)
- [Feature Necessity & Relevancy in ML Classifier Explanations](https://mlsanity.com/paper/2210.15675/) (2022)
- [Distribution learning via neural differential equations: a nonparametric statistical perspective](https://mlsanity.com/paper/2309.01043/) (2023)
- [Efficient Neural Network Approaches for Conditional Optimal Transport with Applications in Bayesian Inference](https://mlsanity.com/paper/2310.16975/) (2023)
- [A deep generative model for probabilistic energy forecasting in power systems: normalizing flows](https://mlsanity.com/paper/2106.09370/) (2021)
- [On the Universality of Volume-Preserving and Coupling-Based Normalizing Flows](https://mlsanity.com/paper/2402.06578/) (2024)
- [Multi-Asset Spot and Option Market Simulation](https://mlsanity.com/paper/2112.06823/) (2021)
- [Bridging Mean-Field Games and Normalizing Flows with Trajectory Regularization](https://mlsanity.com/paper/2206.14990/) (2022)
- [Taming Hyperparameter Tuning in Continuous Normalizing Flows Using the JKO Scheme](https://mlsanity.com/paper/2211.16757/) (2022)
- [Triangular Flows for Generative Modeling: Statistical Consistency, Smoothness Classes, and Fast Rates](https://mlsanity.com/paper/2112.15595/) (2021)
- [Personalized Public Policy Analysis in Social Sciences using Causal-Graphical Normalizing Flows](https://mlsanity.com/paper/2202.03281/) (2022)
- [Autoregressive Quantile Flows for Predictive Uncertainty Estimation](https://mlsanity.com/paper/2112.04643/) (2021)
- [Learning Security Classifiers with Verified Global Robustness Properties](https://mlsanity.com/paper/2105.11363/) (2021)
- [Learning Temporal Point Processes for Efficient Retrieval of Continuous Time Event Sequences](https://mlsanity.com/paper/2202.11485/) (2022)
- [Sparse Flows: Pruning Continuous-depth Models](https://mlsanity.com/paper/2106.12718/) (2021)
- [Neural Bayes Estimators for Irregular Spatial Data using Graph Neural Networks](https://mlsanity.com/paper/2310.02600/) (2023)
- [Structured Neural Networks for Density Estimation and Causal Inference](https://mlsanity.com/paper/2311.02221/) (2023)
- [Deep Non-Crossing Quantiles through the Partial Derivative](https://mlsanity.com/paper/2201.12848/) (2022)
- [Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flows](https://mlsanity.com/paper/2205.07918/) (2022)
- [Semi-Autoregressive Energy Flows: Exploring Likelihood-Free Training of Normalizing Flows](https://mlsanity.com/paper/2206.06672/) (2022)
- [TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed](https://mlsanity.com/paper/2203.10335/) (2022)
- [Advancing Constrained Monotonic Neural Networks: Achieving Universal Approximation Beyond Bounded Activations](https://mlsanity.com/paper/2505.02537/) (2025)
- [A generative flow for conditional sampling via optimal transport](https://mlsanity.com/paper/2307.04102/) (2023)
- [Expressive Monotonic Neural Networks](https://mlsanity.com/paper/2307.07512/) (2023)
- [Transformer Neural Autoregressive Flows](https://mlsanity.com/paper/2401.01855/) (2024)

## Abstract (arXiv)

Monotonic neural networks have recently been proposed as a way to define invertible transformations. These transformations can be combined into powerful autoregressive flows that have been shown to be universal approximators of continuous probability distributions. Architectures that ensure monotonicity typically enforce constraints on weights and activation functions, which enables invertibility but leads to a cap on the expressiveness of the resulting transformations. In this work, we propose the Unconstrained Monotonic Neural Network (UMNN) architecture based on the insight that a function is monotonic as long as its derivative is strictly positive. In particular, this latter condition can be enforced with a free-form neural network whose only constraint is the positiveness of its output. We evaluate our new invertible building block within a new autoregressive flow (UMNN-MAF) and demonstrate its effectiveness on density estimation experiments. We also illustrate the ability of UMNNs to improve variational inference.