# Universal approximation property of invertible neural networks

arXiv: https://arxiv.org/abs/2204.07415 · HTML: https://mlsanity.com/paper/2204.07415/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 74 references that are in the map.
- First posted on arXiv 15 Apr 2022.

## Builds on (most cited first, 30 of 39)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [Analyzing Inverse Problems with Invertible Neural Networks](https://mlsanity.com/paper/1808.04730/) (2018)
- [Parallel WaveNet: Fast High-Fidelity Speech Synthesis](https://mlsanity.com/paper/1711.10433/) (2017)
- [Sorting out Lipschitz function approximation](https://mlsanity.com/paper/1811.05381/) (2018)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Multiplicative Normalizing Flows for Variational Bayesian Neural Networks](https://mlsanity.com/paper/1703.01961/) (2017)
- [Flow-based generative models for Markov chain Monte Carlo in lattice field theory](https://mlsanity.com/paper/1904.12072/) (2019)
- [SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds](https://mlsanity.com/paper/2006.04604/) (2020)
- [Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization](https://mlsanity.com/paper/2012.05942/) (2020)
- [Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators](https://mlsanity.com/paper/2006.11469/) (2020)
- [Deep Learning via Dynamical Systems: An Approximation Perspective](https://mlsanity.com/paper/1912.10382/) (2019)
- [Semi-Supervised Learning with Normalizing Flows](https://mlsanity.com/paper/1912.13025/) (2019)
- [Latent Normalizing Flows for Discrete Sequences](https://mlsanity.com/paper/1901.10548/) (2019)
- [Block Neural Autoregressive Flow](https://mlsanity.com/paper/1904.04676/) (2019)
- [Sum-of-Squares Polynomial Flow](https://mlsanity.com/paper/1905.02325/) (2019)
- [Resampled Priors for Variational Autoencoders](https://mlsanity.com/paper/1810.11428/) (2018)
- [Neural ODE control for classification, approximation and transport](https://mlsanity.com/paper/2104.05278/) (2021)
- [FloWaveNet : A Generative Flow for Raw Audio](https://mlsanity.com/paper/1811.02155/) (2018)
- [Few-shot Domain Adaptation by Causal Mechanism Transfer](https://mlsanity.com/paper/2002.03497/) (2020)
- [Hybrid Models with Deep and Invertible Features](https://mlsanity.com/paper/1902.02767/) (2019)
- [Improving Exploration in Soft-Actor-Critic with Normalizing Flows Policies](https://mlsanity.com/paper/1906.02771/) (2019)
- [Universal Approximation Property of Neural Ordinary Differential Equations](https://mlsanity.com/paper/2012.02414/) (2020)

## Built on by (most cited first, 30 of 30)

- [Learning Linear Causal Representations from Interventions under General Nonlinear Mixing](https://mlsanity.com/paper/2306.02235/) (2023)
- [Identifiability of deep generative models without auxiliary information](https://mlsanity.com/paper/2206.10044/) (2022)
- [Reliable amortized variational inference with physics-based latent distribution correction](https://mlsanity.com/paper/2207.11640/) (2022)
- [Distribution learning via neural differential equations: a nonparametric statistical perspective](https://mlsanity.com/paper/2309.01043/) (2023)
- [On the Universality of Volume-Preserving and Coupling-Based Normalizing Flows](https://mlsanity.com/paper/2402.06578/) (2024)
- [Globally injective and bijective neural operators](https://mlsanity.com/paper/2306.03982/) (2023)
- [Efficient Regression-Based Training of Normalizing Flows for Boltzmann Generators](https://mlsanity.com/paper/2506.01158/) (2025)
- [Enhancing Accuracy in Generative Models via Knowledge Transfer](https://mlsanity.com/paper/2405.16837/) (2024)
- [Whitening Convergence Rate of Coupling-based Normalizing Flows](https://mlsanity.com/paper/2210.14032/) (2022)
- [Stable Training of Normalizing Flows for High-dimensional Variational Inference](https://mlsanity.com/paper/2402.16408/) (2024)
- [Cluster-based classification with neural ODEs via control](https://mlsanity.com/paper/2312.13807/) (2023)
- [Diffeomorphic Measure Matching with Kernels for Generative Modeling](https://mlsanity.com/paper/2402.08077/) (2024)
- [Adaptive deep density approximation for stochastic dynamical systems](https://mlsanity.com/paper/2405.02810/) (2024)
- [Numerical and statistical analysis of NeuralODE with Runge-Kutta time integration](https://mlsanity.com/paper/2503.10729/) (2025)
- [Generative AI for Autonomous Driving: A Review](https://mlsanity.com/paper/2505.15863/) (2025)
- [Enriching Disentanglement: From Logical Definitions to Quantitative Metrics](https://mlsanity.com/paper/2305.11512/) (2023)
- [On the Approximation of Bi-Lipschitz Maps by Invertible Neural Networks](https://mlsanity.com/paper/2308.09367/) (2023)
- [Can neural operators always be continuously discretized?](https://mlsanity.com/paper/2412.03393/) (2024)
- [An invertible generative model for forward and inverse problems](https://mlsanity.com/paper/2509.03910/) (2025)
- [Deep Neural Networks with Symplectic Preservation Properties](https://mlsanity.com/paper/2407.00294/) (2024)
- [Autonomous-Flow-Based Generation](https://mlsanity.com/paper/2511.09902/) (2025)
- [Exact Fixed-Point Constraints in Neural-ODEs with Provable Universality](https://mlsanity.com/paper/2605.10613/) (2026)
- [Using Autoencoders and AutoDiff to Reconstruct Missing Variables in a Set of Time Series](https://mlsanity.com/paper/2308.10496/) (2023)
- [GradINN: Gradient Informed Neural Network](https://mlsanity.com/paper/2409.01914/) (2024)
- [RIE-SenseNet: Riemannian Manifold Embedding of Multi-Source Industrial Sensor Signals for Robust Pattern Recognition](https://mlsanity.com/paper/2502.02428/) (2025)
- [Learning Geometrically-Informed Lyapunov Functions with Deep Diffeomorphic RBF Networks](https://mlsanity.com/paper/2504.02607/) (2025)
- [Sampling Boltzmann distributions via normalizing flow approximation of transport maps](https://mlsanity.com/paper/2603.14258/) (2026)
- [Expressivity of Bi-Lipschitz Normalizing Flows: A Score-Based Diffusion Perspective](https://mlsanity.com/paper/2605.06172/) (2026)
- [Approximating velocity fields with planted attractors via Neural-ODEs for classification purposes](https://mlsanity.com/paper/2606.23550/) (2026)
- [Kinks vs. Smoothness: Identifiability of Real Analytic nICA for Laplace-like Sources](https://mlsanity.com/paper/2609.21926/) (2026)

## Abstract (arXiv)

Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning. However, their attractive properties often come at the cost of restricting the layer designs, which poses a question on their representation power: can we use these models to approximate sufficiently diverse functions? To answer this question, we have developed a general theoretical framework to investigate the representation power of INNs, building on a structure theorem of differential geometry. The framework simplifies the approximation problem of diffeomorphisms, which enables us to show the universal approximation properties of INNs. We apply the framework to two representative classes of INNs, namely Coupling-Flow-based INNs (CF-INNs) and Neural Ordinary Differential Equations (NODEs), and elucidate their high representation power despite the restrictions on their architectures.