# On the expressivity of bi-Lipschitz normalizing flows

arXiv: https://arxiv.org/abs/2107.07232 · HTML: https://mlsanity.com/paper/2107.07232/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 15 Jul 2021.

## Builds on (most cited first, 24 of 24)

- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [Lipschitz regularity of deep neural networks: analysis and efficient estimation](https://mlsanity.com/paper/1805.10965/) (2018)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [Residual Flows for Invertible Generative Modeling](https://mlsanity.com/paper/1906.02735/) (2019)
- [Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators](https://mlsanity.com/paper/2006.11469/) (2020)
- [Semi-Supervised Learning with Normalizing Flows](https://mlsanity.com/paper/1912.13025/) (2019)
- [Understanding and Mitigating Exploding Inverses in Invertible Neural Networks](https://mlsanity.com/paper/2006.09347/) (2020)
- [Generalizable Adversarial Training via Spectral Normalization](https://mlsanity.com/paper/1811.07457/) (2018)
- [Approximation Capabilities of Neural ODEs and Invertible Residual Networks](https://mlsanity.com/paper/1907.12998/) (2019)
- [Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operators](https://mlsanity.com/paper/1903.01014/) (2019)
- [The Expressive Power of a Class of Normalizing Flow Models](https://mlsanity.com/paper/2006.00392/) (2020)
- [Disconnected Manifold Learning for Generative Adversarial Networks](https://mlsanity.com/paper/1806.00880/) (2018)
- [Learning disconnected manifolds: a no GANs land](https://mlsanity.com/paper/2006.04596/) (2020)
- [Representational aspects of depth and conditioning in normalizing flows](https://mlsanity.com/paper/2010.01155/) (2020)
- [A RAD approach to deep mixture models](https://mlsanity.com/paper/1903.07714/) (2019)
- [Variational Mixture of Normalizing Flows](https://mlsanity.com/paper/2009.00585/) (2020)
- [Universal Approximation of Residual Flows in Maximum Mean Discrepancy](https://mlsanity.com/paper/2103.05793/) (2021)

## Built on by (most cited first, 16 of 16)

- [A Dynamical System Perspective for Lipschitz Neural Networks](https://mlsanity.com/paper/2110.12690/) (2021)
- [A mean-field games laboratory for generative modeling](https://mlsanity.com/paper/2304.13534/) (2023)
- [Out-of-Distribution Detection with Reconstruction Error and Typicality-based Penalty](https://mlsanity.com/paper/2212.12641/) (2022)
- [Simulation-based Inference for Cardiovascular Models](https://mlsanity.com/paper/2307.13918/) (2023)
- [Precision-Recall Divergence Optimization for Generative Modeling with GANs and Normalizing Flows](https://mlsanity.com/paper/2305.18910/) (2023)
- [Intention-Conditioned Flow Occupancy Models](https://mlsanity.com/paper/2506.08902/) (2025)
- [Vec2Face-v2: Unveil Human Faces from their Blackbox Features via Attention-based Network in Face Recognition](https://mlsanity.com/paper/2209.04920/) (2022)
- [NF-ULA: Langevin Monte Carlo with Normalizing Flow Prior for Imaging Inverse Problems](https://mlsanity.com/paper/2304.08342/) (2023)
- [Learning to Bound: A Generative Cram'er-Rao Bound](https://mlsanity.com/paper/2203.03695/) (2022)
- [Ergodic Generative Flows](https://mlsanity.com/paper/2505.03561/) (2025)
- [Master's Thesis: Out-of-distribution Detection with Energy-based Models](https://mlsanity.com/paper/2302.12002/) (2023)
- [Rare event modeling with self-regularized normalizing flows: what can we learn from a single failure?](https://mlsanity.com/paper/2502.21110/) (2025)
- [Distance-Preserving Representations for Genomic Spatial Reconstruction](https://mlsanity.com/paper/2408.00911/) (2024)
- [Conditional Distribution Quantization in Machine Learning](https://mlsanity.com/paper/2502.07151/) (2025)
- [Expressivity of Bi-Lipschitz Normalizing Flows: A Score-Based Diffusion Perspective](https://mlsanity.com/paper/2605.06172/) (2026)
- [From Transformers to Weighted Automata: Towards the Verification of Large Language Models](https://mlsanity.com/paper/2610.04569/) (2026)

## Abstract (arXiv)

An invertible function is bi-Lipschitz if both the function and its inverse have bounded Lipschitz constants. Nowadays, most Normalizing Flows are bi-Lipschitz by design or by training to limit numerical errors (among other things). In this paper, we discuss the expressivity of bi-Lipschitz Normalizing Flows and identify several target distributions that are difficult to approximate using such models. Then, we characterize the expressivity of bi-Lipschitz Normalizing Flows by giving several lower bounds on the Total Variation distance between these particularly unfavorable distributions and their best possible approximation. Finally, we discuss potential remedies which include using more complex latent distributions.