# Implicit Normalizing Flows

arXiv: https://arxiv.org/abs/2103.09527 · HTML: https://mlsanity.com/paper/2103.09527/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 40 references that are in the map.
- First posted on arXiv 17 Mar 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [Residual Flows for Invertible Generative Modeling](https://mlsanity.com/paper/1906.02735/) (2019)
- [How to train your neural ODE: the world of Jacobian and kinetic regularization](https://mlsanity.com/paper/2002.02798/) (2020)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver](https://mlsanity.com/paper/1905.12149/) (2019)
- [SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows](https://mlsanity.com/paper/2007.02731/) (2020)
- [Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models](https://mlsanity.com/paper/2002.07101/) (2020)
- [Block Neural Autoregressive Flow](https://mlsanity.com/paper/1904.04676/) (2019)
- [Approximation Capabilities of Neural ODEs and Invertible Residual Networks](https://mlsanity.com/paper/1907.12998/) (2019)
- [VFlow: More Expressive Generative Flows with Variational Data Augmentation](https://mlsanity.com/paper/2002.09741/) (2020)
- [MintNet: Building Invertible Neural Networks with Masked Convolutions](https://mlsanity.com/paper/1907.07945/) (2019)
- [Emerging Convolutions for Generative Normalizing Flows](https://mlsanity.com/paper/1901.11137/) (2019)
- [Continuous-Time Flows for Efficient Inference and Density Estimation](https://mlsanity.com/paper/1709.01179/) (2017)
- [Robust learning with implicit residual networks](https://mlsanity.com/paper/1905.10479/) (2019)

## Built on by (most cited first, 28 of 28)

- [On Training Implicit Models](https://mlsanity.com/paper/2111.05177/) (2021)
- [Stabilizing Equilibrium Models by Jacobian Regularization](https://mlsanity.com/paper/2106.14342/) (2021)
- [Neural Conservation Laws: A Divergence-Free Perspective](https://mlsanity.com/paper/2210.01741/) (2022)
- [Deep Equilibrium Optical Flow Estimation](https://mlsanity.com/paper/2204.08442/) (2022)
- [One-Step Diffusion Distillation via Deep Equilibrium Models](https://mlsanity.com/paper/2401.08639/) (2023)
- [Rectangular Flows for Manifold Learning](https://mlsanity.com/paper/2106.01413/) (2021)
- [Maximum Likelihood Training of Implicit Nonlinear Diffusion Models](https://mlsanity.com/paper/2205.13699/) (2022)
- [Deep Equilibrium Approaches to Diffusion Models](https://mlsanity.com/paper/2210.12867/) (2022)
- [Can Push-forward Generative Models Fit Multimodal Distributions?](https://mlsanity.com/paper/2206.14476/) (2022)
- [TorchDEQ: A Library for Deep Equilibrium Models](https://mlsanity.com/paper/2310.18605/) (2023)
- [Deep Equilibrium Diffusion Restoration with Parallel Sampling](https://mlsanity.com/paper/2311.11600/) (2023)
- [Joint inference and input optimization in equilibrium networks](https://mlsanity.com/paper/2111.13236/) (2021)
- [Invertible Rescaling Network and Its Extensions](https://mlsanity.com/paper/2210.04188/) (2022)
- [A Closer Look at the Adversarial Robustness of Deep Equilibrium Models](https://mlsanity.com/paper/2306.01429/) (2023)
- [Out-of-Distribution Detection with Reconstruction Error and Typicality-based Penalty](https://mlsanity.com/paper/2212.12641/) (2022)
- [Monotone, Bi-Lipschitz, and Polyak-Lojasiewicz Networks](https://mlsanity.com/paper/2402.01344/) (2024)
- [Can convolutional ResNets approximately preserve input distances? A frequency analysis perspective](https://mlsanity.com/paper/2106.02469/) (2021)
- [Invertible Monotone Operators for Normalizing Flows](https://mlsanity.com/paper/2210.08176/) (2022)
- [Reversible Deep Equilibrium Models](https://mlsanity.com/paper/2509.12917/) (2025)
- [Fast Lossless Neural Compression with Integer-Only Discrete Flows](https://mlsanity.com/paper/2206.08869/) (2022)
- [Deep Equilibrium Multimodal Fusion](https://mlsanity.com/paper/2306.16645/) (2023)
- [Differentiable Cyclic Causal Discovery Under Unmeasured Confounders](https://mlsanity.com/paper/2508.08450/) (2025)
- [Improving Adversarial Robustness of DEQs with Explicit Regulations Along the Neural Dynamics](https://mlsanity.com/paper/2306.01435/) (2023)
- [Learning Stable and Passive Neural Differential Equations](https://mlsanity.com/paper/2404.12554/) (2024)
- [Viscos Flows: Variational Schur Conditional Sampling With Normalizing Flows](https://mlsanity.com/paper/2107.02474/) (2021)
- [Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approach](https://mlsanity.com/paper/2602.03297/) (2026)
- [WeaNF: Weak Supervision with Normalizing Flows](https://mlsanity.com/paper/2204.13409/) (2022)
- [Representation Recycling for Streaming Video Analysis](https://mlsanity.com/paper/2204.13492/) (2022)

## Abstract (arXiv)

Normalizing flows define a probability distribution by an explicit invertible transformation boldsymbolz=f(boldsymbolx). In this work, we present implicit normalizing flows (ImpFlows), which generalize normalizing flows by allowing the mapping to be implicitly defined by the roots of an equation F(boldsymbolz, boldsymbolx)= boldsymbol0. ImpFlows build on residual flows (ResFlows) with a proper balance between expressiveness and tractability. Through theoretical analysis, we show that the function space of ImpFlow is strictly richer than that of ResFlows. Furthermore, for any ResFlow with a fixed number of blocks, there exists some function that ResFlow has a non-negligible approximation error. However, the function is exactly representable by a single-block ImpFlow. We propose a scalable algorithm to train and draw samples from ImpFlows. Empirically, we evaluate ImpFlow on several classification and density modeling tasks, and ImpFlow outperforms ResFlow with a comparable amount of parameters on all the benchmarks.