mlsanityData to 7 Oct 2026

Implicit Normalizing Flows

arXiv 2103.09527 ↗First posted 17 Mar 2021stat.ML cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Normalizing flows define a probability distribution by an explicit invertible transformation boldsymbolz=f(boldsymbolx). In this work, we present implicit normalizing flows (ImpFlows), which generalize normalizing flows by allowing the mapping to be implicitly defined by the roots of an equation F(boldsymbolz, boldsymbolx)= boldsymbol0. ImpFlows build on residual flows (ResFlows) with a proper balance between expressiveness and tractability. Through theoretical analysis, we show that the function space of ImpFlow is strictly richer than that of ResFlows. Furthermore, for any ResFlow with a fixed number of blocks, there exists some function that ResFlow has a non-negligible approximation error. However, the function is exactly representable by a single-block ImpFlow. We propose a scalable algorithm to train and draw samples from ImpFlows. Empirically, we evaluate ImpFlow on several classification and density modeling tasks, and ImpFlow outperforms ResFlow with a comparable amount of parameters on all the benchmarks.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 29 of its 40 references that are in the map. First posted on arXiv 17 Mar 2021.

Builds on

29 of 40 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018Neural Ordinary Differential Equationscited by 3,733
  3. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  4. 2015Variational Inference with Normalizing Flowscited by 2,161
  5. 2016Density estimation using Real NVPcited by 2,029
  6. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  7. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  8. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  9. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  10. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  11. 2019Deep Equilibrium Modelscited by 603
  12. 2019Neural Spline Flowscited by 590
17 more
  1. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 588
  2. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
  3. 2018Invertible Residual Networkscited by 364
  4. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
  5. 2019Residual Flows for Invertible Generative Modelingcited by 256
  6. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  7. 2018Neural Autoregressive Flowscited by 207
  8. 2019SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solvercited by 173
  9. 2020SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flowscited by 71
  10. 2020Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Modelscited by 66
  11. 2019Block Neural Autoregressive Flowcited by 62
  12. 2019Approximation Capabilities of Neural ODEs and Invertible Residual Networkscited by 56
  13. 2020VFlow: More Expressive Generative Flows with Variational Data Augmentationcited by 38
  14. 2019MintNet: Building Invertible Neural Networks with Masked Convolutionscited by 35
  15. 2019Emerging Convolutions for Generative Normalizing Flowscited by 31
  16. 2017Continuous-Time Flows for Efficient Inference and Density Estimationcited by 21
  17. 2019Robust learning with implicit residual networkscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 11; 2023: 6; 2024: 2; 2025: 2; 2026: 1.

28 papers.

Sort:

By area:

  1. 2021On Training Implicit Modelscited by 73
  2. 2021Stabilizing Equilibrium Models by Jacobian Regularizationcited by 71
  3. 2022Neural Conservation Laws: A Divergence-Free Perspectivecited by 67
  4. 2022Deep Equilibrium Optical Flow Estimationcited by 63
  5. 2023One-Step Diffusion Distillation via Deep Equilibrium Modelscited by 60
  6. 2021Rectangular Flows for Manifold Learningcited by 49
  7. 2022Maximum Likelihood Training of Implicit Nonlinear Diffusion Modelscited by 42
  8. 2022Deep Equilibrium Approaches to Diffusion Modelscited by 42
  9. 2022Can Push-forward Generative Models Fit Multimodal Distributions?cited by 41
  10. 2023TorchDEQ: A Library for Deep Equilibrium Modelscited by 21
  11. 2023Deep Equilibrium Diffusion Restoration with Parallel Samplingcited by 17
  12. 2021Joint inference and input optimization in equilibrium networkscited by 15
16 more
  1. 2022Invertible Rescaling Network and Its Extensionscited by 15
  2. 2023A Closer Look at the Adversarial Robustness of Deep Equilibrium Modelscited by 13
  3. 2022Out-of-Distribution Detection with Reconstruction Error and Typicality-based Penaltycited by 11
  4. 2024Monotone, Bi-Lipschitz, and Polyak-Lojasiewicz Networkscited by 9
  5. 2021Can convolutional ResNets approximately preserve input distances? A frequency analysis perspectivecited by 7
  6. 2022Invertible Monotone Operators for Normalizing Flowscited by 6
  7. 2025Reversible Deep Equilibrium Modelscited by 5
  8. 2022Fast Lossless Neural Compression with Integer-Only Discrete Flowscited by 4
  9. 2023Deep Equilibrium Multimodal Fusioncited by 4
  10. 2025Differentiable Cyclic Causal Discovery Under Unmeasured Confounderscited by 4
  11. 2023Improving Adversarial Robustness of DEQs with Explicit Regulations Along the Neural Dynamicscited by 2
  12. 2024Learning Stable and Passive Neural Differential Equationscited by 2
  13. 2021Viscos Flows: Variational Schur Conditional Sampling With Normalizing Flowscited by 1
  14. 2026Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approachcited by 1
  15. 2022WeaNF: Weak Supervision with Normalizing Flowscited by 0
  16. 2022Representation Recycling for Streaming Video Analysiscited by 0