mlsanityData to 7 Oct 2026

Whitening Convergence Rate of Coupling-based Normalizing Flows

arXiv 2210.14032 ↗First posted 25 Oct 2022cs.LG stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

Coupling-based normalizing flows (e.g. RealNVP) are a popular family of normalizing flow architectures that work surprisingly well in practice. This calls for theoretical understanding. Existing work shows that such flows weakly converge to arbitrary data distributions. However, they make no statement about the stricter convergence criterion used in practice, the maximum likelihood loss. For the first time, we make a quantitative statement about this kind of convergence: We prove that all coupling-based normalizing flows perform whitening of the data distribution (i.e. diagonalize the covariance matrix) and derive corresponding convergence bounds that show a linear convergence rate in the depth of the flow. Numerical experiments demonstrate the implications of our theory and point at open questions.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 24 of its 52 references that are in the map. First posted on arXiv 25 Oct 2022.

Builds on

24 of 52 references.

References · 24

  1. 2016Density estimation using Real NVPcited by 2,029
  2. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  3. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  4. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  5. 2019Neural Spline Flowscited by 590
  6. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
  7. 2018Analyzing Inverse Problems with Invertible Neural Networkscited by 239
  8. 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
  9. 2018Neural Autoregressive Flowscited by 207
  10. 2018Neural Importance Samplingcited by 154
  11. 2019Unconstrained Monotonic Neural Networkscited by 91
  12. 2020Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximatorscited by 85
12 more
  1. 2019Latent Normalizing Flows for Discrete Sequencescited by 66
  2. 2020Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Modelscited by 66
  3. 2019Sum-of-Squares Polynomial Flowcited by 62
  4. 2019Approximation Capabilities of Neural ODEs and Invertible Residual Networkscited by 56
  5. 2019Cubic-Spline Flowscited by 34
  6. 2020Universal Approximation Property of Neural Ordinary Differential Equationscited by 32
  7. 2022Universal approximation property of invertible neural networkscited by 30
  8. 2020Representational aspects of depth and conditioning in normalizing flowscited by 25
  9. 2020Gaussianization Flowscited by 20
  10. 2020Learning Robust Models Using The Principle of Independent Causal Mechanismscited by 18
  11. 2021Universal Approximation for Log-concave Distributions using Well-conditioned Normalizing Flowscited by 7
  12. 2019Uncertainty-aware performance assessment of optical imaging modalities with invertible neural networkscited by 5

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 1.

6 papers.

  1. 2023JANA: Jointly Amortized Neural Approximation of Complex Bayesian Modelscited by 41
  2. 2023Free-form Flows: Make Any Architecture a Normalizing Flowcited by 26
  3. 2024On the Universality of Volume-Preserving and Coupling-Based Normalizing Flowscited by 18
  4. 2023Lifting Architectural Constraints of Injective Flowscited by 17
  5. 2023On the Convergence Rate of Gaussianization with Random Rotationscited by 5
  6. 2023On the Approximation of Bi-Lipschitz Maps by Invertible Neural Networkscited by 2