mlsanityData to 14 Oct 2026

Embracing the chaos: analysis and diagnosis of numerical instability in variational flows

arXiv 2307.06957 ↗First posted 12 Jul 2023stat.ML cs.LG cs.NAcited by 6

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate the impact of numerical instability on the reliability of sampling, density evaluation, and evidence lower bound (ELBO) estimation in variational flows. We first empirically demonstrate that common flows can exhibit a catastrophic accumulation of error: the numerical flow map deviates significantly from the exact map -- which affects sampling -- and the numerical inverse flow map does not accurately recover the initial input -- which affects density and ELBO computations. Surprisingly though, we find that results produced by flows are often accurate enough for applications despite the presence of serious numerical instability. In this work, we treat variational flows as dynamical systems, and leverage shadowing theory to elucidate this behavior via theoretical guarantees on the error of sampling, density evaluation, and ELBO estimation. Finally, we develop and empirically test a diagnostic procedure that can be used to validate results produced by numerically unstable flows in practice.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 25 of its 57 references that are in the map. First posted on arXiv 12 Jul 2023.

Builds on

25 of 57 references.

References · 25

  1. 2018Neural Ordinary Differential Equationscited by 3,779
  2. 2015Variational Inference with Normalizing Flowscited by 2,170
  3. 2016Density estimation using Real NVPcited by 2,042
  4. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
  5. 2019Neural Spline Flowscited by 596
  6. 2013Black Box Variational Inferencecited by 319
  7. 2022On Neural Differential Equationscited by 268
  8. 2017The Reversible Residual Network: Backpropagation Without Storing Activationscited by 217
  9. 2016Automatic Differentiation Variational Inferencecited by 159
  10. 2020Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODEcited by 76
  11. 2018Sylvester Normalizing Flows for Variational Inferencecited by 72
  12. 2020Understanding and Mitigating Exploding Inverses in Invertible Neural Networkscited by 71
13 more
  1. 2017Reversible Architectures for Arbitrarily Deep Residual Neural Networkscited by 70
  2. 2021Smooth Normalizing Flowscited by 53
  3. 2021MALI: A memory efficient and reverse accurate integrator for Neural ODEscited by 42
  4. 2018Hamiltonian Variational Auto-Encodercited by 41
  5. 2021MCMC Variational Inference via Uncorrected Hamiltonian Annealingcited by 39
  6. 2021Monte Carlo Variational Auto-Encoderscited by 36
  7. 2021Differentiable Annealed Importance Sampling and the Perils of Gradient Noisecited by 36
  8. 2020The Expressive Power of a Class of Normalizing Flow Modelscited by 35
  9. 2021iVPF: Numerical Invertible Volume Preserving Flow for Efficient Lossless Compressioncited by 27
  10. 2022Langevin Diffusion Variational Inferencecited by 26
  11. 2022Bayesian inference via sparse Hamiltonian flowscited by 12
  12. 2022MixFlows: principled variational inference via mixed flowscited by 11
  13. 2021Robust normalizing flows using Bernstein-type polynomialscited by 6

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 3.

6 papers.

  1. 2025FALCON: Few-step Accurate Likelihoods for Continuous Flowscited by 13
  2. 2025Efficient Regression-Based Training of Normalizing Flows for Boltzmann Generatorscited by 8
  3. 2024Variational Learning of Gaussian Process Latent Variable Models through Stochastic Gradient Annealed Importance Samplingcited by 4
  4. 2023Reparameterized Variational Rejection Samplingcited by 2
  5. 2025Asymptotically exact variational flows via involutive MCMC kernelscited by 1
  6. 2024Generative Topological Networkscited by 0