mlsanityData to 14 Oct 2026

Disentangling impact of capacity, objective, batchsize, estimators, and step-size on flow VI

arXiv 2412.08824 ↗First posted 11 Dec 2024cs.LG stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

Normalizing flow-based variational inference (flow VI) is a promising approximate inference approach, but its performance remains inconsistent across studies. Numerous algorithmic choices influence flow VI's performance. We conduct a step-by-step analysis to disentangle the impact of some of the key factors: capacity, objectives, gradient estimators, number of gradient estimates (batchsize), and step-sizes. Each step examines one factor while neutralizing others using insights from the previous steps and/or using extensive parallel computation. To facilitate high-fidelity evaluation, we curate a benchmark of synthetic targets that represent common posterior pathologies and allow for exact sampling. We provide specific recommendations for different factors and propose a flow VI recipe that matches or surpasses leading turnkey Hamiltonian Monte Carlo (HMC) methods.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 44 of its 101 references that are in the map. First posted on arXiv 11 Dec 2024.

Builds on

44 of 101 references.

References · 44

  1. 2016Density estimation using Real NVPcited by 2,042
  2. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
  3. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  4. 2015Importance Weighted Autoencoderscited by 456
  5. 2013Black Box Variational Inferencecited by 319
  6. 2017A Conceptual Introduction to Hamiltonian Monte Carlocited by 209
  7. 2019Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyrocited by 191
  8. 2022Optimal Transport Tools (OTT): A JAX Toolbox for all things Wassersteincited by 101
  9. 2020VarGrad: A Low-Variance Gradient Estimator for Variational Inferencecited by 75
  10. 2019NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transportcited by 74
  11. 2020Understanding and Mitigating Exploding Inverses in Invertible Neural Networkscited by 71
  12. 2022Continual Repeated Annealed Flow Transport Monte Carlocited by 66
32 more
  1. 2024Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Samplingcited by 56
  2. 2022Variational methods for simulation-based inferencecited by 43
  3. 2020Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimizationcited by 37
  4. 2021Local-Global MCMC kernels: the best of both worldscited by 22
  5. 2024Batch and match: black-box variational inference with a score-based divergencecited by 22
  6. 2020Robust, Accurate Stochastic Optimization for Variational Inferencecited by 21
  7. 2020On the Difficulty of Unbiased Alpha Divergence Minimizationcited by 21
  8. 2019Multilevel Monte Carlo Variational Inferencecited by 15
  9. 2021Generalized Doubly Reparameterized Gradient Estimatorscited by 14
  10. 2022Transport Elliptical Slice Samplingcited by 12
  11. 2022MixFlows: principled variational inference via mixed flowscited by 11
  12. 2024Variational Inference for Uncertainty Quantification: an Analysis of Trade-offscited by 11
  13. 2020tfp.mcmc: Modern Markov Chain Monte Carlo Tools Built for Modern Hardwarecited by 10
  14. 2020Joint Stochastic Approximation and Its Application to Learning Discrete Latent Variable Modelscited by 10
  15. 2021Empirical Evaluation of Biased Methods for Alpha Divergence Minimizationcited by 10
  16. 2021Nested hat R: Assessing the convergence of Markov chain Monte Carlo when running many short chainscited by 9
  17. 2019Stochastic Variational Inference via Upper Boundcited by 8
  18. 2023Bridging the Gap Between Variational Inference and Wasserstein Gradient Flowscited by 8
  19. 2020MetFlow: A New Efficient Method for Bridging the Gap between Markov Chain Monte Carlo and Variational Inferencecited by 7
  20. 2023On the 1-Wasserstein Distance between Location-Scale Distributions and the Effect of Differential Privacycited by 7
  21. 2019Automatically Batching Control-Intensive Programs for Modern Acceleratorscited by 6
  22. 2022Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradientscited by 6
  23. 2024Stable Training of Normalizing Flows for High-dimensional Variational Inferencecited by 6
  24. 2019A Rule for Gradient Estimator Selection, with an Application to Variational Inferencecited by 5
  25. 2024Fast and Unified Path Gradient Estimators for Normalizing Flowscited by 5
  26. 2021Automatic Guide Generation for Stan via NumPyrocited by 3
  27. 2023U-Statistics for Importance-Weighted Variational Inferencecited by 3
  28. 2020Stochastic Normalizing Flowscited by 2
  29. 2022PAVI: Plate-Amortized Variational Inferencecited by 2
  30. 2024Differentiable Annealed Importance Sampling Minimizes The Symmetrized Kullback-Leibler Divergence Between Initial and Target Distributioncited by 2
  31. 2019On importance-weighted autoencoderscited by 1
  32. 2024Understanding and mitigating difficulties in posterior predictive evaluationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 1.

5 papers.

  1. 2025Path Gradients after Flow Matchingcited by 3
  2. 2025Model-Informed Flows for Bayesian Inferencecited by 3
  3. 2024Empirical evaluation of normalizing flows in Markov Chain Monte Carlocited by 1
  4. 2025Rotated Mean-Field Variational Inference and Iterative Gaussianizationcited by 1
  5. 2026Amortized Factor Inference Networks for Posterior Inferencecited by 0