mlsanityData to 7 Oct 2026

Transport Score Climbing: Variational Inference Using Forward KL and Adaptive Neural Transport

arXiv 2202.01841 ↗First posted 3 Feb 2022stat.ML cs.LG stat.COcited by 8

Not on the map. Why?

Abstract (arXiv)

Variational inference often minimizes the "reverse" Kullbeck-Leibler (KL) KL(q||p) from the approximate distribution q to the posterior p. Recent work studies the "forward" KL KL(p||q), which unlike reverse KL does not lead to variational approximations that underestimate uncertainty. This paper introduces Transport Score Climbing (TSC), a method that optimizes KL(p||q) by using Hamiltonian Monte Carlo (HMC) and a novel adaptive transport map. The transport map improves the trajectory of HMC by acting as a change of variable between the latent variable space and a warped space. TSC uses HMC samples to dynamically train the transport map while optimizing KL(p||q). TSC leverages synergies, where better transport maps lead to better HMC sampling, which then leads to better transport maps. We demonstrate TSC on synthetic and real data. We find that TSC achieves competitive performance when training variational autoencoders on large-scale data.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 23 of its 58 references that are in the map. First posted on arXiv 3 Feb 2022.

Builds on

23 of 58 references.

References · 23

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2015Variational Inference with Normalizing Flowscited by 2,161
  4. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  5. 2016Density estimation using Real NVPcited by 2,029
  6. 2016Variational Inference: A Review for Statisticianscited by 1,466
  7. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  8. 2015Importance Weighted Autoencoderscited by 454
  9. 2017TensorFlow Distributionscited by 105
  10. 2019NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transportcited by 73
  11. 2014Reweighted Wake-Sleepcited by 66
  12. 2019Elements of Sequential Monte Carlocited by 60
11 more
  1. 2018Yes, but Did It Work?: Evaluating Variational Inferencecited by 52
  2. 2018Avoiding Latent Variable Collapse With Generative Skip Modelscited by 47
  3. 2020Markovian Score Climbing: Variational Inference with KL(p||q)cited by 46
  4. 2018Hamiltonian Variational Auto-Encodercited by 41
  5. 2016On the Quantitative Analysis of Decoder-Based Generative Modelscited by 40
  6. 2015Neural Adaptive Sequential Monte Carlocited by 34
  7. 2021Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergencecited by 29
  8. 2017Rapid Mixing of Hamiltonian Monte Carlo on Strongly Log-Concave Distributionscited by 12
  9. 2020Joint Stochastic Approximation and Its Application to Learning Discrete Latent Variable Modelscited by 10
  10. 2019Reweighted Expectation Maximizationcited by 9
  11. 2020Unbiased Gradient Estimation for Variational Auto-Encoders using Coupled Markov Chainscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 0; 2024: 3; 2025: 2; 2026: 1.

8 papers.

  1. 2022A Variational Perspective on Generative Flow Networkscited by 39
  2. 2024Markovian Flow Matching: Accelerating MCMC with Continuous Normalizing Flowscited by 9
  3. 2022Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradientscited by 6
  4. 2024Inclusive KL Gradient Flows: Otto-Wasserstein, Fisher-Rao-Gaussian, and Local-Estimator Dynamicscited by 4
  5. 2024VISA: Variational Inference with Sequential Sample-Average Approximationscited by 1
  6. 2025Globally Convergent Variational Inferencecited by 1
  7. 2025Neural Posterior Estimation with Autoregressive Tiling for Detecting Objects in Astronomical Imagescited by 1
  8. 2026Amortized Factor Inference Networks for Posterior Inferencecited by 0