mlsanityData to 7 Oct 2026

Stein transport for Bayesian inference

arXiv 2409.01464 ↗First posted 2 Sep 2024stat.ML cs.LG cs.NAcited by 11

Not on the map. Why?

Abstract (arXiv)

We introduce textitStein transport, a novel methodology for Bayesian inference designed to efficiently push an ensemble of particles along a predefined curve of tempered probability distributions. The driving vector field is chosen from a reproducing kernel Hilbert space and can be derived either through a suitable kernel ridge regression formulation or as an infinitesimal optimal transport map in the Stein geometry. The update equations of Stein transport resemble those of Stein variational gradient descent (SVGD), but introduce a time-varying score function as well as specific weights attached to the particles. While SVGD relies on convergence in the long-time limit, Stein transport reaches its posterior approximation at finite time t=1. Studying the mean-field limit, we discuss the errors incurred by regularisation and finite-particle effects, and we connect Stein transport to birth-death dynamics and Fisher-Rao gradient flows. In a series of experiments, we show that in comparison to SVGD, Stein transport not only often reaches more accurate posterior approximations with a significantly reduced computational budget, but that it also effectively mitigates the variance collapse phenomenon commonly observed in SVGD.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 54 of its 112 references that are in the map. First posted on arXiv 2 Sep 2024.

Builds on

54 of 112 references.

References · 54

  1. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  2. 2022Flow Matching for Generative Modelingcited by 5,426
  3. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  4. 2016Variational Inference: A Review for Statisticianscited by 1,466
  5. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  6. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  7. 2016Kernel Mean Embedding of Distributions: A Review and Beyondcited by 360
  8. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  9. 2018Gaussian Processes and Kernel Methods: A Review on Connections and Equivalencescited by 188
  10. 2017Stein Variational Gradient Descent as Gradient Flowcited by 149
  11. 2016A Kernel Test of Goodness of Fitcited by 143
  12. 2015Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measurescited by 138
38 more
  1. 2021Solving and Learning Nonlinear PDEs with Gaussian Processescited by 128
  2. 2019Maximum Mean Discrepancy Gradient Flowcited by 123
  3. 2017Measuring Sample Quality with Kernelscited by 119
  4. 2020Interacting particle solutions of Fokker-Planck equations through gradient-log-density estimationcited by 87
  5. 2017FALKON: An Optimal Large Scale Kernel Methodcited by 82
  6. 2019On the geometry of Stein variational gradient descentcited by 82
  7. 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
  8. 2020SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergencecited by 64
  9. 2020A Non-Asymptotic Analysis for Stein Variational Gradient Descentcited by 61
  10. 2014Control functionals for Monte Carlo integrationcited by 59
  11. 2021Kernel Stein Discrepancy Descentcited by 57
  12. 2013The Information Geometry of Mirror Descentcited by 53
  13. 2018A Stein variational Newton methodcited by 49
  14. 2017Message Passing Stein Variational Gradient Descentcited by 47
  15. 2019Accelerating Langevin Sampling with Birth-deathcited by 45
  16. 2024Optimal experimental design: Formulations and computationscited by 45
  17. 2024Statistical optimal transportcited by 44
  18. 2018Scaling limit of the Stein variational gradient descent: the mean field regimecited by 35
  19. 2021Stein's Method Meets Computational Statistics: A Review of Some Recent Developmentscited by 32
  20. 2022Birth-death dynamics for sampling: Global convergence, approximations and their asymptoticscited by 30
  21. 2024Sampling in Unit Time with Kernel Fisher-Rao Flowcited by 27
  22. 2023Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flowcited by 25
  23. 2024Liouville Flow Importance Samplercited by 25
  24. 2020Sliced Kernelized Stein Discrepancycited by 23
  25. 2020Information Newton's flow: second-order optimization method in probability spacecited by 22
  26. 2023Sampling via Gradient Flows in the Space of Probability Measurescited by 20
  27. 2023A Fisher-Rao gradient flow for entropy-regularised Markov decision processes in Polish spacescited by 19
  28. 2018Stochastic Gradient MCMC with Repulsive Forcescited by 18
  29. 2022Regularized Stein Variational Gradient Flowcited by 18
  30. 2023Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariancecited by 16
  31. 2023An Explicit Expansion of the Kullback-Leibler Divergence along its Fisher-Rao Gradient Flowcited by 16
  32. 2023A connection between Tempering and Entropic Mirror Descentcited by 16
  33. 2021Stein Variational Gradient Descent: many-particle and long-time asymptoticscited by 15
  34. 2022Grassmann Stein Variational Gradient Descentcited by 12
  35. 2024Measure transport with kernel mean embeddingscited by 11
  36. 2014Poisson's equation in nonlinear filteringcited by 10
  37. 2020Measure Transport with Kernel Stein Discrepancycited by 8
  38. 2021Ensemble Inference Methods for Models With Noisy and Expensive Likelihoodscited by 8

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 3.

11 papers.

  1. 2025Telegrapher's Generative Model via Kac Flowscited by 7
  2. 2025Sequential Monte Carlo approximations of Wasserstein--Fisher--Rao gradient flowscited by 5
  3. 2024Inclusive KL Gradient Flows: Otto-Wasserstein, Fisher-Rao-Gaussian, and Local-Estimator Dynamicscited by 4
  4. 2025A note on the unique properties of the Kullback--Leibler divergence for sampling via gradient flowscited by 4
  5. 2025Towards understanding Accelerated Stein Variational Gradient Flow -- Analysis of Generalized Bilinear Kernels for Gaussian target distributionscited by 2
  6. 2025An operator splitting analysis of Wasserstein--Fisher--Rao gradient flowscited by 2
  7. 2025Machine-Learned Sampling of Conditioned Path Measurescited by 0
  8. 2025Adaptive Kernel Selection for Stein Variational Gradient Descentcited by 0
  9. 2026Preservation of Log-Concavity and Convergence of Wasserstein-Fisher-Rao Gradient Flowscited by 0
  10. 2026Interacting particle guidance for sampling reward-tilted generative priorscited by 0
  11. 2026Density Ratio Estimation with Stein Displacement Fieldscited by 0