mlsanityData to 14 Oct 2026

Fisher meets Feynman: score-based variational inference with a product of experts

arXiv 2510.21598 ↗First posted 24 Oct 2025stat.ML cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

We introduce a highly expressive yet distinctly tractable family for black-box variational inference (BBVI). Each member of this family is a weighted product of experts (PoE), and each weighted expert in the product is proportional to a multivariate t-distribution. These products of experts can model distributions with skew, heavy tails, and multiple modes, but to use them for BBVI, we must be able to sample from their densities. We show how to do this by reformulating these products of experts as latent variable models with auxiliary Dirichlet random variables. These Dirichlet variables emerge from a Feynman identity, originally developed for loop integrals in quantum field theory, that expresses the product of multiple fractions (or in our case, t-distributions) as an integral over the simplex. We leverage this simplicial latent space to draw weighted samples from these products of experts -- samples which BBVI then uses to find the PoE that best approximates a target density. Given a collection of experts, we derive an iterative procedure to optimize the exponents that determine their geometric weighting in the PoE. At each iteration, this procedure minimizes a regularized Fisher divergence to match the scores of the variational and target densities at a batch of samples drawn from the current approximation. This minimization reduces to a convex quadratic program, and we prove under general conditions that these updates converge exponentially fast to a near-optimal weighting of experts. We conclude by evaluating this approach on a variety of synthetic and real-world target distributions.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 35 of its 60 references that are in the map. First posted on arXiv 24 Oct 2025.

Builds on

35 of 60 references.

References · 35

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  4. 2013Auto-Encoding Variational Bayescited by 4,053
  5. 2015Variational Inference with Normalizing Flowscited by 2,170
  6. 2016Density estimation using Real NVPcited by 2,042
  7. 2016Variational Inference: A Review for Statisticianscited by 1,472
  8. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
  9. 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 417
  10. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  11. 2013Black Box Variational Inferencecited by 319
  12. 2018Multimodal Generative Models for Scalable Weakly-Supervised Learningcited by 200
23 more
  1. 2017OSQP: An Operator Splitting Solver for Quadratic Programscited by 161
  2. 2016Automatic Differentiation Variational Inferencecited by 159
  3. 2013Stochastic Gradient Descent, Weighted Sampling, and the Randomized Kaczmarz algorithmcited by 133
  4. 2018Semi-Implicit Variational Inferencecited by 55
  5. 2016Variational Boosting: Iteratively Refining Posterior Approximationscited by 37
  6. 2018Unbiased Implicit Variational Inferencecited by 33
  7. 2023Provable convergence guarantees for black-box variational inferencecited by 29
  8. 2024BlackJAX: Composable Bayesian inference in JAXcited by 26
  9. 2016Boosting Variational Inferencecited by 22
  10. 2023On the Convergence of Black-Box Variational Inferencecited by 22
  11. 2024Batch and match: black-box variational inference with a score-based divergencecited by 22
  12. 2020Automatic Differentiation Variational Inference with Mixturescited by 19
  13. 2023Variational Inference with Gaussian Score Matchingcited by 17
  14. 2018Boosting Black Box Variational Inferencecited by 14
  15. 2019Universal Boosting Variational Inferencecited by 13
  16. 2022MixFlows: principled variational inference via mixed flowscited by 11
  17. 2022Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flowscited by 11
  18. 2024EigenVI: score-based variational inference with orthogonal function expansionscited by 11
  19. 2021The computational asymptotics of Gaussian variational inference and the Laplace approximationcited by 8
  20. 2024Batch, match, and patch: low-rank approximations for score-based variational inferencecited by 8
  21. 2022Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selectioncited by 7
  22. 2023Implicit Variational Inference for High-Dimensional Posteriorscited by 5
  23. 2024Nonparametric Automatic Differentiation Variational Inference with Spline Approximationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Mode Collapse Is Cheap to Detect: A Ground-Truth-Free Pre-Flight Check for Neural Samplerscited by 0
  2. 2026QuanVI: Score-based Variational Inference via Quantum Maximally Mixed Statescited by 0