mlsanityData to 14 Oct 2026

Even More Guarantees for Variational Inference in the Presence of Symmetries

arXiv 2604.21407 ↗First posted 23 Apr 2026cs.LG stat.CO stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

When approximating an intractable density via variational inference (VI) the variational family is typically chosen as a simple parametric family that very likely does not contain the target. This raises the question: Under which conditions can we recover characteristics of the target despite misspecification? In this work, we extend previous theoretical results on robust VI with location-scale families under target symmetries in two substantial ways: (1) We open them up to a wider range of divergences by providing sufficient conditions for exact recovery of the target mean and correlation matrix when using the forward Kullback-Leibler divergence and α-divergences. (2) By doing so, we find that we can drop the restrictive assumption of a log-concave target made in previous work, allowing us to give guarantees for a wider range of targets, including multi-modal ones. In our experiments, we show how our guarantees can serve as guidelines for the choice of the variational family and α-value and we illustrate on a diverse set of examples how and why optimization can fail in the absence of our sufficient conditions.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 34 of its 72 references that are in the map. First posted on arXiv 23 Apr 2026.

Builds on

34 of 72 references.

References · 34

  1. 2021Convex optimizationcited by 2,515
  2. 2016Variational Inference: A Review for Statisticianscited by 1,472
  3. 2018Pyro: Deep Universal Probabilistic Programmingcited by 380
  4. 2013Black Box Variational Inferencecited by 319
  5. 2016Automatic Differentiation Variational Inferencecited by 159
  6. 2022Algorithms to estimate Shapley value feature attributionscited by 115
  7. 2016R'enyi Divergence Variational Inferencecited by 104
  8. 2017Frequentist Consistency of Variational Bayescited by 60
  9. 2018Yes, but Did It Work?: Evaluating Variational Inferencecited by 53
  10. 2017Convergence Rates of Variational Posterior Distributionscited by 45
  11. 2017Covariances, Robustness, and Variational Bayescited by 40
  12. 2023Subtractive Mixture Models via Squaring: Representation and Learningcited by 39
22 more
  1. 2021Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergencecited by 29
  2. 2023Provable convergence guarantees for black-box variational inferencecited by 29
  3. 2019Validated Variational Inference via Practical Posterior Error Boundscited by 26
  4. 2021Assessing Fairness in the Presence of Missing Datacited by 24
  5. 2019Variational Bayes under Model Misspecificationcited by 22
  6. 2022A Unified Perspective on Natural Gradient Variational Inference with Gaussian Mixture Modelscited by 22
  7. 2017On Statistical Optimality of Variational Bayescited by 21
  8. 2020On the Difficulty of Unbiased Alpha Divergence Minimizationcited by 21
  9. 2017Safe Adaptive Importance Samplingcited by 19
  10. 2020Automatic Differentiation Variational Inference with Mixturescited by 19
  11. 2023Variational Inference with Gaussian Score Matchingcited by 17
  12. 2019Universal Boosting Variational Inferencecited by 13
  13. 2021Adaptive Importance Sampling meets Mirror Descent: a Bias-variance tradeoffcited by 13
  14. 2019What to Expect of Classifiers? Reasoning about Logistic Regression with Missing Featurescited by 12
  15. 2024Variational Inference for Uncertainty Quantification: an Analysis of Trade-offscited by 11
  16. 2024Variational Inference in Location-Scale Families: Exact Recovery of the Mean and Correlation Matrixcited by 9
  17. 2021Mixture weights optimisation for Alpha-Divergence Variational Inferencecited by 6
  18. 2024E2M: Double Bounded α-Divergence Optimization for Tensor-based Discrete Density Estimationcited by 4
  19. 2023Adaptive importance sampling for heavy-tailed distributions via α-divergence minimizationcited by 3
  20. 2024Efficient Mixture Learning in Black-Box Variational Inferencecited by 3
  21. 2025Generalized Guarantees for Variational Inference in the Presence of Even and Elliptical Symmetrycited by 3
  22. 2026How to Approximate Inference with Subtractive Mixture Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Gaussian Mean Field Variational Inference can Overestimate Predictive Variancecited by 0