# Variational Inference in Location-Scale Families: Exact Recovery of the Mean and Correlation Matrix

arXiv: https://arxiv.org/abs/2410.11067 · HTML: https://mlsanity.com/paper/2410.11067/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 60 references that are in the map.
- First posted on arXiv 14 Oct 2024.

## Builds on (most cited first, 26 of 26)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Automatic Differentiation Variational Inference](https://mlsanity.com/paper/1603.00788/) (2016)
- [R'enyi Divergence Variational Inference](https://mlsanity.com/paper/1602.02311/) (2016)
- [Variational inference via Wasserstein gradient flows](https://mlsanity.com/paper/2205.15902/) (2022)
- [Concentration of tempered posteriors and of their variational approximations](https://mlsanity.com/paper/1706.09293/) (2017)
- [Frequentist Consistency of Variational Bayes](https://mlsanity.com/paper/1705.03439/) (2017)
- [Yes, but Did It Work?: Evaluating Variational Inference](https://mlsanity.com/paper/1802.02538/) (2018)
- [Convergence Rates of Variational Posterior Distributions](https://mlsanity.com/paper/1712.02519/) (2017)
- [Sparsity information and regularization in the horseshoe and other shrinkage priors](https://mlsanity.com/paper/1707.01694/) (2017)
- [Pareto Smoothed Importance Sampling](https://mlsanity.com/paper/1507.02646/) (2015)
- [Covariances, Robustness, and Variational Bayes](https://mlsanity.com/paper/1709.02536/) (2017)
- [Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimization](https://mlsanity.com/paper/2006.10343/) (2020)
- [Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space](https://mlsanity.com/paper/2304.05398/) (2023)
- [Challenges and Opportunities in High-dimensional Variational Inference](https://mlsanity.com/paper/2103.01085/) (2021)
- [Provable convergence guarantees for black-box variational inference](https://mlsanity.com/paper/2306.03638/) (2023)
- [Validated Variational Inference via Practical Posterior Error Bounds](https://mlsanity.com/paper/1910.04102/) (2019)
- [Pathfinder: Parallel quasi-Newton variational inference](https://mlsanity.com/paper/2108.03782/) (2021)
- [Batch and match: black-box variational inference with a score-based divergence](https://mlsanity.com/paper/2402.14758/) (2024)
- [Robust, Accurate Stochastic Optimization for Variational Inference](https://mlsanity.com/paper/2009.00666/) (2020)
- [Algorithms for mean-field variational inference via polyhedral optimization in the Wasserstein space](https://mlsanity.com/paper/2312.02849/) (2023)
- [Variational Inference for Uncertainty Quantification: an Analysis of Trade-offs](https://mlsanity.com/paper/2403.13748/) (2024)
- [The Kernel Interaction Trick: Fast Bayesian Discovery of Pairwise Interactions in High Dimensions](https://mlsanity.com/paper/1905.06501/) (2019)
- [The Shrinkage-Delinkage Trade-off: An Analysis of Factorized Gaussian Approximations for Variational Inference](https://mlsanity.com/paper/2302.09163/) (2023)
- [Variational inference based on a subclass of closed skew normals](https://mlsanity.com/paper/2306.02813/) (2023)

## Built on by (most cited first, 9 of 9)

- [Generalized Guarantees for Variational Inference in the Presence of Even and Elliptical Symmetry](https://mlsanity.com/paper/2511.01064/) (2025)
- [The Within-Orbit Adaptive Leapfrog No-U-Turn Sampler](https://mlsanity.com/paper/2506.18746/) (2025)
- [Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inference](https://mlsanity.com/paper/2505.21721/) (2025)
- [Asymptotically exact variational flows via involutive MCMC kernels](https://mlsanity.com/paper/2506.02162/) (2025)
- [Convergence of projected stochastic natural gradient variational inference for various step size and sample or batch size schedules](https://mlsanity.com/paper/2604.00683/) (2026)
- [Symmetry Guarantees Statistic Recovery in Variational Inference](https://mlsanity.com/paper/2604.18310/) (2026)
- [Even More Guarantees for Variational Inference in the Presence of Symmetries](https://mlsanity.com/paper/2604.21407/) (2026)
- [Position: The Time for Sampling Is Now! Charting a New Course for Bayesian Deep Learning](https://mlsanity.com/paper/2605.21765/) (2026)
- [Gaussian Mean Field Variational Inference can Overestimate Predictive Variance](https://mlsanity.com/paper/2606.25745/) (2026)

## Abstract (arXiv)

Given an intractable target density p, variational inference (VI) attempts to find the best approximation q from a tractable family Q. This is typically done by minimizing the exclusive Kullback-Leibler divergence, KL(q||p). In practice, Q is not rich enough to contain p, and the approximation is misspecified even when it is a unique global minimizer of KL(q||p). In this paper, we analyze the robustness of VI to these misspecifications when p exhibits certain symmetries and Q is a location-scale family that shares these symmetries. We prove strong guarantees for VI not only under mild regularity conditions but also in the face of severe misspecifications. Namely, we show that (i) VI recovers the mean of p when p exhibits an textiteven symmetry, and (ii) it recovers the correlation matrix of p when in addition~p exhibits an textitelliptical symmetry. These guarantees hold for the mean even when q is factorized and p is not, and for the correlation matrix even when~q and~p behave differently in their tails. We analyze various regimes of Bayesian inference where these symmetries are useful idealizations, and we also investigate experimentally how VI behaves in their absence.