# Linear Response Methods for Accurate Covariance Estimates from Mean Field Variational Bayes

arXiv: https://arxiv.org/abs/1506.04088 · HTML: https://mlsanity.com/paper/1506.04088/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2015.

## Built on by (most cited first, 14 of 14)

- [Semi-Implicit Variational Inference via Score Matching](https://mlsanity.com/paper/2308.10014/) (2023)
- [Black Box Variational Inference with a Deterministic Objective: Faster, More Accurate, and Even More Black Box](https://mlsanity.com/paper/2304.05527/) (2023)
- [Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Acceleration](https://mlsanity.com/paper/2310.17153/) (2023)
- [Structured Stochastic Gradient MCMC](https://mlsanity.com/paper/2107.09028/) (2021)
- [Efficient Semi-Implicit Variational Inference](https://mlsanity.com/paper/2101.06070/) (2021)
- [Toward a Taxonomy of Trust for Probabilistic Machine Learning](https://mlsanity.com/paper/2112.03270/) (2021)
- [Kernel Semi-Implicit Variational Inference](https://mlsanity.com/paper/2405.18997/) (2024)
- [Sample Average Approximation for Black-Box VI](https://mlsanity.com/paper/2304.06803/) (2023)
- [Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correction](https://mlsanity.com/paper/2606.06303/) (2026)
- [VFDS: Variational Foresight Dynamic Selection in Bayesian Neural Networks for Efficient Human Activity Recognition](https://mlsanity.com/paper/2204.00130/) (2022)
- [VISA: Variational Inference with Sequential Sample-Average Approximations](https://mlsanity.com/paper/2403.09429/) (2024)
- [Quantification of Uncertainties in Probabilistic Deep Neural Network by Implementing Boosting of Variational Inference](https://mlsanity.com/paper/2503.13909/) (2025)
- [A Kernel Approach for Semi-implicit Variational Inference](https://mlsanity.com/paper/2601.12023/) (2026)
- [Deep Learning and Bayesian inference for Inverse Problems](https://mlsanity.com/paper/2308.15492/) (2023)

## Abstract (arXiv)

Mean field variational Bayes (MFVB) is a popular posterior approximation method due to its fast runtime on large-scale data sets. However, it is well known that a major failing of MFVB is that it underestimates the uncertainty of model variables (sometimes severely) and provides no information about model variable covariance. We generalize linear response methods from statistical physics to deliver accurate uncertainty estimates for model variables---both for individual variables and coherently across variables. We call our method linear response variational Bayes (LRVB). When the MFVB posterior approximation is in the exponential family, LRVB has a simple, analytic form, even for non-conjugate models. Indeed, we make no assumptions about the form of the true posterior. We demonstrate the accuracy and scalability of our method on a range of models for both simulated and real data.