# Low-rank variational Bayes correction to the Laplace method

arXiv: https://arxiv.org/abs/2111.12945 · HTML: https://mlsanity.com/paper/2111.12945/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 54 references that are in the map.
- First posted on arXiv 25 Nov 2021.

## Builds on (most cited first, 16 of 16)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [A Non-Asymptotic Analysis for Stein Variational Gradient Descent](https://mlsanity.com/paper/2006.09797/) (2020)
- [Hamiltonian Monte Carlo for Hierarchical Models](https://mlsanity.com/paper/1312.0906/) (2013)
- [Stochastic Particle Gradient Descent for Infinite Ensembles](https://mlsanity.com/paper/1712.05438/) (2017)
- [Message Passing Stein Variational Gradient Descent](https://mlsanity.com/paper/1711.04425/) (2017)
- [GIBBON: General-purpose Information-Based Bayesian OptimisatioN](https://mlsanity.com/paper/2102.03324/) (2021)
- [Scaling limit of the Stein variational gradient descent: the mean field regime](https://mlsanity.com/paper/1805.04035/) (2018)
- [Deep Learning: A Bayesian Perspective](https://mlsanity.com/paper/1706.00473/) (2017)
- [A Bayes-Optimal View on Adversarial Examples](https://mlsanity.com/paper/2002.08859/) (2020)
- [Exactly Sparse Gaussian Variational Inference with Application to Derivative-Free Batch Nonlinear State Estimation](https://mlsanity.com/paper/1911.08333/) (2019)
- [A Bayesian Perspective of Statistical Machine Learning for Big Data](https://mlsanity.com/paper/1811.04788/) (2018)
- [Bounding errors of Expectation-Propagation](https://mlsanity.com/paper/1601.02387/) (2016)
- [Spatial modelling with R-INLA: A review](https://mlsanity.com/paper/1802.06350/) (2018)
- [New frontiers in Bayesian modeling using the INLA package in R](https://mlsanity.com/paper/1907.10426/) (2019)

## Built on by (most cited first, 2 of 2)

- [Variational Low-Rank Adaptation Using IVON](https://mlsanity.com/paper/2411.04421/) (2024)
- [Improving LoRA with Variational Learning](https://mlsanity.com/paper/2506.14280/) (2025)

## Abstract (arXiv)

Approximate inference methods like the Laplace method, Laplace approximations and variational methods, amongst others, are popular methods when exact inference is not feasible due to the complexity of the model or the abundance of data. In this paper we propose a hybrid approximate method called Low-Rank Variational Bayes correction (VBC), that uses the Laplace method and subsequently a Variational Bayes correction in a lower dimension, to the joint posterior mean. The cost is essentially that of the Laplace method which ensures scalability of the method, in both model complexity and data size. Models with fixed and unknown hyperparameters are considered, for simulated and real examples, for small and large datasets.