# A deterministic and computable Bernstein-von Mises theorem

arXiv: https://arxiv.org/abs/1904.02505 · HTML: https://mlsanity.com/paper/1904.02505/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Apr 2019.

## Built on by (most cited first, 2 of 2)

- [A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness](https://mlsanity.com/paper/2205.00403/) (2022)
- [Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations](https://mlsanity.com/paper/2202.10638/) (2022)

## Abstract (arXiv)

Bernstein-von Mises results (BvM) establish that the Laplace approximation is asymptotically correct in the large-data limit. However, these results are inappropriate for computational purposes since they only hold over most, and not all, datasets and involve hard-to-estimate constants. In this article, I present a new BvM theorem which bounds the Kullback-Leibler (KL) divergence between a fixed log-concave density fleft(boldsymbolθright) and its Laplace approximation. The bound goes to 0 as the higher-derivatives of fleft(boldsymbolθright) tend to 0 and fleft(boldsymbolθright) becomes increasingly Gaussian. The classical BvM theorem in the IID large-data asymptote is recovered as a corollary. Critically, this theorem further suggests a number of computable approximations of the KL divergence with the most promising being: [ KLleft(gLAP,fright)≈frac12Var_boldsymbolθ~ gleft(boldsymbolθright)left(logleft[fleft(boldsymbolθright)right]-logleft[gLAPleft(boldsymbolθright)right]right) ] An empirical investigation of these bounds in the logistic classification model reveals that these approximations are great surrogates for the KL divergence. This result, and future results of a similar nature, could provide a path towards rigorously controlling the error due to the Laplace approximation and more modern approximation methods.