# Geometric variational inference

arXiv: https://arxiv.org/abs/2105.10470 · HTML: https://mlsanity.com/paper/2105.10470/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 40 references that are in the map.
- First posted on arXiv 21 May 2021.

## Builds on (most cited first, 9 of 9)

- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Automatic Differentiation Variational Inference](https://mlsanity.com/paper/1603.00788/) (2016)
- [Large-scale Log-determinant Computation through Stochastic Chebyshev Expansions](https://mlsanity.com/paper/1503.06394/) (2015)
- [Metric Gaussian Variational Inference](https://mlsanity.com/paper/1901.11033/) (2019)
- [Entropic Trace Estimates for Log Determinants](https://mlsanity.com/paper/1704.07223/) (2017)
- [Field dynamics inference for local and causal interactions](https://mlsanity.com/paper/1902.02624/) (2019)
- [A Geometric Variational Approach to Bayesian Inference](https://mlsanity.com/paper/1707.09714/) (2017)

## Built on by (most cited first, 12 of 12)

- [GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Framework](https://mlsanity.com/paper/2312.16429/) (2023)
- [Riemannian Laplace Approximation with the Fisher Metric](https://mlsanity.com/paper/2311.02766/) (2023)
- [Fast and robust Bayesian Inference using Gaussian Processes with GPry](https://mlsanity.com/paper/2211.02045/) (2022)
- [Probabilistic Autoencoder using Fisher Information](https://mlsanity.com/paper/2110.14947/) (2021)
- [Sparse Kernel Gaussian Processes through Iterative Charted Refinement (ICR)](https://mlsanity.com/paper/2206.10634/) (2022)
- [Information Field Theory and Artificial Intelligence](https://mlsanity.com/paper/2112.10133/) (2021)
- [Reconstructing the Universe with Variational self-Boosted Sampling](https://mlsanity.com/paper/2206.15433/) (2022)
- [Re-Envisioning Numerical Information Field Theory (NIFTy.re): A Library for Gaussian Processes and Variational Inference](https://mlsanity.com/paper/2402.16683/) (2024)
- [Attention to Entropic Communication](https://mlsanity.com/paper/2307.11423/) (2023)
- [Variational Inference Optimized Using the Curved Geometry of Coupled Free Energy](https://mlsanity.com/paper/2506.09091/) (2025)
- [GraphGP: Scalable Gaussian Processes with Vecchia's Approximation](https://mlsanity.com/paper/2606.11402/) (2026)
- [The Degeneracy Distillery](https://mlsanity.com/paper/2606.23838/) (2026)

## Abstract (arXiv)

Efficiently accessing the information contained in non-linear and high dimensional probability distributions remains a core challenge in modern statistics. Traditionally, estimators that go beyond point estimates are either categorized as Variational Inference (VI) or Markov-Chain Monte-Carlo (MCMC) techniques. While MCMC methods that utilize the geometric properties of continuous probability distributions to increase their efficiency have been proposed, VI methods rarely use the geometry. This work aims to fill this gap and proposes geometric Variational Inference (geoVI), a method based on Riemannian geometry and the Fisher information metric. It is used to construct a coordinate transformation that relates the Riemannian manifold associated with the metric to Euclidean space. The distribution, expressed in the coordinate system induced by the transformation, takes a particularly simple form that allows for an accurate variational approximation by a normal distribution. Furthermore, the algorithmic structure allows for an efficient implementation of geoVI which is demonstrated on multiple examples, ranging from low-dimensional illustrative ones to non-linear, hierarchical Bayesian inverse problems in thousands of dimensions.