# Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flows

arXiv: https://arxiv.org/abs/2406.17263 · HTML: https://mlsanity.com/paper/2406.17263/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 99 references that are in the map.
- First posted on arXiv 25 Jun 2024.

## Builds on (most cited first, 27 of 27)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Variational inference via Wasserstein gradient flows](https://mlsanity.com/paper/2205.15902/) (2022)
- [Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks](https://mlsanity.com/paper/1905.10961/) (2019)
- [Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models](https://mlsanity.com/paper/1703.04265/) (2017)
- [Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations](https://mlsanity.com/paper/1709.00037/) (2017)
- [Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations](https://mlsanity.com/paper/1906.02914/) (2019)
- [Accelerating Langevin Sampling with Birth-death](https://mlsanity.com/paper/1905.09863/) (2019)
- [Birth-death dynamics for sampling: Global convergence, approximations and their asymptotics](https://mlsanity.com/paper/2211.00450/) (2022)
- [Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flow](https://mlsanity.com/paper/2301.01766/) (2023)
- [Sampling in Unit Time with Kernel Fisher-Rao Flow](https://mlsanity.com/paper/2401.03892/) (2024)
- [Ensemble Kalman Methods: A Mean Field Perspective](https://mlsanity.com/paper/2209.11371/) (2022)
- [Sampling via Gradient Flows in the Space of Probability Measures](https://mlsanity.com/paper/2310.03597/) (2023)
- [Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance](https://mlsanity.com/paper/2302.11024/) (2023)
- [An Explicit Expansion of the Kullback-Leibler Divergence along its Fisher-Rao Gradient Flow](https://mlsanity.com/paper/2302.12229/) (2023)
- [A connection between Tempering and Entropic Mirror Descent](https://mlsanity.com/paper/2310.11914/) (2023)
- [Optimal transport natural gradient for statistical manifolds with continuous sample space](https://mlsanity.com/paper/1805.08380/) (2018)
- [Tikhonov Regularization Within Ensemble Kalman Inversion](https://mlsanity.com/paper/1901.10382/) (2019)
- [Measure transport with kernel mean embeddings](https://mlsanity.com/paper/2401.12967/) (2024)
- [Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians](https://mlsanity.com/paper/2406.04012/) (2024)
- [Gaussian variational approximation for high-dimensional state space models](https://mlsanity.com/paper/1801.07873/) (2018)
- [Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equations](https://mlsanity.com/paper/2404.01145/) (2024)
- [Transporting Higher-Order Quadrature Rules: Quasi-Monte Carlo Points and Sparse Grids for Mixture Distributions](https://mlsanity.com/paper/2308.10081/) (2023)
- [Accelerate Langevin Sampling with Birth-Death Process and Exploration Component](https://mlsanity.com/paper/2305.05529/) (2023)
- [Ensemble-Based Annealed Importance Sampling](https://mlsanity.com/paper/2401.15645/) (2024)
- [Modeling, simulation and validation of supersonic parachute inflation dynamics during Mars landing](https://mlsanity.com/paper/1912.01658/) (2019)

## Built on by (most cited first, 9 of 9)

- [Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approach](https://mlsanity.com/paper/2506.03979/) (2025)
- [Fisher-Rao Gradient Flow: Geodesic Convexity and Functional Inequalities](https://mlsanity.com/paper/2407.15693/) (2024)
- [Sequential Monte Carlo approximations of Wasserstein--Fisher--Rao gradient flows](https://mlsanity.com/paper/2506.05905/) (2025)
- [Connections between sequential Bayesian inference and evolutionary dynamics](https://mlsanity.com/paper/2411.16366/) (2024)
- [New affine invariant ensemble samplers and their dimensional scaling](https://mlsanity.com/paper/2505.02987/) (2025)
- [Stable Derivative Free Gaussian Mixture Variational Inference for Bayesian Inverse Problems](https://mlsanity.com/paper/2501.04259/) (2025)
- [An operator splitting analysis of Wasserstein--Fisher--Rao gradient flows](https://mlsanity.com/paper/2511.18060/) (2025)
- [Spectral-factorized Positive-definite Curvature Learning for NN Training](https://mlsanity.com/paper/2502.06268/) (2025)
- [Properties and limitations of geometric tempering for gradient flow dynamics](https://mlsanity.com/paper/2604.20301/) (2026)

## Abstract (arXiv)

In this paper, we study efficient approximate sampling for probability distributions known up to normalization constants. We specifically focus on a problem class arising in Bayesian inference for large-scale inverse problems in science and engineering applications. The computational challenges we address with the proposed methodology are: (i) the need for repeated evaluations of expensive forward models; (ii) the potential existence of multiple modes; and (iii) the fact that gradient of, or adjoint solver for, the forward model might not be feasible. While existing Bayesian inference methods meet some of these challenges individually, we propose a framework that tackles all three systematically. Our approach builds upon the Fisher-Rao gradient flow in probability space, yielding a dynamical system for probability densities that converges towards the target distribution at a uniform exponential rate. This rapid convergence is advantageous for the computational burden outlined in (i). We apply Gaussian mixture approximations with operator splitting techniques to simulate the flow numerically; the resulting approximation can capture multiple modes thus addressing (ii). Furthermore, we employ the Kalman methodology to facilitate a derivative-free update of these Gaussian components and their respective weights, addressing the issue in (iii). The proposed methodology results in an efficient derivative-free sampler flexible enough to handle multi-modal distributions: Gaussian Mixture Kalman Inversion (GMKI). The effectiveness of GMKI is demonstrated both theoretically and numerically in several experiments with multimodal target distributions, including proof-of-concept and two-dimensional examples, as well as a large-scale application: recovering the Navier-Stokes initial condition from solution data at positive times.