# Optimal transport natural gradient for statistical manifolds with continuous sample space

arXiv: https://arxiv.org/abs/1805.08380 · HTML: https://mlsanity.com/paper/1805.08380/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2018.

## Built on by (most cited first, 14 of 14)

- [Sampling via Gradient Flows in the Space of Probability Measures](https://mlsanity.com/paper/2310.03597/) (2023)
- [Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance](https://mlsanity.com/paper/2302.11024/) (2023)
- [Dynamic Flows on Curved Space Generated by Labeled Data](https://mlsanity.com/paper/2302.00061/) (2023)
- [Neural Wasserstein Gradient Flows for Maximum Mean Discrepancies with Riesz Kernels](https://mlsanity.com/paper/2301.11624/) (2023)
- [Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flows](https://mlsanity.com/paper/2406.17263/) (2024)
- [Wasserstein Gradient Flows for Optimizing Gaussian Mixture Policies](https://mlsanity.com/paper/2305.10411/) (2023)
- [Understanding Stochastic Natural Gradient Variational Inference](https://mlsanity.com/paper/2406.01870/) (2024)
- [Efficient Natural Gradient Descent Methods for Large-Scale PDE-Based Optimization Problems](https://mlsanity.com/paper/2202.06236/) (2022)
- [Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equations](https://mlsanity.com/paper/2404.01145/) (2024)
- [A theory of learning with constrained weight-distribution](https://mlsanity.com/paper/2206.08933/) (2022)
- [Information Geometry of Wasserstein Statistics on Shapes and Affine Deformations](https://mlsanity.com/paper/2307.12508/) (2023)
- [Manifold learning in metric spaces](https://mlsanity.com/paper/2503.16187/) (2025)
- [Embedding Functional Data: Multidimensional Scaling and Manifold Learning](https://mlsanity.com/paper/2208.14540/) (2022)
- [Optimal Transport for ε-Contaminated Credal Sets: To the Memory of Sayan Mukherjee](https://mlsanity.com/paper/2410.03267/) (2024)

## Abstract (arXiv)

We study the Wasserstein natural gradient in parametric statistical models with continuous sample spaces. Our approach is to pull back the L2-Wasserstein metric tensor in the probability density space to a parameter space, equipping the latter with a positive definite metric tensor, under which it becomes a Riemannian manifold, named the Wasserstein statistical manifold. In general, it is not a totally geodesic sub-manifold of the density space, and therefore its geodesics will differ from the Wasserstein geodesics, except for the well-known Gaussian distribution case, a fact which can also be validated under our framework. We use the sub-manifold geometry to derive a gradient flow and natural gradient descent method in the parameter space. When parametrized densities lie in bR, the induced metric tensor establishes an explicit formula. In optimization problems, we observe that the natural gradient descent outperforms the standard gradient descent when the Wasserstein distance is the objective function. In such a case, we prove that the resulting algorithm behaves similarly to the Newton method in the asymptotic regime. The proof calculates the exact Hessian formula for the Wasserstein distance, which further motivates another preconditioner for the optimization process. To the end, we present examples to illustrate the effectiveness of the natural gradient in several parametric statistical models, including the Gaussian measure, Gaussian mixture, Gamma distribution, and Laplace distribution.