# Information Newton's flow: second-order optimization method in probability space

arXiv: https://arxiv.org/abs/2001.04341 · HTML: https://mlsanity.com/paper/2001.04341/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jan 2020.

## Built on by (most cited first, 24 of 24)

- [Projected Wasserstein gradient descent for high-dimensional Bayesian inference](https://mlsanity.com/paper/2102.06350/) (2021)
- [Particle-based Variational Inference with Preconditioned Functional Gradient Flow](https://mlsanity.com/paper/2211.13954/) (2022)
- [Sampling via Gradient Flows in the Space of Probability Measures](https://mlsanity.com/paper/2310.03597/) (2023)
- [Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance](https://mlsanity.com/paper/2302.11024/) (2023)
- [Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent](https://mlsanity.com/paper/2305.14076/) (2023)
- [Fisher-Rao Gradient Flow: Geodesic Convexity and Functional Inequalities](https://mlsanity.com/paper/2407.15693/) (2024)
- [Mirror and Preconditioned Gradient Descent in Wasserstein Space](https://mlsanity.com/paper/2406.08938/) (2024)
- [Stein transport for Bayesian inference](https://mlsanity.com/paper/2409.01464/) (2024)
- [Scaling Up Bayesian Uncertainty Quantification for Inverse Problems using Deep Neural Networks](https://mlsanity.com/paper/2101.03906/) (2021)
- [Particle-based Variational Inference with Generalized Wasserstein Gradient Flow](https://mlsanity.com/paper/2310.16516/) (2023)
- [Distributionally Robust Optimization via Iterative Algorithms in Continuous Probability Spaces](https://mlsanity.com/paper/2412.20556/) (2024)
- [Worst-case generation via minimax optimization in Wasserstein space](https://mlsanity.com/paper/2512.08176/) (2025)
- [A stochastic Stein Variational Newton method](https://mlsanity.com/paper/2204.09039/) (2022)
- [Wasserstein Gradient Boosting: A Framework for Distribution-Valued Supervised Learning](https://mlsanity.com/paper/2405.09536/) (2024)
- [Information Geometry of Wasserstein Statistics on Shapes and Affine Deformations](https://mlsanity.com/paper/2307.12508/) (2023)
- [A Particle Algorithm for Mean-Field Variational Inference](https://mlsanity.com/paper/2412.20385/) (2024)
- [A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equations](https://mlsanity.com/paper/2411.06278/) (2024)
- [Towards understanding Accelerated Stein Variational Gradient Flow -- Analysis of Generalized Bilinear Kernels for Gaussian target distributions](https://mlsanity.com/paper/2509.04008/) (2025)
- [Hessian-guided Perturbed Wasserstein Gradient Flows for Escaping Saddle Points](https://mlsanity.com/paper/2509.16974/) (2025)
- [Wasserstein Saddle-Free Newton: Saddle Escape and Local Spectral Convergence](https://mlsanity.com/paper/2605.17963/) (2026)
- [Continuous-time Riemannian SGD and SVRG Flows on Wasserstein Probabilistic Space](https://mlsanity.com/paper/2401.13530/) (2024)
- [Subspace Langevin Monte Carlo](https://mlsanity.com/paper/2412.13928/) (2024)
- [Information Geometry of Message Passing](https://mlsanity.com/paper/2608.15922/) (2026)
- [Trust-Region Optimization for Smooth Potential-Interaction Energies in Wasserstein Space](https://mlsanity.com/paper/2610.08883/) (2026)

## Abstract (arXiv)

We introduce a framework for Newton's flows in probability space with information metrics, named information Newton's flows. Here two information metrics are considered, including both the Fisher-Rao metric and the Wasserstein-2 metric. A known fact is that overdamped Langevin dynamics correspond to Wasserstein gradient flows of Kullback-Leibler (KL) divergence. Extending this fact to Wasserstein Newton's flows, we derive Newton's Langevin dynamics. We provide examples of Newton's Langevin dynamics in both one-dimensional space and Gaussian families. For the numerical implementation, we design sampling efficient variational methods in affine models and reproducing kernel Hilbert space (RKHS) to approximate Wasserstein Newton's directions. We also establish convergence results of the proposed information Newton's method with approximated directions. Several numerical examples from Bayesian sampling problems are shown to demonstrate the effectiveness of the proposed method.