# Convergence Properties of Natural Gradient Descent for Minimizing KL Divergence

arXiv: https://arxiv.org/abs/2504.19259 · HTML: https://mlsanity.com/paper/2504.19259/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 34 references that are in the map.
- First posted on arXiv 27 Apr 2025.

## Builds on (most cited first, 16 of 16)

- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Revisiting Natural Gradient for Deep Networks](https://mlsanity.com/paper/1301.3584/) (2013)
- [On the Convergence Rates of Policy Gradient Methods](https://mlsanity.com/paper/2201.07443/) (2022)
- [Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints](https://mlsanity.com/paper/1408.3595/) (2014)
- [Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach](https://mlsanity.com/paper/1806.01316/) (2018)
- [Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks](https://mlsanity.com/paper/1905.10961/) (2019)
- [Linear Convergence of Natural Policy Gradient Methods with Log-Linear Policies](https://mlsanity.com/paper/2210.01400/) (2022)
- [Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks](https://mlsanity.com/paper/2010.00879/) (2020)
- [Accelerating Natural Gradient with Higher-Order Invariance](https://mlsanity.com/paper/1803.01273/) (2018)
- [Beyond Convexity -- Contraction and Global Convergence of Gradient Descent](https://mlsanity.com/paper/1806.06655/) (2018)
- [Geometry and convergence of natural policy gradient methods](https://mlsanity.com/paper/2211.02105/) (2022)
- [Invariance Properties of the Natural Gradient in Overparametrised Systems](https://mlsanity.com/paper/2206.15273/) (2022)
- [A Study of Condition Numbers for First-Order Optimization](https://mlsanity.com/paper/2012.05782/) (2020)
- [Continuous-time Lower Bounds for Gradient-based Algorithms](https://mlsanity.com/paper/2002.03546/) (2020)
- [The Speed-Robustness Trade-Off for First-Order Methods with Additive Gradient Noise](https://mlsanity.com/paper/2109.05059/) (2021)
- [On the Natural Gradient of the Evidence Lower Bound](https://mlsanity.com/paper/2307.11249/) (2023)

## Abstract (arXiv)

The Kullback-Leibler (KL) divergence plays a central role in probabilistic machine learning, where it commonly serves as the canonical loss function. Optimization in such settings is often performed over the probability simplex, where the choice of parameterization significantly impacts convergence. In this work, we study the problem of minimizing the KL divergence and analyze the behavior of gradient-based optimization algorithms under two dual coordinate systems within the framework of information geometry- the exponential family (θ coordinates) and the mixture family (η coordinates). We compare Euclidean gradient descent (GD) in these coordinates with the coordinate-invariant natural gradient descent (NGD), where the natural gradient is a Riemannian gradient that incorporates the intrinsic geometry of the underlying statistical model. In continuous time, we prove that the convergence rates of GD in the θ and η coordinates provide lower and upper bounds, respectively, on the convergence rate of NGD. Moreover, under affine reparameterizations of the dual coordinates, the convergence rates of GD in η and θ coordinates can be scaled to 2c and frac2c, respectively, for any c>0, while NGD maintains a fixed convergence rate of 2, remaining invariant to such transformations and sandwiched between them. Although this suggests that NGD may not exhibit uniformly superior convergence in continuous time, we demonstrate that its advantages become pronounced in discrete time, where it achieves faster convergence and greater robustness to noise, outperforming GD. Our analysis hinges on bounding the spectrum and condition number of the Hessian of the KL divergence at the optimum, which coincides with the Fisher information matrix.