# Black-box optimization using geodesics in statistical manifolds

arXiv: https://arxiv.org/abs/1309.7168 · HTML: https://mlsanity.com/paper/1309.7168/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2013.

## Built on by (most cited first, 4 of 4)

- [Fast Moving Natural Evolution Strategy for High-Dimensional Problems](https://mlsanity.com/paper/2201.11422/) (2022)
- [Towards a Principled Learning Rate Adaptation for Natural Evolution Strategies](https://mlsanity.com/paper/2112.10680/) (2021)
- [Properties of the Concrete distribution](https://mlsanity.com/paper/2211.01306/) (2022)
- [An Invariant Information Geometric Method for High-Dimensional Online Optimization](https://mlsanity.com/paper/2401.01579/) (2024)

## Abstract (arXiv)

Information geometric optimization (IGO) is a general framework for stochastic optimization problems aiming at limiting the influence of arbitrary parametrization choices. The initial problem is transformed into the optimization of a smooth function on a Riemannian manifold, defining a parametrization-invariant first order differential equation. However, in practice, it is necessary to discretize time, and then, parametrization invariance holds only at first order in the step size. We define the Geodesic IGO update (GIGO), which uses the Riemannian manifold structure to obtain an update entirely independent from the parametrization of the manifold. We test it with classical objective functions. Thanks to Noether's theorem from classical mechanics, we find an efficient way to write a first order differential equation satisfied by the geodesics of the statistical manifold of Gaussian distributions, and thus to compute the corresponding GIGO update. We then compare GIGO, pure rank-μ CMA-ES citeCMATuto and xNES citexNES (two previous algorithms that can be recovered by the IGO framework), and show that while the GIGO and xNES updates coincide when the mean is fixed, they are different in general, contrary to previous intuition. We then define a new algorithm (Blockwise GIGO) that recovers the xNES update from abstract principles.