# Conjugate Natural Selection

arXiv: https://arxiv.org/abs/2208.13898 · HTML: https://mlsanity.com/paper/2208.13898/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 29 Aug 2022.

## Builds on (most cited first, 12 of 12)

- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning](https://mlsanity.com/paper/1704.00805/) (2017)
- [The Bayesian Learning Rule](https://mlsanity.com/paper/2107.04562/) (2021)
- [The Information Geometry of Mirror Descent](https://mlsanity.com/paper/1310.7780/) (2013)
- [Independent Natural Policy Gradient Always Converges in Markov Potential Games](https://mlsanity.com/paper/2110.10614/) (2021)
- [Neural Replicator Dynamics](https://mlsanity.com/paper/1906.00190/) (2019)
- [On the Dynamics of Inference and Learning](https://mlsanity.com/paper/2204.12939/) (2022)
- [Efficient Natural Gradient Descent Methods for Large-Scale PDE-Based Optimization Problems](https://mlsanity.com/paper/2202.06236/) (2022)
- [Hessian informed mirror descent](https://mlsanity.com/paper/2106.13477/) (2021)
- [Riemannian Natural Gradient Methods](https://mlsanity.com/paper/2207.07287/) (2022)
- [Information-geometric optimization with natural selection](https://mlsanity.com/paper/1912.03395/) (2019)
- [Momentum Accelerates Evolutionary Dynamics](https://mlsanity.com/paper/2007.02449/) (2020)

## Abstract (arXiv)

We prove that Fisher-Rao natural gradient descent (FR-NGD) optimally approximates the continuous time replicator equation (an essential model of evolutionary dynamics), and term this correspondence "conjugate natural selection". This correspondence promises alternative approaches for evolutionary computation over continuous or high-dimensional hypothesis spaces. As a special case, FR-NGD also provides the optimal approximation of continuous Bayesian inference when hypotheses compete on the basis of predicting actual observations. In this case, the method avoids the need to compute prior probabilities. We demonstrate our findings on a non-convex optimization problem and a system identification task for a stochastic process with time-varying parameters.