# Direct From Darwin: Deriving Advanced Optimizers From Evolutionary First Principles

arXiv: https://arxiv.org/abs/2605.05284 · HTML: https://mlsanity.com/paper/2605.05284/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 60 references that are in the map.
- First posted on arXiv 6 May 2026.

## Builds on (most cited first, 14 of 14)

- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [Shampoo: Preconditioned Stochastic Tensor Optimization](https://mlsanity.com/paper/1802.09568/) (2018)
- [Challenges in Training PINNs: A Loss Landscape Perspective](https://mlsanity.com/paper/2402.01868/) (2024)
- [Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2501.16371/) (2025)
- [An operator preconditioning perspective on training in physics-informed machine learning](https://mlsanity.com/paper/2310.05801/) (2023)
- [The Newton-Muon Optimizer](https://mlsanity.com/paper/2604.01472/) (2026)
- [Muon2: Boosting Muon via Adaptive Second-Moment Preconditioning](https://mlsanity.com/paper/2604.09967/) (2026)
- [Correspondence between neuroevolution and gradient descent](https://mlsanity.com/paper/2008.06643/) (2020)
- [Evolutionary Optimization of Physics-Informed Neural Networks: Advancing Generalizability by the Baldwin Effect](https://mlsanity.com/paper/2312.03243/) (2023)
- [Information-geometric optimization with natural selection](https://mlsanity.com/paper/1912.03395/) (2019)
- [The Paradox of Success in Evolutionary and Bioinspired Optimization: Revisiting Critical Issues, Key Studies, and Methodological Pathways](https://mlsanity.com/paper/2501.07515/) (2025)
- [Evolutionary Algorithms in the Light of SGD: Limit Equivalence, Minima Flatness, and Transfer Learning](https://mlsanity.com/paper/2306.09991/) (2023)
- [The Price equation reveals a universal force-metric-bias law of algorithmic learning and natural selection](https://mlsanity.com/paper/2507.18549/) (2025)

## Abstract (arXiv)

Evolutionary computation has long promised to deliver both high-performance optimization tools as well as rigorous scientific simulations of Darwinian evolution. However, modern algorithms frequently abandon evolutionary fidelity for physics-inspired heuristics or superficial biological metaphors. This paper derives a suite of advanced gradient-based optimization algorithms directly from evolutionary first principles. We introduce Darwinian Lineage Simulations (DLS) to prove that, in an asexual context, Fisher's and Wright's historically opposed views of evolution are actually formally equivalent; One can partition Fisher's deterministically-evolving total population into Wright's randomly-drifting sub-populations. We prove that proper bookkeeping requires introducing a specific kind of structured noise (the DLS noise relation). Crucially, any bookkeeping choices which satisfy this relation will yield a faithful simulation of evolution. Using this vast representational freedom, we prove that a broad family of battle-tested optimization algorithms are already perfectly compatible with evolutionary dynamics. These include: Stochastic Gradient Descent as well as many regularizations/approximations of Newton's method and Natural Gradient Descent. By simply adding DLS noise (i.e., evolutionarily faithful genetic drift), these algorithms become scientifically valid in silico simulations of Darwinian evolution. Finally, we demonstrate that even the state-of-the-art Adam optimizer can be brought into evolutionary compliance through a minor mathematical surgery.