# Evolution Strategies at the Hyperscale

arXiv: https://arxiv.org/abs/2511.16652 · HTML: https://mlsanity.com/paper/2511.16652/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 79 references that are in the map.
- First posted on arXiv 20 Nov 2025.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [Population Based Training of Neural Networks](https://mlsanity.com/paper/1711.09846/) (2017)
- [Fine-Tuning Language Models with Just Forward Passes](https://mlsanity.com/paper/2305.17333/) (2023)
- [The Illusion of State in State-Space Models](https://mlsanity.com/paper/2404.08819/) (2024)
- [Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/1712.06567/) (2017)
- [RWKV-7 "Goose" with Expressive Dynamic State Evolution](https://mlsanity.com/paper/2503.14456/) (2025)
- [Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark](https://mlsanity.com/paper/2402.11592/) (2024)
- [Discovered Policy Optimisation](https://mlsanity.com/paper/2210.05639/) (2022)
- [Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning](https://mlsanity.com/paper/2402.16801/) (2024)
- [The MiniPile Challenge for Data-Efficient Language Models](https://mlsanity.com/paper/2304.08442/) (2023)
- [Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits](https://mlsanity.com/paper/2002.02518/) (2020)
- [VeLO: Training Versatile Learned Optimizers by Scaling Up](https://mlsanity.com/paper/2211.09760/) (2022)
- [Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies](https://mlsanity.com/paper/2112.13835/) (2021)
- [Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Tasks](https://mlsanity.com/paper/2410.23208/) (2024)
- [Discovering Evolution Strategies via Meta-Black-Box Optimization](https://mlsanity.com/paper/2211.11260/) (2022)
- [Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning](https://mlsanity.com/paper/2509.24372/) (2025)
- [Zeroth-Order Fine-Tuning of LLMs in Random Subspaces](https://mlsanity.com/paper/2410.08989/) (2024)
- [Can Learned Optimization Make Reinforcement Learning Less Difficult?](https://mlsanity.com/paper/2407.07082/) (2024)
- [Simplified Minimal Gated Unit Variations for Recurrent Neural Networks](https://mlsanity.com/paper/1701.03452/) (2017)
- [Derivative-Free Optimization for Low-Rank Adaptation in Large Language Models](https://mlsanity.com/paper/2403.01754/) (2024)
- [Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-Single](https://mlsanity.com/paper/2304.11153/) (2023)

## Built on by (most cited first, 25 of 25)

- [Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training](https://mlsanity.com/paper/2604.01499/) (2026)
- [Evolutionary Strategies lead to Catastrophic Forgetting in LLMs](https://mlsanity.com/paper/2601.20861/) (2026)
- [Pretraining Recurrent Networks without Recurrence](https://mlsanity.com/paper/2606.06479/) (2026)
- [Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies](https://mlsanity.com/paper/2605.30148/) (2026)
- [Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirements](https://mlsanity.com/paper/2608.17310/) (2026)
- [Fine-Tuning Language Models to Know What They Know](https://mlsanity.com/paper/2602.02605/) (2026)
- [PopuLoRA: Co-Evolving LLM Populations for Reasoning Self-Play](https://mlsanity.com/paper/2605.16727/) (2026)
- [Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies](https://mlsanity.com/paper/2608.12679/) (2026)
- [Evolutionary System 2 Reasoning: An Empirical Proof](https://mlsanity.com/paper/2512.05760/) (2025)
- [Dual-Evolving Reinforcement Learning for Automated Reward Design](https://mlsanity.com/paper/2512.13399/) (2025)
- [Training Non-Differentiable Networks via Optimal Transport](https://mlsanity.com/paper/2605.01928/) (2026)
- [Evolutionary Intelligence for Scientific Discovery: From Evolutionary Computation to Cumulative Discovery Systems](https://mlsanity.com/paper/2607.09025/) (2026)
- [Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging](https://mlsanity.com/paper/2608.05541/) (2026)
- [Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO](https://mlsanity.com/paper/2608.27351/) (2026)
- [Coverage, Not Credit: Failure-Credit Routing of Zeroth-Order Perturbation Budgets Does Not Improve On-Pool Sample Efficiency for LLM Agents](https://mlsanity.com/paper/2608.28011/) (2026)
- [A Little Rank Goes a Long Way: Random Scaffolds with LoRA Adapters Are All You Need](https://mlsanity.com/paper/2604.08749/) (2026)
- [Evolving Many Worlds: Towards Open-Ended Discovery in Petri Dish NCA via Population-Based Training](https://mlsanity.com/paper/2604.11248/) (2026)
- [Inference-Native Zeroth-Order Optimization for LLMs](https://mlsanity.com/paper/2605.28760/) (2026)
- [Gradient-Free Training of Spiking Neural Networks via Low-Rank Evolution Strategies](https://mlsanity.com/paper/2605.30361/) (2026)
- [Knowing in Advance When an Evolutionary Outer Loop Will Not Help: A Pre-Registered Cheap-Baseline Screening Rule](https://mlsanity.com/paper/2606.29119/) (2026)
- [One More Time: Revisiting Neural Quantum States from a Reinforcement Learning Perspective](https://mlsanity.com/paper/2607.02292/) (2026)
- [Can Bayesian Optimization Efficiently Find a Strong Single Expert in Neural Thickets?](https://mlsanity.com/paper/2608.10867/) (2026)
- [EGGROLL, Unrolled: Understanding and Improving Low-Rank Evolution Strategies at Scale](https://mlsanity.com/paper/2609.10980/) (2026)
- [Continual Reinforcement Learning with Neuroevolution](https://mlsanity.com/paper/2610.01583/) (2026)
- [DART-ES: Difficulty-Aware Reweighting and Targeted Replay for Fine-Tuning LLMs with Evolution Strategies](https://mlsanity.com/paper/2610.06993/) (2026)

## Abstract (arXiv)

Evolution Strategies (ES) is a class of powerful black-box optimisation methods that are highly parallelisable and can handle non-differentiable and noisy objectives. However, na"ive ES becomes prohibitively expensive at scale on GPUs due to the low arithmetic intensity of batched matrix multiplications with unstructured random perturbations. We introduce Evolution Guided GeneRal Optimisation via Low-rank Learning (EGGROLL), which improves arithmetic intensity by structuring individual perturbations as rank-r matrices, resulting in a hundredfold increase in training speed for billion-parameter models at large population sizes, achieving up to 91% of the throughput of pure batch inference. We provide a rigorous theoretical analysis of Gaussian ES for high-dimensional parameter objectives, investigating conditions needed for ES updates to converge in high dimensions. Our results reveal a linearising effect, and proving consistency between EGGROLL and ES as parameter dimension increases. Our experiments show that EGGROLL: (1) enables the stable pretraining of nonlinear recurrent language models that operate purely in integer datatypes, (2) is competitive with GRPO for post-training LLMs on reasoning tasks, and (3) does not compromise performance compared to ES in tabula rasa RL settings, despite being faster.