mlsanityData to 7 Oct 2026

Evolution Strategies at the Hyperscale

arXiv 2511.16652 ↗First posted 20 Nov 2025cs.LG cs.AIcited by 22

Not on the map. Why?

Abstract (arXiv)

Evolution Strategies (ES) is a class of powerful black-box optimisation methods that are highly parallelisable and can handle non-differentiable and noisy objectives. However, na"ive ES becomes prohibitively expensive at scale on GPUs due to the low arithmetic intensity of batched matrix multiplications with unstructured random perturbations. We introduce Evolution Guided GeneRal Optimisation via Low-rank Learning (EGGROLL), which improves arithmetic intensity by structuring individual perturbations as rank-r matrices, resulting in a hundredfold increase in training speed for billion-parameter models at large population sizes, achieving up to 91% of the throughput of pure batch inference. We provide a rigorous theoretical analysis of Gaussian ES for high-dimensional parameter objectives, investigating conditions needed for ES updates to converge in high dimensions. Our results reveal a linearising effect, and proving consistency between EGGROLL and ES as parameter dimension increases. Our experiments show that EGGROLL: (1) enables the stable pretraining of nonlinear recurrent language models that operate purely in integer datatypes, (2) is competitive with GRPO for post-training LLMs on reasoning tasks, and (3) does not compromise performance compared to ES in tabula rasa RL settings, despite being faster.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 33 of its 79 references that are in the map. First posted on arXiv 20 Nov 2025.

Builds on

33 of 79 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  6. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  7. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  8. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  9. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  10. 2016The CMA Evolution Strategy: A Tutorialcited by 364
  11. 2017Population Based Training of Neural Networkscited by 351
  12. 2023Fine-Tuning Language Models with Just Forward Passescited by 337
21 more
  1. 2024The Illusion of State in State-Space Modelscited by 181
  2. 2017Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learningcited by 148
  3. 2025RWKV-7 "Goose" with Expressive Dynamic State Evolutioncited by 134
  4. 2024Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmarkcited by 113
  5. 2022Discovered Policy Optimisationcited by 103
  6. 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 80
  7. 2023The MiniPile Challenge for Data-Efficient Language Modelscited by 74
  8. 2020Provably Efficient Online Hyperparameter Optimization with Population-Based Banditscited by 61
  9. 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
  10. 2021Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategiescited by 52
  11. 2022Discovering Evolution Strategies via Meta-Black-Box Optimizationcited by 44
  12. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 40
  13. 2025Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learningcited by 28
  14. 2024Zeroth-Order Fine-Tuning of LLMs in Random Subspacescited by 23
  15. 2024Can Learned Optimization Make Reinforcement Learning Less Difficult?cited by 14
  16. 2017Simplified Minimal Gated Unit Variations for Recurrent Neural Networkscited by 13
  17. 2024Derivative-Free Optimization for Low-Rank Adaptation in Large Language Modelscited by 11
  18. 2023Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-Singlecited by 7
  19. 2023Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategiescited by 5
  20. 2025How Should We Meta-Learn Reinforcement Learning Algorithms?cited by 5
  21. 2024ADIOS: Antibody Development via Opponent Shapingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 20.

22 papers.

Sort:

By area:

  1. 2026Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Trainingcited by 7
  2. 2026Evolutionary Strategies lead to Catastrophic Forgetting in LLMscited by 6
  3. 2026Pretraining Recurrent Networks without Recurrencecited by 5
  4. 2026Fine-Tuning Language Models to Know What They Knowcited by 2
  5. 2026PopuLoRA: Co-Evolving LLM Populations for Reasoning Self-Playcited by 2
  6. 2026Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategiescited by 2
  7. 2026Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategiescited by 2
  8. 2026Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirementscited by 2
  9. 2025Evolutionary System 2 Reasoning: An Empirical Proofcited by 1
  10. 2025Differentiable Evolutionary Reinforcement Learningcited by 1
  11. 2026Evolutionary Intelligence for Scientific Discovery: From Evolutionary Computation to Cumulative Discovery Systemscited by 1
  12. 2026Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Mergingcited by 1
10 more
  1. 2026Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPOcited by 1
  2. 2026Coverage, Not Credit: Failure-Credit Routing of Zeroth-Order Perturbation Budgets Does Not Improve On-Pool Sample Efficiency for LLM Agentscited by 1
  3. 2026Evolving Many Worlds: Towards Open-Ended Discovery in Petri Dish NCA via Population-Based Trainingcited by 0
  4. 2026Inference-Native Zeroth-Order Optimization for LLMscited by 0
  5. 2026Gradient-Free Training of Spiking Neural Networks via Low-Rank Evolution Strategiescited by 0
  6. 2026Knowing in Advance When an Evolutionary Outer Loop Will Not Help: A Pre-Registered Cheap-Baseline Screening Rulecited by 0
  7. 2026One More Time: Revisiting Neural Quantum States from a Reinforcement Learning Perspectivecited by 0
  8. 2026Can Bayesian Optimization Efficiently Find a Strong Single Expert in Neural Thickets?cited by 0
  9. 2026EGGROLL, Unrolled: Understanding and Improving Low-Rank Evolution Strategies at Scalecited by 0
  10. 2026Continual Reinforcement Learning with Neuroevolutioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms
  2. 2026Cooperative Coevolution for Resource-Constrained Agentic LLM Post-Training
  3. 2026ESSAM: A Novel Competitive Evolution Strategies Approach to Reinforcement Learning for Memory Efficient LLMs Fine-Tuning
  4. 2026Large Language Model-Powered Evolutionary Code Optimization on a Phylogenetic Tree
  5. 2026Hill Sampling for Test-Time Scaling: A Simple and Better Alternative to Repeated Sampling, Evolution, and Training
  6. 2025A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving
  7. 2026Evolutionary Soups: Evolving Mixture-of-Experts for Multi-Objective LLM Alignment
  8. 2025Population-Evolve: a Parallel Sampling and Evolutionary Method for LLM Math Reasoning