mlsanityData to 7 Oct 2026

Evolution Strategies as a Scalable Alternative to Reinforcement Learning

arXiv 1703.03864 ↗First posted 10 Mar 2017stat.ML cs.AI cs.LGcited by 555

Not on the map. Why?

Abstract (arXiv)

We explore the use of Evolution Strategies (ES), a class of black box optimization algorithms, as an alternative to popular MDP-based RL techniques such as Q-learning and Policy Gradients. Experiments on MuJoCo and Atari show that ES is a viable solution strategy that scales extremely well with the number of CPUs available: By using a novel communication strategy based on common random numbers, our ES implementation only needs to communicate scalars, making it possible to scale to over a thousand parallel workers. This allows us to solve 3D humanoid walking in 10 minutes and obtain competitive results on most Atari games after one hour of training. In addition, we highlight several advantages of ES as a black box optimization technique: it is invariant to action frequency and delayed rewards, tolerant of extremely long horizons, and does not need temporal discounting or value function approximation.

Key facts. Cited by 555 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 5; 2020: 3; 2021: 91; 2022: 87; 2023: 88; 2024: 72; 2025: 95; 2026: 114.

555 papers.

Sort:

By area:

  1. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  2. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  3. 2021GFlowNet Foundationscited by 286
  4. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 254
  5. 2021Trustworthy AI: From Principles to Practicescited by 206
  6. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  7. 2024Barren Plateaus in Variational Quantum Computingcited by 130
  8. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  9. 2022Discovered Policy Optimisationcited by 103
  10. 2021Gradients are Not All You Needcited by 82
  11. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
  12. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
38 more
  1. 2022Scaling Forward Gradient With Local Lossescited by 65
  2. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 59
  3. 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
  4. 2021Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategiescited by 52
  5. 2024Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costscited by 51
  6. 2021Fast and Efficient Locomotion via Learned Gait Transitionscited by 50
  7. 2021A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimizationcited by 46
  8. 2023Dataset Distillation with Convexified Implicit Gradientscited by 46
  9. 2021Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spacescited by 45
  10. 2022Discovering Evolution Strategies via Meta-Black-Box Optimizationcited by 44
  11. 2023Learning Cut Selection for Mixed-Integer Linear Programming via Hierarchical Sequence Modelcited by 44
  12. 2021Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planningcited by 43
  13. 2021MLGO: a Machine Learning Guided Compiler Optimizations Frameworkcited by 40
  14. 2021Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Conceptscited by 40
  15. 2025Towards General-Purpose Model-Free Reinforcement Learningcited by 40
  16. 2022Recent Advances for Quantum Neural Networks in Generative Learningcited by 39
  17. 2022evosax: JAX-based Evolution Strategiescited by 39
  18. 2022Approximating Gradients for Differentiable Quality Diversity in Reinforcement Learningcited by 37
  19. 2022Automated Dynamic Algorithm Configurationcited by 37
  20. 2023MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learningcited by 35
  21. 2023Discovering Attention-Based Genetic Algorithms via Meta-Black-Box Optimizationcited by 33
  22. 2022Human-level Atari 200x fastercited by 28
  23. 2023Gradient-Free Textual Inversioncited by 28
  24. 2025Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learningcited by 28
  25. 2022MuZero with Self-competition for Rate Control in VP9 Video Compressioncited by 27
  26. 2023Brain-inspired learning in artificial neural networks: a reviewcited by 26
  27. 2023DPZero: Private Fine-Tuning of Language Models without Backpropagationcited by 26
  28. 2021Introducing Symmetries to Black Box Meta Reinforcement Learningcited by 25
  29. 2025SPIDER: Scalable Physics-Informed Dexterous Retargetingcited by 25
  30. 2026Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weightscited by 25
  31. 2023Evolutionary Reinforcement Learning: A Surveycited by 24
  32. 2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learningcited by 24
  33. 2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
  34. 2021On the Convergence of Prior-Guided Zeroth-Order Optimization Algorithmscited by 23
  35. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  36. 2021The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learningcited by 23
  37. 2022Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discoverycited by 23
  38. 2023Convolutional Channel-wise Competitive Learning for the Forward-Forward Algorithmcited by 23

50 most cited of 555 All 555 (JSON)