# Efficacy of Modern Neuro-Evolutionary Strategies for Continuous Control Optimization

arXiv: https://arxiv.org/abs/1912.05239 · HTML: https://mlsanity.com/paper/1912.05239/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2019.

## Built on by (most cited first, 15 of 15)

- [Approximating Gradients for Differentiable Quality Diversity in Reinforcement Learning](https://mlsanity.com/paper/2202.03666/) (2022)
- [Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey](https://mlsanity.com/paper/2110.01411/) (2021)
- [Brain-inspired learning in artificial neural networks: a review](https://mlsanity.com/paper/2305.11252/) (2023)
- [Training Diverse High-Dimensional Controllers by Scaling Covariance Matrix Adaptation MAP-Annealing](https://mlsanity.com/paper/2210.02622/) (2022)
- [Multitask Neuroevolution for Reinforcement Learning with Long and Short Episodes](https://mlsanity.com/paper/2203.10844/) (2022)
- [AutoQD: Automatic Discovery of Diverse Behaviors with Quality-Diversity Optimization](https://mlsanity.com/paper/2506.05634/) (2025)
- [The Role of Morphological Variation in Evolutionary Robotics: Maximizing Performance and Robustness](https://mlsanity.com/paper/2208.02809/) (2022)
- [Utilizing Evolution Strategies to Train Transformers in Reinforcement Learning](https://mlsanity.com/paper/2501.13883/) (2025)
- [Policy Search with Rare Significant Events: Choosing the Right Partner to Cooperate with](https://mlsanity.com/paper/2103.06846/) (2021)
- [Qualitative Differences Between Evolutionary Strategies and Reinforcement Learning Methods for Control of Autonomous Agents](https://mlsanity.com/paper/2205.07592/) (2022)
- [Utilizing Novelty-based Evolution Strategies to Train Transformers in Reinforcement Learning](https://mlsanity.com/paper/2502.06301/) (2025)
- [Bio-Inspired Plastic Neural Networks for Zero-Shot Out-of-Distribution Generalization in Complex Animal-Inspired Robots](https://mlsanity.com/paper/2503.12406/) (2025)
- [Accelerating Deep Reinforcement Learning for Digital Twin Network Optimization with Evolutionary Strategies](https://mlsanity.com/paper/2202.00360/) (2022)
- [Learning and evolution: factors influencing an effective combination](https://mlsanity.com/paper/2306.11761/) (2023)
- [Harnessing Bounded-Support Evolution Strategies for Policy Refinement](https://mlsanity.com/paper/2511.09923/) (2025)

## Abstract (arXiv)

We analyze the efficacy of modern neuro-evolutionary strategies for continuous control optimization. Overall, the results collected on a wide variety of qualitatively different benchmark problems indicate that these methods are generally effective and scale well with respect to the number of parameters and the complexity of the problem. Moreover, they are relatively robust with respect to the setting of hyper-parameters. The comparison of the most promising methods indicates that the OpenAI-ES algorithm outperforms or equals the other algorithms on all considered problems. Moreover, we demonstrate how the reward functions optimized for reinforcement learning methods are not necessarily effective for evolutionary strategies and vice versa. This finding can lead to reconsideration of the relative efficacy of the two classes of algorithm since it implies that the comparisons performed to date are biased toward one or the other class.