# Challenges in High-dimensional Reinforcement Learning with Evolution Strategies

arXiv: https://arxiv.org/abs/1806.01224 · HTML: https://mlsanity.com/paper/1806.01224/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2018.

## Built on by (most cited first, 6 of 6)

- [Proximal Policy Gradient Arborescence for Quality Diversity Reinforcement Learning](https://mlsanity.com/paper/2305.13795/) (2023)
- [Derivative-Free Reinforcement Learning: A Review](https://mlsanity.com/paper/2102.05710/) (2021)
- [Solving Deep Reinforcement Learning Tasks with Evolution Strategies and Linear Policy Networks](https://mlsanity.com/paper/2402.06912/) (2024)
- [Hybrid Self-Attention NEAT: A novel evolutionary approach to improve the NEAT algorithm](https://mlsanity.com/paper/2112.03670/) (2021)
- [Shape Change and Control of Pressure-based Soft Agents](https://mlsanity.com/paper/2205.00467/) (2022)
- [Discovering new robust local search algorithms with neuro-evolution](https://mlsanity.com/paper/2501.04747/) (2025)

## Abstract (arXiv)

Evolution Strategies (ESs) have recently become popular for training deep neural networks, in particular on reinforcement learning tasks, a special form of controller design. Compared to classic problems in continuous direct search, deep networks pose extremely high-dimensional optimization problems, with many thousands or even millions of variables. In addition, many control problems give rise to a stochastic fitness function. Considering the relevance of the application, we study the suitability of evolution strategies for high-dimensional, stochastic problems. Our results give insights into which algorithmic mechanisms of modern ES are of value for the class of problems at hand, and they reveal principled limitations of the approach. They are in line with our theoretical understanding of ESs. We show that combining ESs that offer reduced internal algorithm cost with uncertainty handling techniques yields promising methods for this class of problems.