mlsanityData to 14 Oct 2026

Evolution Strategies for Deep RL pretraining

arXiv 2604.00066 ↗First posted 31 Mar 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Although Deep Reinforcement Learning has proven highly effective for complex decision-making problems, it demands significant computational resources and careful parameter adjustment in order to develop successful strategies. Evolution strategies offer a more straightforward, derivative-free approach that is less computationally costly and simpler to deploy. However, ES generally do not match the performance levels achieved by DRL, which calls into question their suitability for more demanding scenarios. This study examines the performance of ES and DRL across tasks of varying difficulty, including Flappy Bird, Breakout and Mujoco environments, as well as whether ES could be used for initial training to enhance DRL algorithms. The results indicate that ES do not consistently train faster than DRL. When used as a preliminary training step, they only provide benefits in less complex environments (Flappy Bird) and show minimal or no improvement in training efficiency or stability across different parameter settings when applied to more sophisticated tasks (Breakout and MuJoCo Walker).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 6 of its 8 references that are in the map. First posted on arXiv 31 Mar 2026.

Builds on

6 of 8 references.

References · 6

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
  4. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  5. 2019Challenges of Real-World Reinforcement Learningcited by 291
  6. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 109

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning
  2. 2026Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training
  3. 2023Pre-training with Synthetic Data Helps Offline Reinforcement Learning
  4. 2026Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies
  5. 2026RLDISCOVER: LLM-driven co-evolution of reinforcement learning algorithms
  6. 2024Robust Deep Reinforcement Learning Through Adversarial Attacks and Training : A Survey
  7. 2026Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies
  8. 2025Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategies