mlsanityData to 14 Oct 2026

EvoRL: A GPU-accelerated Framework for Evolutionary Reinforcement Learning

arXiv 2501.15129 ↗First posted 25 Jan 2025cs.NEcited by 1

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Evolutionary Reinforcement Learning (EvoRL) has emerged as a promising approach to overcoming the limitations of traditional reinforcement learning (RL) by integrating the Evolutionary Computation (EC) paradigm with RL. However, the population-based nature of EC significantly increases computational costs, thereby restricting the exploration of algorithmic design choices and scalability in large-scale settings. To address this challenge, we introduce textttEvoRL, the first end-to-end EvoRL framework optimized for GPU acceleration. The framework executes the entire training pipeline on accelerators, including environment simulations and EC processes, leveraging hierarchical parallelism through vectorization and compilation techniques to achieve superior speed and scalability. This design enables the efficient training of large populations on a single machine. In addition to its performance-oriented design, textttEvoRL offers a comprehensive platform for EvoRL research, encompassing implementations of traditional RL algorithms (e.g., A2C, PPO, DDPG, TD3, SAC), Evolutionary Algorithms (e.g., CMA-ES, OpenES, ARS), and hybrid EvoRL paradigms such as Evolutionary-guided RL (e.g., ERL, CEM-RL) and Population-Based AutoRL (e.g., PBT). The framework's modular architecture and user-friendly interface allow researchers to seamlessly integrate new components, customize algorithms, and conduct fair benchmarking and ablation studies. The project is open-source and available at: https://github.com/EMI-Group/evorl.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 53 of its 87 references that are in the map. First posted on arXiv 25 Jan 2025.

Builds on

53 of 87 references.

References · 53

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  6. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  7. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  8. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
  9. 2016Deep Exploration via Bootstrapped DQNcited by 527
  10. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  11. 2016The CMA Evolution Strategy: A Tutorialcited by 377
  12. 2017Population Based Training of Neural Networkscited by 367
38 more
  1. 2018Distributed Prioritized Experience Replaycited by 269
  2. 2017Count-Based Exploration with Neural Density Modelscited by 257
  3. 2020pymoo: Multi-objective Optimization in Pythoncited by 200
  4. 2020Revisiting Fundamentals of Experience Replaycited by 175
  5. 2018Deep Reinforcement Learning and the Deadly Triadcited by 174
  6. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 149
  7. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 122
  8. 2020Effective Diversity in Population Based Reinforcement Learningcited by 119
  9. 2020Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehiclescited by 110
  10. 2018GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learningcited by 90
  11. 2018Evolved Policy Gradientscited by 88
  12. 2018Evolution-Guided Policy Gradient in Reinforcement Learningcited by 83
  13. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 63
  14. 2018CEM-RL: Combining evolutionary and gradient-based methods for policy searchcited by 58
  15. 2023Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAXcited by 58
  16. 2020Smooth Exploration for Robotic Reinforcement Learningcited by 54
  17. 2023Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learningcited by 51
  18. 2020Discount Factor as a Regularizer in Reinforcement Learningcited by 45
  19. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 45
  20. 2017PlatEMO: A MATLAB Platform for Evolutionary Multi-Objective Optimizationcited by 41
  21. 2019Reinforcement Learning for Market Making in a Multi-agent Dealer Marketcited by 39
  22. 2019Evolving Rewards to Automate Reinforcement Learningcited by 31
  23. 2023DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Trainingcited by 29
  24. 2018Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Ataricited by 28
  25. 2019Proximal Distilled Evolutionary Reinforcement Learningcited by 27
  26. 2019On Inductive Biases in Deep Reinforcement Learningcited by 24
  27. 2022Combining Evolution and Deep Reinforcement Learning for Policy Search: a Surveycited by 23
  28. 2023EvoX: A Distributed GPU-accelerated Framework for Scalable Evolutionary Computationcited by 18
  29. 2023QDax: A Library for Quality-Diversity and Population-based Algorithms with Hardware Accelerationcited by 17
  30. 2021Mava: a research library for distributed multi-agent reinforcement learning in JAXcited by 16
  31. 2022ERL-Re2: Efficient Evolutionary Reinforcement Learning with Shared State Representation and Individual Policy Representationcited by 14
  32. 2020Online Hyper-parameter Tuning in Off-policy Learning via Evolutionary Strategiescited by 11
  33. 2023minimax: Efficient Baselines for Autocurricula in JAXcited by 10
  34. 2023EvoTorch: Scalable Evolutionary Computation in Pythoncited by 9
  35. 2024EvIL: Evolution Strategies for Generalisable Imitation Learningcited by 8
  36. 2022Evolutionary Action Selection for Gradient-based Policy Learningcited by 7
  37. 2022Fast Population-Based Reinforcement Learning on a Single Machinecited by 7
  38. 2021Towards Automatic Actor-Critic Solutions to Continuous Controlcited by 5

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Evolutionary Reinforcement Learning: A Systematic Review and Future Directions
  2. 2023BiERL: A Meta Evolutionary Reinforcement Learning Framework via Bilevel Optimization
  3. 2024Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms
  4. 2024Ancestral Reinforcement Learning: Unifying Zeroth-Order Optimization and Genetic Algorithms for Reinforcement Learning
  5. 2026RLDISCOVER: LLM-driven co-evolution of reinforcement learning algorithms
  6. 2026Large Language Model-Powered Evolutionary Code Optimization on a Phylogenetic Tree
  7. 2025Evolutionary Policy Optimization
  8. 2026Evolutionary Discovery of Reinforcement Learning Algorithms via Large Language Models