mlsanityData to 7 Oct 2026

Combining Evolution and Deep Reinforcement Learning for Policy Search: a Survey

arXiv 2203.14009 ↗First posted 26 Mar 2022cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

Deep neuroevolution and deep Reinforcement Learning have received a lot of attention in the last years. Some works have compared them, highlighting theirs pros and cons, but an emerging trend consists in combining them so as to benefit from the best of both worlds. In this paper, we provide a survey of this emerging trend by organizing the literature into related groups of works and casting all the existing combinations in each group into a generic framework. We systematically cover all easily available papers irrespective of their publication status, focusing on the combination mechanisms rather than on the experimental results. In total, we cover 45 algorithms more recent than 2017. We hope this effort will favor the growth of the domain by facilitating the understanding of the relationships between the methods, leading to deeper analyses, outlining missing useful comparisons and suggesting new combinations of mechanisms.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 66 of its 111 references that are in the map. First posted on arXiv 26 Mar 2022.

Builds on

66 of 111 references.

References · 66

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  8. 2017Hindsight Experience Replaycited by 1,096
  9. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  10. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  11. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  12. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
38 more
  1. 2017Population Based Training of Neural Networkscited by 351
  2. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  3. 2016Sample Efficient Actor-Critic with Experience Replaycited by 185
  4. 2021Embodied Intelligence via Learning and Evolutioncited by 154
  5. 2020Effective Diversity in Population Based Reinforcement Learningcited by 116
  6. 2017Quality and Diversity Optimization: A Unifying Modular Frameworkcited by 111
  7. 2017Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agentscited by 107
  8. 2017Behavior Trees in Robotics and AI: An Introductioncited by 105
  9. 2019Exploring Model-based Planning with Policy Networkscited by 90
  10. 2020Sample-efficient Cross-Entropy Method for Real-time Planningcited by 90
  11. 2018Evolved Policy Gradientscited by 86
  12. 2018Evolution-Guided Policy Gradient in Reinforcement Learningcited by 78
  13. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  14. 2018Reinforcement Learning for Improving Agent Designcited by 55
  15. 2018CEM-RL: Combining evolutionary and gradient-based methods for policy searchcited by 53
  16. 2018Structured Evolution with Compact Architectures for Scalable Policy Optimizationcited by 48
  17. 2017Stein Variational Policy Gradientcited by 42
  18. 2018GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithmscited by 40
  19. 2022Approximating Gradients for Differentiable Quality Diversity in Reinforcement Learningcited by 37
  20. 2019Collaborative Evolutionary Reinforcement Learningcited by 36
  21. 2019Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learningcited by 34
  22. 2020Model-Predictive Control via Cross-Entropy and Gradient-Based Optimizationcited by 32
  23. 2020Sample-Efficient Automated Deep Reinforcement Learningcited by 31
  24. 2018Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Ataricited by 26
  25. 2019Proximal Distilled Evolutionary Reinforcement Learningcited by 24
  26. 2019Deep Reinforcement Learning using Genetic Algorithm for Parameter Optimizationcited by 23
  27. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  28. 2020Population-Guided Parallel Policy Search for Reinforcement Learningcited by 20
  29. 2020GRAC: Self-Guided and Self-Regularized Actor-Criticcited by 20
  30. 2019Q-Learning for Continuous Actions with Cross-Entropy Guided Policiescited by 17
  31. 2021Derivative-Free Reinforcement Learning: A Reviewcited by 15
  32. 2017Safe Mutations for Deep and Recurrent Neural Networks through Output Gradientscited by 14
  33. 2020An Efficient Asynchronous Method for Integrating Evolutionary and Gradient-based Policy Searchcited by 12
  34. 2022A Surrogate-Assisted Controller for Expensive Evolutionary Reinforcement Learningcited by 11
  35. 2018Policy Search in Continuous Action Domains: an Overviewcited by 10
  36. 2020Online Hyper-parameter Tuning in Off-policy Learning via Evolutionary Strategiescited by 9
  37. 2018Genetic-Gated Networks for Deep Reinforcementcited by 7
  38. 2020Cooperative Heterogeneous Deep Reinforcement Learningcited by 7

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 2; 2025: 9; 2026: 1.

20 papers.

Sort:

By area:

  1. 2023Evolutionary Reinforcement Learning: A Surveycited by 24
  2. 2024Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithmscited by 23
  3. 2022ERL-Re2: Efficient Evolutionary Reinforcement Learning with Shared State Representation and Individual Policy Representationcited by 11
  4. 2023Improving Deep Policy Gradients with Value Function Searchcited by 7
  5. 2023Quality Diversity under Sparse Reward and Sparse Interaction: Application to Grasping in Roboticscited by 6
  6. 2025Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challengescited by 4
  7. 2023Artificial Collective Intelligence Engineering: a Survey of Concepts and Perspectivescited by 3
  8. 2025Utilizing Evolution Strategies to Train Transformers in Reinforcement Learningcited by 2
  9. 2025Survey and Tutorial of Reinforcement Learning Methods in Process Systems Engineeringcited by 2
  10. 2023Evolving Constrained Reinforcement Learning Policycited by 1
  11. 2023Evolutionary Strategy Guided Reinforcement Learning via MultiBuffer Communicationcited by 1
  12. 2024Evolutionary Reinforcement Learning via Cooperative Coevolutioncited by 1
8 more
  1. 2025Utilizing Novelty-based Evolution Strategies to Train Transformers in Reinforcement Learningcited by 1
  2. 2025Surrogate-Assisted Evolutionary Reinforcement Learning Based on Autoencoder and Hyperbolic Neural Networkcited by 1
  3. 2022Curiosity creates Diversity in Policy Searchcited by 0
  4. 2025Synergizing Reinforcement Learning and Genetic Algorithms for Neural Combinatorial Optimizationcited by 0
  5. 2025Robust Dynamic Material Handling via Adaptive Constrained Evolutionary Reinforcement Learningcited by 0
  6. 2025DyBBT: Dynamic Balance via Bandit-inspired Targeting for Dialog Policy with Cognitive Dual-Systemscited by 0
  7. 2025Harnessing Bounded-Support Evolution Strategies for Policy Refinementcited by 0
  8. 2026Contextual Quality-Diversity Evolutionary Reinforcement Learning for HVAC Control in Tropical Commercial Buildingscited by 0