mlsanityData to 7 Oct 2026

Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

arXiv 2110.01411 ↗First posted 28 Sep 2021cs.LG cs.AIcited by 23

Not on the map. Why?

Abstract (arXiv)

Deep Reinforcement Learning (DRL) and Evolution Strategies (ESs) have surpassed human-level control in many sequential decision-making problems, yet many open challenges still exist. To get insights into the strengths and weaknesses of DRL versus ESs, an analysis of their respective capabilities and limitations is provided. After presenting their fundamental concepts and algorithms, a comparison is provided on key aspects such as scalability, exploration, adaptation to dynamic environments, and multi-agent learning. Then, the benefits of hybrid algorithms that combine concepts from DRL and ESs are highlighted. Finally, to have an indication about how they compare in real-world applications, a survey of the literature for the set of applications they support is provided.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 131 of its 313 references that are in the map. First posted on arXiv 28 Sep 2021.

Builds on

131 of 313 references.

References · 131

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2016Improved Techniques for Training GANscited by 2,938
  6. 2015Trust Region Policy Optimizationcited by 2,503
  7. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  8. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  9. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  10. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  11. 2017Hindsight Experience Replaycited by 1,096
  12. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
38 more
  1. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  2. 2018Exploration by Random Network Distillationcited by 966
  3. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  4. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  5. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  6. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  7. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  8. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  9. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  10. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  11. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  12. 2016Deep Exploration via Bootstrapped DQNcited by 518
  13. 2019Model-Based Reinforcement Learning for Ataricited by 495
  14. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  15. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  16. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  17. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  18. 2017A Tutorial on Thompson Samplingcited by 420
  19. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 377
  20. 2016Learning to reinforcement learncited by 369
  21. 2016The CMA Evolution Strategy: A Tutorialcited by 364
  22. 2016Learning Multiagent Communication with Backpropagationcited by 355
  23. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  24. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  25. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  26. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  27. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
  28. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  29. 2019Self-Supervised Exploration via Disagreementcited by 263
  30. 2018Distributed Prioritized Experience Replaycited by 260
  31. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  32. 2017Count-Based Exploration with Neural Density Modelscited by 255
  33. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  34. 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
  35. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 207
  36. 2018An Introduction to Deep Reinforcement Learningcited by 201
  37. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 200
  38. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 189

50 most cited of 131 All 131 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 3; 2024: 7; 2025: 5; 2026: 2.

23 papers.

Sort:

By area:

  1. 2022Open-environment Machine Learningcited by 48
  2. 2022Automated Dynamic Algorithm Configurationcited by 37
  3. 2024Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithmscited by 23
  4. 2022Combining Evolution and Deep Reinforcement Learning for Policy Search: a Surveycited by 20
  5. 2023Games for Artificial Intelligence Research: A Review and Perspectivescited by 14
  6. 2025Hybrid Action Based Reinforcement Learning for Multi-Objective Compatible Autonomous Drivingcited by 7
  7. 2022Evolutionary Action Selection for Gradient-based Policy Learningcited by 5
  8. 2022Pearl: Parallel Evolutionary and Reinforcement Learning Librarycited by 2
  9. 2024UnifiedRL: A Reinforcement Learning Algorithm Tailored for Multi-Task Fusion in Large-Scale Recommender Systemscited by 2
  10. 2025Utilizing Evolution Strategies to Train Transformers in Reinforcement Learningcited by 2
  11. 2024Evolutionary Computation for the Design and Enrichment of General-Purpose Artificial Intelligence Systems: Survey and Prospectscited by 1
  12. 2024The Digital Ecosystem of Beliefs: does evolution favour AI over humans?cited by 1
11 more
  1. 2025Utilizing Novelty-based Evolution Strategies to Train Transformers in Reinforcement Learningcited by 1
  2. 2026SMC-ES: Automated synthesis of formally verified control policiescited by 1
  3. 2022Covy: An AI-powered Robot with a Compound Vision System for Detecting Breaches in Social Distancingcited by 0
  4. 2023Efficient Reinforcement Learning for Jumping Monopodscited by 0
  5. 2023A Review of Deep Reinforcement Learning in Serverless Computing: Function Scheduling and Resource Auto-Scalingcited by 0
  6. 2024Hard-Thresholding Meets Evolution Strategies in Reinforcement Learningcited by 0
  7. 2024Visuospatial navigation from the bottom-up: without vestibular integration, distance prediction, or mapscited by 0
  8. 2024EnhancedRL: An Enhanced-State Reinforcement Learning Algorithm for Multi-Task Fusion in Recommender Systemscited by 0
  9. 2025Toward Safety-First Human-Like Decision Making for Autonomous Vehicles in Time-Varying Traffic Flowcited by 0
  10. 2025Harnessing Bounded-Support Evolution Strategies for Policy Refinementcited by 0
  11. 2026Safe and Robust Neural Policy Learning with Statistical Verification for Sim-to-Real Deployment in Roboticscited by 0