mlsanityData to 7 Oct 2026

Evolutionary Action Selection for Gradient-based Policy Learning

arXiv 2201.04286 ↗First posted 12 Jan 2022cs.NE cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Evolutionary Algorithms (EAs) and Deep Reinforcement Learning (DRL) have recently been integrated to take the advantage of the both methods for better exploration and exploitation.The evolutionary part in these hybrid methods maintains a population of policy networks.However, existing methods focus on optimizing the parameters of policy network, which is usually high-dimensional and tricky for EA.In this paper, we shift the target of evolution from high-dimensional parameter space to low-dimensional action space.We propose Evolutionary Action Selection-Twin Delayed Deep Deterministic Policy Gradient (EAS-TD3), a novel hybrid method of EA and DRL.In EAS, we focus on optimizing the action chosen by the policy network and attempt to obtain high-quality actions to promote policy learning through an evolutionary algorithm. We conduct several experiments on challenging continuous control tasks.The result shows that EAS-TD3 shows superior performance over other state-of-art methods.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 30 of its 64 references that are in the map. First posted on arXiv 12 Jan 2022.

Builds on

30 of 64 references.

References · 30

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  6. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  7. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  8. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  9. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  10. 2014Robots that can adapt like animalscited by 203
  11. 2017Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learningcited by 148
  12. 2018On Learning Intrinsic Rewards for Policy Gradient Methodscited by 100
18 more
  1. 2018Evolution-Guided Policy Gradient in Reinforcement Learningcited by 78
  2. 2018CEM-RL: Combining evolutionary and gradient-based methods for policy searchcited by 53
  3. 2020Diversity Policy Gradient for Sample Efficient Quality-Diversity Optimizationcited by 53
  4. 2018GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithmscited by 40
  5. 2019Word-level Textual Adversarial Attacking as Combinatorial Optimizationcited by 37
  6. 2019Collaborative Evolutionary Reinforcement Learningcited by 36
  7. 2017Deep Deterministic Policy Gradient for Urban Traffic Light Controlcited by 30
  8. 2018Learning Self-Imitating Diverse Policiescited by 29
  9. 2021Self-Imitation Learning by Planningcited by 25
  10. 2019Proximal Distilled Evolutionary Reinforcement Learningcited by 24
  11. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  12. 2020Population-Guided Parallel Policy Search for Reinforcement Learningcited by 20
  13. 2020GRAC: Self-Guided and Self-Regularized Actor-Criticcited by 20
  14. 2019Q-Learning for Continuous Actions with Cross-Entropy Guided Policiescited by 17
  15. 2019Integrating Behavior Cloning and Reinforcement Learning for Improved Performance in Dense and Sparse Reward Environmentscited by 17
  16. 2021Derivative-Free Reinforcement Learning: A Reviewcited by 15
  17. 2020An Efficient Asynchronous Method for Integrating Evolutionary and Gradient-based Policy Searchcited by 12
  18. 2021Soft Actor-Critic with Cross-Entropy Policy Optimizationcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1; 2024: 1; 2025: 1.

5 papers.

  1. 2023Evolutionary Reinforcement Learning: A Surveycited by 24
  2. 2024Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithmscited by 23
  3. 2022Combining Evolution and Deep Reinforcement Learning for Policy Search: a Surveycited by 20
  4. 2022ERL-Re2: Efficient Evolutionary Reinforcement Learning with Shared State Representation and Individual Policy Representationcited by 11
  5. 2025Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategiescited by 0