Optimal Actor-Critic Policy with Optimized Training Datasets
Not on the map. Why?
Abstract (arXiv)
Actor-critic (AC) algorithms are known for their efficacy and high performance in solving reinforcement learning problems, but they also suffer from low sampling efficiency. An AC based policy optimization process is iterative and needs to frequently access the agent-environment system to evaluate and update the policy by rolling out the policy, collecting rewards and states (i.e. samples), and learning from them. It ultimately requires a huge number of samples to learn an optimal policy. To improve sampling efficiency, we propose a strategy to optimize the training dataset that contains significantly less samples collected from the AC process. The dataset optimization is made of a best episode only operation, a policy parameter-fitness model, and a genetic algorithm module. The optimal policy network trained by the optimized training dataset exhibits superior performance compared to many contemporary AC algorithms in controlling autonomous dynamical systems. Evaluation on standard benchmarks show that the method improves sampling efficiency, ensures faster convergence to optima, and is more data-efficient than its counterparts.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 29 of its 44 references that are in the map. First posted on arXiv 16 Aug 2021.
Builds on
29 of 44 references.
References · 29
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
- 2015Trust Region Policy Optimizationcited by 2,503
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
- 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
- 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
- 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
- 2015End-to-End Training of Deep Visuomotor Policiescited by 847
- 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
17 more
- 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
- 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
- 2016Sample Efficient Actor-Critic with Experience Replaycited by 185
- 2018Learning to Walk via Deep Reinforcement Learningcited by 153
- 2017Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learningcited by 148
- 2018Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learningcited by 121
- 2017Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximationcited by 119
- 2019A Survey of Optimization Methods from a Machine Learning Perspectivecited by 90
- 2018Evolution-Guided Policy Gradient in Reinforcement Learningcited by 78
- 2016Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Criticcited by 77
- 2019Data Efficient Reinforcement Learning for Legged Robotscited by 60
- 2017Evolving Deep Convolutional Neural Networks for Image Classificationcited by 54
- 2019Collaborative Evolutionary Reinforcement Learningcited by 36
- 2019Deep Reinforcement Learning using Genetic Algorithm for Parameter Optimizationcited by 23
- 2020Scalable Multi-Task Imitation Learning with Autonomous Improvementcited by 21
- 2018A General End-to-end Diagnosis Framework for Manufacturing Systemscited by 2
- 2018GADAM: Genetic-Evolutionary ADAM for Deep Neural Network Optimizationcited by 1
Built on by
Citing papers per year: 2022: 1; 2023: 1.
2 papers.