Minimizing Human Assistance: Augmenting a Single Demonstration for Deep Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
The use of human demonstrations in reinforcement learning has proven to significantly improve agent performance. However, any requirement for a human to manually 'teach' the model is somewhat antithetical to the goals of reinforcement learning. This paper attempts to minimize human involvement in the learning process while retaining the performance advantages by using a single human example collected through a simple-to-use virtual reality simulation to assist with RL training. Our method augments a single demonstration to generate numerous human-like demonstrations that, when combined with Deep Deterministic Policy Gradients and Hindsight Experience Replay (DDPG + HER) significantly improve training time on simple tasks and allows the agent to solve a complex task (block stacking) that DDPG + HER alone cannot solve. The model achieves this significant training advantage using a single human example, requiring less than a minute of human input. Moreover, despite learning from a human example, the agent is not constrained to human-level performance, often learning a policy that is significantly different from the human demonstration.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 20 of its 35 references that are in the map. First posted on arXiv 22 Sep 2022.
Builds on
20 of 35 references.
References · 20
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
- 2017Hindsight Experience Replaycited by 1,096
- 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
- 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
- 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
- 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
- 2019Active Domain Randomizationcited by 161
- 2018Learning to Walk via Deep Reinforcement Learningcited by 153
- 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 124
8 more
- 2018Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learningcited by 121
- 2017Data-efficient Deep Reinforcement Learning for Dexterous Manipulationcited by 61
- 2019Data Efficient Reinforcement Learning for Legged Robotscited by 60
- 2018Reinforcement Learning for UAV Attitude Controlcited by 40
- 2018A Practical Approach to Insertion with Variable Socket Position Using Deep Reinforcement Learningcited by 35
- 2018Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methodscited by 32
- 2018Pre-training with Non-expert Human Demonstration for Deep Reinforcement Learningcited by 8
- 2020Dynamic Experience Replaycited by 7
Built on by
Citing papers per year: 2023: 3.
3 papers.