Learning Complicated Manipulation Skills via Deterministic Policy with Limited Demonstrations
Not on the map. Why?
Abstract (arXiv)
Combined with demonstrations, deep reinforcement learning can efficiently develop policies for manipulators. However, it takes time to collect sufficient high-quality demonstrations in practice. And human demonstrations may be unsuitable for robots. The non-Markovian process and over-reliance on demonstrations are further challenges. For example, we found that RL agents are sensitive to demonstration quality in manipulation tasks and struggle to adapt to demonstrations directly from humans. Thus it is challenging to leverage low-quality and insufficient demonstrations to assist reinforcement learning in training better policies, and sometimes, limited demonstrations even lead to worse performance. We propose a new algorithm named TD3fG (TD3 learning from a generator) to solve these problems. It forms a smooth transition from learning from experts to learning from experience. This innovation can help agents extract prior knowledge while reducing the detrimental effects of the demonstrations. Our algorithm performs well in Adroit manipulator and MuJoCo tasks with limited demonstrations.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 33 references that are in the map. First posted on arXiv 29 Mar 2023.
Builds on
22 of 33 references.
References · 22
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
- 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
- 2016OpenAI Gymcited by 1,684
- 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
- 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
- 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
- 2015End-to-End Training of Deep Visuomotor Policiescited by 847
- 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
- 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
10 more
- 2019Behavior Regularized Offline Reinforcement Learningcited by 654
- 2018Learning Dexterous In-Hand Manipulationcited by 643
- 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
- 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
- 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
- 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
- 2019Challenges of Real-World Reinforcement Learningcited by 282
- 2016Query-Efficient Imitation Learning for End-to-End Autonomous Drivingcited by 83
- 2013Tractable Inference for Complex Stochastic Processescited by 20
- 2020Policy Learning Using Weak Supervisioncited by 13
Built on by
Citing papers per year: 2025: 1.
1 paper.