Exploring CausalWorld: Enhancing robotic manipulation via knowledge transfer and curriculum learning
Not on the map. Why?
Abstract (arXiv)
This study explores a learning-based tri-finger robotic arm manipulating task, which requires complex movements and coordination among the fingers. By employing reinforcement learning, we train an agent to acquire the necessary skills for proficient manipulation. To enhance the efficiency and effectiveness of the learning process, two knowledge transfer strategies, fine-tuning and curriculum learning, were utilized within the soft actor-critic architecture. Fine-tuning allows the agent to leverage pre-trained knowledge and adapt it to new tasks. Several variations like model transfer, policy transfer, and across-task transfer were implemented and evaluated. To eliminate the need for pretraining, curriculum learning decomposes the advanced task into simpler, progressive stages, mirroring how humans learn. The number of learning stages, the context of the sub-tasks, and the transition timing were found to be the critical design parameters. The key factors of two learning strategies and corresponding effects were explored in context-aware and context-unaware scenarios, enabling us to identify the scenarios where the methods demonstrate optimal performance, derive conclusive insights, and contribute to a broader range of learning-based engineering applications.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 41 references that are in the map. First posted on arXiv 25 Mar 2024.
Builds on
27 of 41 references.
References · 27
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
- 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
- 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
- 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
- 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 259
- 2020CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learningcited by 109
- 2016Sim-to-Real Robot Learning from Pixels with Progressive Netscited by 89
- 2019CoPhy: Counterfactual Learning of Physical Dynamicscited by 80
- 2020Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learningcited by 53
15 more
- 2020Self-Paced Deep Reinforcement Learningcited by 49
- 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
- 2021Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Reviewcited by 34
- 2021Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFingercited by 34
- 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 33
- 2023Stable and Safe Reinforcement Learning via a Barrier-Lyapunov Actor-Critic Approachcited by 11
- 2020Guided Curriculum Learning for Walking Over Complex Terraincited by 10
- 2023Multi-modal Facial Action Unit Detection with Large Pre-trained Models for the 5th Competition on Affective Behavior Analysis in-the-wildcited by 8
- 2019An Optimization Framework for Task Sequencing in Curriculum Learningcited by 7
- 2022Causal Counterfactuals for Improving the Robustness of Reinforcement Learningcited by 5
- 2023Automatic Curriculum Learning with Gradient Reward Signalscited by 3
- 2024Stable and Safe Human-aligned Reinforcement Learning through Neural Ordinary Differential Equationscited by 3
- 2022End-to-end Reinforcement Learning of Robotic Manipulation with Robust Keypoints Representationcited by 2
- 2022Human Decision Makings on Curriculum Reinforcement Learning with Difficulty Adjustmentcited by 2
- 2013Knowledge Engineering Within A Generalized Bayesian Frameworkcited by 1
Built on by
Citing papers per year: 2025: 2.
2 papers.