Bidirectional Progressive Neural Networks with Episodic Return Progress for Emergent Task Sequencing and Robotic Skill Transfer
On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation
Abstract (arXiv)
Human brain and behavior provide a rich venue that can inspire novel control and learning methods for robotics. In an attempt to exemplify such a development by inspiring how humans acquire knowledge and transfer skills among tasks, we introduce a novel multi-task reinforcement learning framework named Episodic Return Progress with Bidirectional Progressive Neural Networks (ERP-BPNN). The proposed ERP-BPNN model (1) learns in a human-like interleaved manner by (2) autonomous task switching based on a novel intrinsic motivation signal and, in contrast to existing methods, (3) allows bidirectional skill transfer among tasks. ERP-BPNN is a general architecture applicable to several multi-task learning settings; in this paper, we present the details of its neural architecture and show its ability to enable effective learning and skill transfer among morphologically different robots in a reaching task. The developed Bidirectional Progressive Neural Network (BPNN) architecture enables bidirectional skill transfer without requiring incremental training and seamlessly integrates with online task arbitration. The task arbitration mechanism developed is based on soft Episodic Return progress (ERP), a novel intrinsic motivation (IM) signal. To evaluate our method, we use quantifiable robotics metrics such as 'expected distance to goal' and 'path straightness' in addition to the usual reward-based measure of episodic return common in reinforcement learning. With simulation experiments, we show that ERP-BPNN achieves faster cumulative convergence and improves performance in all metrics considered among morphologically different robots compared to the baselines.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 22 of its 64 references that are in the map. First posted on arXiv 6 Mar 2024.
Builds on
22 of 64 references.
References · 22
- 2014Adam: A Method for Stochastic Optimizationcited by 37,093
- 2017Proximal Policy Optimization Algorithmscited by 14,592
- 2016Progressive Neural Networkscited by 1,347
- 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,322
- 2015Prioritized Experience Replaycited by 1,078
- 2018Exploration by Random Network Distillationcited by 971
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 739
- 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 363
- 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 351
- 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 271
- 2017Distral: Robust Multitask Reinforcement Learningcited by 231
- 2018Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networkscited by 113
10 more
- 2014Learning to Executecited by 106
- 2016Sim-to-Real Robot Learning from Pixels with Progressive Netscited by 89
- 2017Surprise-Based Intrinsic Motivation for Deep Reinforcement Learningcited by 83
- 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 80
- 2018CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learningcited by 79
- 2016Deep Reinforcement Learning with Successor Features for Navigation across Similar Environmentscited by 50
- 2021From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligencecited by 35
- 2020Knowledge Transfer in Multi-Task Deep Reinforcement Learning for Continuous Controlcited by 27
- 2019Adaptive Scheduling for Multi-Task Learningcited by 24
- 2019Generalization in Transfer Learningcited by 7
Built on by
Citing papers per year: 2025: 1.
1 paper.