# Knowledge Transfer in Multi-Task Deep Reinforcement Learning for Continuous Control

arXiv: https://arxiv.org/abs/2010.07494 · HTML: https://mlsanity.com/paper/2010.07494/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2020.

## Built on by (most cited first, 28 of 28)

- [Conservative Data Sharing for Multi-Task Offline Reinforcement Learning](https://mlsanity.com/paper/2109.08128/) (2021)
- [PaCo: Parameter-Compositional Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2210.11653/) (2022)
- [Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning](https://mlsanity.com/paper/2207.04703/) (2022)
- [Polybot: Training One Policy Across Robots While Embracing Variability](https://mlsanity.com/paper/2307.03719/) (2023)
- [Towards Versatile Embodied Navigation](https://mlsanity.com/paper/2210.16822/) (2022)
- [Multi-Task Imitation Learning for Linear Dynamical Systems](https://mlsanity.com/paper/2212.00186/) (2022)
- [Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learners](https://mlsanity.com/paper/2505.23150/) (2025)
- [Using Both Demonstrations and Language Instructions to Efficiently Learn Robotic Tasks](https://mlsanity.com/paper/2210.04476/) (2022)
- [Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulation](https://mlsanity.com/paper/2406.09738/) (2024)
- [Multi-Agent Policy Transfer via Task Relationship Modeling](https://mlsanity.com/paper/2203.04482/) (2022)
- [Offline Multi-task Transfer RL with Representational Penalization](https://mlsanity.com/paper/2402.12570/) (2024)
- [HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning](https://mlsanity.com/paper/2405.18080/) (2024)
- [Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining](https://mlsanity.com/paper/2410.00564/) (2024)
- [Switch Trajectory Transformer with Distributional Value Approximation for Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2203.07413/) (2022)
- [Beyond Supervised Continual Learning: a Review](https://mlsanity.com/paper/2208.14307/) (2022)
- [Not All Tasks Are Equally Difficult: Multi-Task Deep Reinforcement Learning with Dynamic Depth Routing](https://mlsanity.com/paper/2312.14472/) (2023)
- [QMP: Q-switch Mixture of Policies for Multi-Task Behavior Sharing](https://mlsanity.com/paper/2302.00671/) (2023)
- [Efficient Bayesian Policy Reuse with a Scalable Observation Model in Deep Reinforcement Learning](https://mlsanity.com/paper/2204.07729/) (2022)
- [On the Convergence Theory of Meta Reinforcement Learning with Personalized Policies](https://mlsanity.com/paper/2209.10072/) (2022)
- [Meta Generative Flow Networks with Personalization for Task-Specific Adaptation](https://mlsanity.com/paper/2306.09742/) (2023)
- [Projected Task-Specific Layers for Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2309.08776/) (2023)
- [Bidirectional Progressive Neural Networks with Episodic Return Progress for Emergent Task Sequencing and Robotic Skill Transfer](https://mlsanity.com/paper/2403.04001/) (2024)
- [Centralized Reward Agent for Knowledge Sharing and Transfer in Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2408.10858/) (2024)
- [Mixture-of-World Models: Scaling Multi-Task Reinforcement Learning with Modular Latent Dynamics](https://mlsanity.com/paper/2602.01270/) (2026)
- [Canoe : A System for Collaborative Learning for Neural Nets](https://mlsanity.com/paper/2108.12124/) (2021)
- [VAO: Validation-Aligned Optimization for Cross-Task Generative Auto-Bidding](https://mlsanity.com/paper/2510.07760/) (2025)
- [Multi-action Tangled Program Graphs for Multi-task Reinforcement Learning with Continuous Control](https://mlsanity.com/paper/2604.25369/) (2026)
- [Out-of-Distribution Generalisation with Sequence Models in Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2609.03667/) (2026)

## Abstract (arXiv)

While Deep Reinforcement Learning (DRL) has emerged as a promising approach to many complex tasks, it remains challenging to train a single DRL agent that is capable of undertaking multiple different continuous control tasks. In this paper, we present a Knowledge Transfer based Multi-task Deep Reinforcement Learning framework (KTM-DRL) for continuous control, which enables a single DRL agent to achieve expert-level performance in multiple different tasks by learning from task-specific teachers. In KTM-DRL, the multi-task agent first leverages an offline knowledge transfer algorithm designed particularly for the actor-critic architecture to quickly learn a control policy from the experience of task-specific teachers, and then it employs an online learning algorithm to further improve itself by learning from new online transition samples under the guidance of those teachers. We perform a comprehensive empirical study with two commonly-used benchmarks in the MuJoCo continuous control task suite. The experimental results well justify the effectiveness of KTM-DRL and its knowledge transfer and online learning algorithms, as well as its superiority over the state-of-the-art by a large margin.