# Sharing Knowledge in Multi-Task Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2401.09561 · HTML: https://mlsanity.com/paper/2401.09561/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 38 references that are in the map.
- First posted on arXiv 17 Jan 2024.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 14 of 14)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [Deep Exploration via Bootstrapped DQN](https://mlsanity.com/paper/1602.04621/) (2016)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [Distral: Robust Multitask Reinforcement Learning](https://mlsanity.com/paper/1707.04175/) (2017)
- [The Benefit of Multitask Representation Learning](https://mlsanity.com/paper/1505.06279/) (2015)
- [Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning](https://mlsanity.com/paper/1511.06342/) (2015)
- [DARLA: Improving Zero-Shot Transfer in Reinforcement Learning](https://mlsanity.com/paper/1707.08475/) (2017)
- [Multi-task Deep Reinforcement Learning with PopArt](https://mlsanity.com/paper/1809.04474/) (2018)
- [Sample Complexity of Multi-task Reinforcement Learning](https://mlsanity.com/paper/1309.6821/) (2013)
- [MushroomRL: Simplifying Reinforcement Learning Research](https://mlsanity.com/paper/2001.01102/) (2020)
- [A chain rule for the expected suprema of Gaussian processes](https://mlsanity.com/paper/1411.2635/) (2014)
- [Compositional Transfer in Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1906.11228/) (2019)

## Built on by (most cited first, 30 of 44)

- [A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications](https://mlsanity.com/paper/2503.07137/) (2025)
- [R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO](https://mlsanity.com/paper/2505.16673/) (2025)
- [One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotion](https://mlsanity.com/paper/2409.06366/) (2024)
- [Multi-Task Reinforcement Learning for Quadrotors](https://mlsanity.com/paper/2412.12442/) (2024)
- [AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers](https://mlsanity.com/paper/2411.11188/) (2024)
- [Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformer](https://mlsanity.com/paper/2505.24378/) (2025)
- [Offline Multitask Representation Learning for Reinforcement Learning](https://mlsanity.com/paper/2403.11574/) (2024)
- [HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning](https://mlsanity.com/paper/2405.18080/) (2024)
- [Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL](https://mlsanity.com/paper/2405.18520/) (2024)
- [Fast and Sample Efficient Multi-Task Representation Learning in Stochastic Contextual Bandits](https://mlsanity.com/paper/2410.02068/) (2024)
- [A Behavior-Aware Approach for Deep Reinforcement Learning in Non-stationary Environments without Known Change Points](https://mlsanity.com/paper/2405.14214/) (2024)
- [LIMT: Language-Informed Multi-Task Visual World Models](https://mlsanity.com/paper/2407.13466/) (2024)
- [Trajectory World Models for Heterogeneous Environments](https://mlsanity.com/paper/2502.01366/) (2025)
- [One Model for All Tasks: Leveraging Efficient World Models in Multi-Task Planning](https://mlsanity.com/paper/2509.07945/) (2025)
- [Subequivariant Reinforcement Learning in 3D Multi-Entity Physical Environments](https://mlsanity.com/paper/2407.12505/) (2024)
- [Enabling Energy-Efficient Simultaneous Multi-Task Reinforcement Learning through Spiking Neural Networks with Active Dendrites for Bio-inspired Generalist Agents](https://mlsanity.com/paper/2412.04847/) (2024)
- [Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2507.06628/) (2025)
- [Sparsity-Driven Plasticity in Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2508.06871/) (2025)
- [Prompt and Parameter Co-Optimization for Large Language Models](https://mlsanity.com/paper/2509.24245/) (2025)
- [Clustering-Based Weight Orthogonalization for Stabilizing Deep Reinforcement Learning](https://mlsanity.com/paper/2511.11607/) (2025)
- [Prismatic World Model: Learning Compositional Dynamics for Planning in Hybrid Systems](https://mlsanity.com/paper/2512.08411/) (2025)
- [The Power of Active Multi-Task Learning in Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2405.11226/) (2024)
- [How to Solve Contextual Goal-Oriented Problems with Offline Datasets?](https://mlsanity.com/paper/2408.07753/) (2024)
- [Centralized Reward Agent for Knowledge Sharing and Transfer in Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2408.10858/) (2024)
- [Towards Understanding the Benefit of Multitask Representation Learning in Decision Process](https://mlsanity.com/paper/2503.00345/) (2025)
- [Multi-task Learning for Heterogeneous Multi-source Block-Wise Missing Data](https://mlsanity.com/paper/2505.24413/) (2025)
- [Policy Search, Retrieval, and Composition via Task Similarity in Collaborative Agentic Systems](https://mlsanity.com/paper/2506.05577/) (2025)
- [Simple Optimizers for Convex Aligned Multi-Objective Optimization](https://mlsanity.com/paper/2509.05811/) (2025)
- [Active Exploration in Bayesian Model-based Reinforcement Learning for Robot Manipulation](https://mlsanity.com/paper/2404.01867/) (2024)
- [Natural Policy Gradient and Actor Critic Methods for Constrained Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2405.02456/) (2024)

## Abstract (arXiv)

We study the benefit of sharing representations among tasks to enable the effective use of deep neural networks in Multi-Task Reinforcement Learning. We leverage the assumption that learning from different tasks, sharing common properties, is helpful to generalize the knowledge of them resulting in a more effective feature extraction compared to learning a single task. Intuitively, the resulting set of features offers performance benefits when used by Reinforcement Learning algorithms. We prove this by providing theoretical guarantees that highlight the conditions for which is convenient to share representations among tasks, extending the well-known finite-time bounds of Approximate Value-Iteration to the multi-task setting. In addition, we complement our analysis by proposing multi-task extensions of three Reinforcement Learning algorithms that we empirically evaluate on widely used Reinforcement Learning benchmarks showing significant improvements over the single-task counterparts in terms of sample efficiency and performance.