Data to 7 Oct 2026

Distral: Robust Multitask Reinforcement Learning

arXiv 1707.04175 ↗First posted 13 Jul 2017cs.LG stat.MLcited by 231

Not on the map. Why?

Abstract (arXiv)

Most deep reinforcement learning algorithms are data inefficient in complex and rich environments, limiting their applicability to many scenarios. One direction for improving data efficiency is multitask learning with shared neural network parameters, where efficiency may be improved through transfer across related tasks. In practice, however, this is not usually observed, because gradients from different tasks can interfere negatively, making learning unstable and sometimes even less data efficient. Another issue is the different reward schemes between tasks, which can easily lead to one task dominating the learning of a shared model. We propose a new approach for joint training of multiple tasks, which we refer to as Distral (Distill & transfer learning). Instead of sharing parameters between the different workers, we propose to share a "distilled" policy that captures common behaviour across tasks. Each worker is trained to solve its own task while constrained to stay close to the shared policy, while the shared policy is trained by distillation to be the centroid of all task policies. Both aspects of the learning process are derived by optimizing a joint objective function. We show that our approach supports efficient transfer on complex 3D environments, outperforming several related methods. Moreover, the proposed learning process is more robust and more stable---attributes that are critical in deep reinforcement learning.

Key facts. Cited by 231 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 2; 2021: 48; 2022: 57; 2023: 34; 2024: 27; 2025: 32; 2026: 30.

231 papers.

Sort:

By area:

2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
2022The Primacy Bias in Deep Reinforcement Learningcited by 242
2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 213
2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 124

38 more

2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 102
2023UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learningcited by 101
2023Confronting Reward Model Overoptimization with Constrained RLHFcited by 96
2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
2022Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trialscited by 78
2022Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizescited by 72
2023Independent Component Alignment for Multi-Task Learningcited by 69
2022In Defense of the Unitary Scalarization for Deep Multi-Task Learningcited by 65
2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
2022Federated Reinforcement Learning with Environment Heterogeneitycited by 61
2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
2021Robot Learning from Randomized Simulations: A Reviewcited by 56
2022Retrieval-Augmented Reinforcement Learningcited by 53
2021Temporal Abstraction in Reinforcement Learning with the Successor Representationcited by 52
2021Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learningcited by 49
2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 46
2021Near-optimal Representation Learning for Linear Bandits and Linear RLcited by 45
2021Bootstrapped Meta-Learningcited by 45
2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Expertscited by 45
2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 42
2024Sharing Knowledge in Multi-Task Deep Reinforcement Learningcited by 42
2024RLDG: Robotic Generalist Policy Distillation via Reinforcement Learningcited by 42
2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 40
2022Hypernetworks in Meta-Reinforcement Learningcited by 39
2021Interesting Object, Curious Agent: Learning Task-Agnostic Explorationcited by 38
2021Reinforcement Learning for Adaptive Mesh Refinementcited by 37
2022Skill-based Meta-Reinforcement Learningcited by 36
2023Scaling Pareto-Efficient Decision Making Via Offline Multi-Objective RLcited by 36
2021Efficient Transformers in Reinforcement Learning using Actor-Learner Distillationcited by 35
2021Lifelong Robotic Reinforcement Learning by Retaining Experiencescited by 34
2021On the Power of Multitask Representation Learning in Linear MDPcited by 30
2021Universal Trading for Order Execution with Oracle Policy Distillationcited by 29
2022Provable Benefit of Multitask Representation Learning in Reinforcement Learningcited by 29
2022Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learningcited by 28
2022Towards Versatile Embodied Navigationcited by 26
2021Block Contextual MDPs for Continual Learningcited by 25

50 most cited of 231 All 231 (JSON)