# A Model-based Approach for Sample-efficient Multi-task Reinforcement Learning

arXiv: https://arxiv.org/abs/1907.04964 · HTML: https://mlsanity.com/paper/1907.04964/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2019.

## Built on by (most cited first, 12 of 12)

- [Supervised Pretraining Can Learn In-Context Reinforcement Learning](https://mlsanity.com/paper/2306.14892/) (2023)
- [Pareto Set Learning for Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2501.06773/) (2025)
- [Sequoia: A Software Framework to Unify Continual Learning Research](https://mlsanity.com/paper/2108.01005/) (2021)
- [Provably Efficient Multi-Task Reinforcement Learning with Model Transfer](https://mlsanity.com/paper/2107.08622/) (2021)
- [On the link between conscious function and general intelligence in humans and machines](https://mlsanity.com/paper/2204.05133/) (2022)
- [Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning](https://mlsanity.com/paper/2406.05064/) (2024)
- [Fractional Transfer Learning for Deep Model-Based Reinforcement Learning](https://mlsanity.com/paper/2108.06526/) (2021)
- [Multi-Source Transfer Learning for Deep Model-Based Reinforcement Learning](https://mlsanity.com/paper/2205.14410/) (2022)
- [On The Transferability of Deep-Q Networks](https://mlsanity.com/paper/2110.02639/) (2021)
- [Robot in a China Shop: Using Reinforcement Learning for Location-Specific Navigation Behaviour](https://mlsanity.com/paper/2106.01434/) (2021)
- [Uncertainty-aware transfer across tasks using hybrid model-based successor feature reinforcement learning](https://mlsanity.com/paper/2310.10818/) (2023)
- [Efficient Open-world Reinforcement Learning via Knowledge Distillation and Autonomous Rule Discovery](https://mlsanity.com/paper/2311.14270/) (2023)

## Abstract (arXiv)

The aim of multi-task reinforcement learning is two-fold: (1) efficiently learn by training against multiple tasks and (2) quickly adapt, using limited samples, to a variety of new tasks. In this work, the tasks correspond to reward functions for environments with the same (or similar) dynamical models. We propose to learn a dynamical model during the training process and use this model to perform sample-efficient adaptation to new tasks at test time. We use significantly fewer samples by performing policy optimization only in a "virtual" environment whose transitions are given by our learned dynamical model. Our algorithm sequentially trains against several tasks. Upon encountering a new task, we first warm-up a policy on our learned dynamical model, which requires no new samples from the environment. We then adapt the dynamical model with samples from this policy in the real environment. We evaluate our approach on several continuous control benchmarks and demonstrate its efficacy over MAML, a state-of-the-art meta-learning algorithm, on these tasks.