# Intrinsically Motivated Open-Ended Multi-Task Learning Using Transfer Learning to Discover Task Hierarchy

arXiv: https://arxiv.org/abs/2102.09854 · HTML: https://mlsanity.com/paper/2102.09854/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 56 references that are in the map.
- First posted on arXiv 19 Feb 2021.

## Builds on (most cited first, 11 of 11)

- [Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation](https://mlsanity.com/paper/1604.06057/) (2016)
- [Learning by Playing - Solving Sparse Reward Tasks from Scratch](https://mlsanity.com/paper/1802.10567/) (2018)
- [Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning](https://mlsanity.com/paper/1708.02190/) (2017)
- [CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning](https://mlsanity.com/paper/1810.06284/) (2018)
- [Interactive Policy Learning through Confidence-Based Autonomy](https://mlsanity.com/paper/1401.3439/) (2014)
- [CLIC: Curriculum Learning and Imitation for object Control in non-rewarding environments](https://mlsanity.com/paper/1901.09720/) (2019)
- [Action Representations in Robotics: A Taxonomy and Systematic Classification](https://mlsanity.com/paper/1809.04317/) (2018)
- [Hierarchical Affordance Discovery using Intrinsic Motivation](https://mlsanity.com/paper/2009.10968/) (2020)
- [Active choice of teachers, learning strategies and goals for a socially guided intrinsic motivation learner](https://mlsanity.com/paper/1804.06819/) (2018)
- [Socially Guided Intrinsic Motivation for Robot Learning of Motor Skills](https://mlsanity.com/paper/1804.07269/) (2018)
- [Learning a Set of Interrelated Tasks by Using Sequences of Motor Policies for a Strategic Intrinsically Motivated Learner](https://mlsanity.com/paper/1810.04877/) (2018)

## Built on by (most cited first, 4 of 4)

- [A Survey of Human Activity Recognition in Smart Homes Based on IoT Sensors Algorithms: Taxonomies, Challenges, and Opportunities with Deep Learning](https://mlsanity.com/paper/2111.04418/) (2021)
- [Multi-task Hierarchical Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/2305.12633/) (2023)
- [Robots Learn Increasingly Complex Tasks with Intrinsic Motivation and Automatic Curriculum Learning](https://mlsanity.com/paper/2202.10222/) (2022)
- [HERAKLES: Hierarchical Skill Compilation for Open-ended LLM Agents](https://mlsanity.com/paper/2508.14751/) (2025)

## Abstract (arXiv)

In open-ended continuous environments, robots need to learn multiple parameterised control tasks in hierarchical reinforcement learning. We hypothesise that the most complex tasks can be learned more easily by transferring knowledge from simpler tasks, and faster by adapting the complexity of the actions to the task. We propose a task-oriented representation of complex actions, called procedures, to learn online task relationships and unbounded sequences of action primitives to control the different observables of the environment. Combining both goal-babbling with imitation learning, and active learning with transfer of knowledge based on intrinsic motivation, our algorithm self-organises its learning process. It chooses at any given time a task to focus on; and what, how, when and from whom to transfer knowledge. We show with a simulation and a real industrial robot arm, in cross-task and cross-learner transfer settings, that task composition is key to tackle highly complex tasks. Task decomposition is also efficiently transferred across different embodied learners and by active imitation, where the robot requests just a small amount of demonstrations and the adequate type of information. The robot learns and exploits task dependencies so as to learn tasks of every complexity.