# Model Primitive Hierarchical Lifelong Reinforcement Learning

arXiv: https://arxiv.org/abs/1903.01567 · HTML: https://mlsanity.com/paper/1903.01567/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2019.

## Built on by (most cited first, 5 of 5)

- [A Survey on Interpretable Reinforcement Learning](https://mlsanity.com/paper/2112.13112/) (2021)
- [Intelligent problem-solving as integrated hierarchical reinforcement learning](https://mlsanity.com/paper/2208.08731/) (2022)
- [Structure in Deep Reinforcement Learning: A Survey and Open Problems](https://mlsanity.com/paper/2306.16021/) (2023)
- [EUCLID: Towards Efficient Unsupervised Reinforcement Learning with Multi-choice Dynamics Model](https://mlsanity.com/paper/2210.00498/) (2022)
- [From Lab to Pocket: A Novel Continual Learning-based Mobile Application for Screening COVID-19](https://mlsanity.com/paper/2410.12589/) (2024)

## Abstract (arXiv)

Learning interpretable and transferable subpolicies and performing task decomposition from a single, complex task is difficult. Some traditional hierarchical reinforcement learning techniques enforce this decomposition in a top-down manner, while meta-learning techniques require a task distribution at hand to learn such decompositions. This paper presents a framework for using diverse suboptimal world models to decompose complex task solutions into simpler modular subpolicies. This framework performs automatic decomposition of a single source task in a bottom up manner, concurrently learning the required modular subpolicies as well as a controller to coordinate them. We perform a series of experiments on high dimensional continuous action control tasks to demonstrate the effectiveness of this approach at both complex single task learning and lifelong learning. Finally, we perform ablation studies to understand the importance and robustness of different elements in the framework and limitations to this approach.