# Multitask Soft Option Learning

arXiv: https://arxiv.org/abs/1904.01033 · HTML: https://mlsanity.com/paper/1904.01033/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2019.

## Built on by (most cited first, 13 of 13)

- [Multi-Task Reinforcement Learning with Context-based Representations](https://mlsanity.com/paper/2102.06177/) (2021)
- [Learning to Modulate pre-trained Models in RL](https://mlsanity.com/paper/2306.14884/) (2023)
- [Block Contextual MDPs for Continual Learning](https://mlsanity.com/paper/2110.06972/) (2021)
- [Your Policy Regularizer is Secretly an Adversary](https://mlsanity.com/paper/2203.12592/) (2022)
- [A Mixture of Surprises for Unsupervised Reinforcement Learning](https://mlsanity.com/paper/2210.06702/) (2022)
- [Priors, Hierarchy, and Information Asymmetry for Skill Transfer in Reinforcement Learning](https://mlsanity.com/paper/2201.08115/) (2022)
- [Hierarchical Imitation Learning for Stochastic Environments](https://mlsanity.com/paper/2309.14003/) (2023)
- [Skills Regularized Task Decomposition for Multi-task Offline Reinforcement Learning](https://mlsanity.com/paper/2408.15593/) (2024)
- [InnateCoder: Learning Programmatic Options with Foundation Models](https://mlsanity.com/paper/2505.12508/) (2025)
- [PRIMA: Planner-Reasoner Inside a Multi-task Reasoning Agent](https://mlsanity.com/paper/2202.00531/) (2022)
- [Matching options to tasks using Option-Indexed Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2206.05750/) (2022)
- [Reusable Options through Gradient-based Meta Learning](https://mlsanity.com/paper/2212.11726/) (2022)
- [Unveiling Options with Neural Decomposition](https://mlsanity.com/paper/2410.11262/) (2024)

## Abstract (arXiv)

We present Multitask Soft Option Learning(MSOL), a hierarchical multitask framework based on Planning as Inference. MSOL extends the concept of options, using separate variational posteriors for each task, regularized by a shared prior. This ''soft'' version of options avoids several instabilities during training in a multitask setting, and provides a natural way to learn both intra-option policies and their terminations. Furthermore, it allows fine-tuning of options for new tasks without forgetting their learned policies, leading to faster training without reducing the expressiveness of the hierarchical policy. We demonstrate empirically that MSOL significantly outperforms both hierarchical and flat transfer-learning baselines.