# Generalizing to New Tasks via One-Shot Compositional Subgoals

arXiv: https://arxiv.org/abs/2205.07716 · HTML: https://mlsanity.com/paper/2205.07716/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 16 May 2022.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 24 of 24)

- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning](https://mlsanity.com/paper/1910.11956/) (2019)
- [Learning Latent Plans from Play](https://mlsanity.com/paper/1903.01973/) (2019)
- [Semi-parametric Topological Memory for Navigation](https://mlsanity.com/paper/1803.00653/) (2018)
- [Multi-Agent Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1807.09936/) (2018)
- [Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning](https://mlsanity.com/paper/1706.05064/) (2017)
- [Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction](https://mlsanity.com/paper/1703.01030/) (2017)
- [IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data](https://mlsanity.com/paper/1911.05321/) (2019)
- [Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning](https://mlsanity.com/paper/1712.07294/) (2017)
- [TACO: Learning Task Decomposition via Temporal Alignment for Control](https://mlsanity.com/paper/1803.01840/) (2018)
- [Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2006.11485/) (2020)
- [Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2110.13625/) (2021)
- [Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learning](https://mlsanity.com/paper/1805.11240/) (2018)
- [Interactive Policy Learning through Confidence-Based Autonomy](https://mlsanity.com/paper/1401.3439/) (2014)
- [Learning from Trajectories via Subgoal Discovery](https://mlsanity.com/paper/1911.07224/) (2019)
- [Fast Policy Learning through Imitation and Reinforcement](https://mlsanity.com/paper/1805.10413/) (2018)
- [Hallucinative Topological Memory for Zero-Shot Visual Planning](https://mlsanity.com/paper/2002.12336/) (2020)
- [Subgoal Search For Complex Reasoning Tasks](https://mlsanity.com/paper/2108.11204/) (2021)
- [Intention-Net: Integrating Planning and Deep Learning for Goal-Directed Autonomous Navigation](https://mlsanity.com/paper/1710.05627/) (2017)
- [Active Hierarchical Exploration with Stable Subgoal Representation Learning](https://mlsanity.com/paper/2105.14750/) (2021)
- [SKILL-IL: Disentangling Skill and Knowledge in Multitask Imitation Learning](https://mlsanity.com/paper/2205.03130/) (2022)
- [Robot in a China Shop: Using Reinforcement Learning for Location-Specific Navigation Behaviour](https://mlsanity.com/paper/2106.01434/) (2021)

## Built on by (most cited first, 1 of 1)

- [Multi-Domain Learning with Global Expert Mapping](https://mlsanity.com/paper/2604.18842/) (2026)

## Abstract (arXiv)

The ability to generalize to previously unseen tasks with little to no supervision is a key challenge in modern machine learning research. It is also a cornerstone of a future "General AI". Any artificially intelligent agent deployed in a real world application, must adapt on the fly to unknown environments. Researchers often rely on reinforcement and imitation learning to provide online adaptation to new tasks, through trial and error learning. However, this can be challenging for complex tasks which require many timesteps or large numbers of subtasks to complete. These "long horizon" tasks suffer from sample inefficiency and can require extremely long training times before the agent can learn to perform the necessary longterm planning. In this work, we introduce CASE which attempts to address these issues by training an Imitation Learning agent using adaptive "near future" subgoals. These subgoals are recalculated at each step using compositional arithmetic in a learned latent representation space. In addition to improving learning efficiency for standard long-term tasks, this approach also makes it possible to perform one-shot generalization to previously unseen tasks, given only a single reference trajectory for the task in a different environment. Our experiments show that the proposed approach consistently outperforms the previous state-of-the-art compositional Imitation Learning approach by 30%.