# Behavior Self-Organization Supports Task Inference for Continual Robot Learning

arXiv: https://arxiv.org/abs/2107.04533 · HTML: https://mlsanity.com/paper/2107.04533/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 9 Jul 2021.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables](https://mlsanity.com/paper/1903.08254/) (2019)
- [Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation](https://mlsanity.com/paper/1710.04615/) (2017)
- [Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours](https://mlsanity.com/paper/1509.06825/) (2015)
- [Task2Vec: Task Embedding for Meta-Learning](https://mlsanity.com/paper/1902.03545/) (2019)
- [Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation](https://mlsanity.com/paper/1707.03374/) (2017)
- [Imitation Learning from Imperfect Demonstration](https://mlsanity.com/paper/1901.09387/) (2019)
- [Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning](https://mlsanity.com/paper/1706.05064/) (2017)
- [Evolved Policy Gradients](https://mlsanity.com/paper/1802.04821/) (2018)
- [Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots](https://mlsanity.com/paper/1301.4862/) (2013)
- [Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning](https://mlsanity.com/paper/2003.01239/) (2020)
- [A Meta-MDP Approach to Exploration for Lifelong Reinforcement Learning](https://mlsanity.com/paper/1902.00843/) (2019)
- [Single Episode Policy Transfer in Reinforcement Learning](https://mlsanity.com/paper/1910.07719/) (2019)
- [Learning Context-aware Task Reasoning for Efficient Meta-reinforcement Learning](https://mlsanity.com/paper/2003.01373/) (2020)
- [On-Policy Robot Imitation Learning from a Converging Supervisor](https://mlsanity.com/paper/1907.03423/) (2019)
- [Autonomous Reinforcement Learning of Multiple Interrelated Tasks](https://mlsanity.com/paper/1906.01374/) (2019)

## Built on by (most cited first, 4 of 4)

- [A Survey of Continual Reinforcement Learning](https://mlsanity.com/paper/2506.21872/) (2025)
- [Map-based Experience Replay: A Memory-Efficient Solution to Catastrophic Forgetting in Reinforcement Learning](https://mlsanity.com/paper/2305.02054/) (2023)
- [Continual Robot Learning using Self-Supervised Task Inference](https://mlsanity.com/paper/2309.04974/) (2023)
- [Regime-Adaptive Continual Learning for Portfolio Management](https://mlsanity.com/paper/2606.00143/) (2026)

## Abstract (arXiv)

Recent advances in robot learning have enabled robots to become increasingly better at mastering a predefined set of tasks. On the other hand, as humans, we have the ability to learn a growing set of tasks over our lifetime. Continual robot learning is an emerging research direction with the goal of endowing robots with this ability. In order to learn new tasks over time, the robot first needs to infer the task at hand. Task inference, however, has received little attention in the multi-task learning literature. In this paper, we propose a novel approach to continual learning of robotic control tasks. Our approach performs unsupervised learning of behavior embeddings by incrementally self-organizing demonstrated behaviors. Task inference is made by finding the nearest behavior embedding to a demonstrated behavior, which is used together with the environment state as input to a multi-task policy trained with reinforcement learning to optimize performance over tasks. Unlike previous approaches, our approach makes no assumptions about task distribution and requires no task exploration to infer tasks. We evaluate our approach in experiments with concurrently and sequentially presented tasks and show that it outperforms other multi-task learning approaches in terms of generalization performance and convergence speed, particularly in the continual learning setting.