# Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations

arXiv: https://arxiv.org/abs/2209.11908 · HTML: https://mlsanity.com/paper/2209.11908/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 19 of its 67 references that are in the map.
- First posted on arXiv 24 Sep 2022.

## Placed in

- Offline RL and imitation learning from data / [Robot learning from demonstration](https://mlsanity.com/offlinerl/robot-learning-from-demonstration/)

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://mlsanity.com/paper/2202.02005/) (2022)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation](https://mlsanity.com/paper/1811.02790/) (2018)
- [Learning from Suboptimal Demonstration via Self-Supervised Reward Regression](https://mlsanity.com/paper/2010.11723/) (2020)
- [f-IRL: Inverse Reinforcement Learning via State Marginal Matching](https://mlsanity.com/paper/2011.04709/) (2020)
- [Efficient Model Learning for Human-Robot Collaborative Tasks](https://mlsanity.com/paper/1405.6341/) (2014)
- [Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillation](https://mlsanity.com/paper/2001.00503/) (2020)
- [Robotic Table Tennis with Model-Free Reinforcement Learning](https://mlsanity.com/paper/2003.14398/) (2020)
- [Interpretable and Personalized Apprenticeship Scheduling: Learning Interpretable Scheduling Policies from Heterogeneous User Demonstrations](https://mlsanity.com/paper/1906.06397/) (2019)
- [Iterated Reasoning with Mutual Information in Cooperative and Byzantine Decentralized Teaming](https://mlsanity.com/paper/2201.08484/) (2022)
- [Lifelong Inverse Reinforcement Learning](https://mlsanity.com/paper/2207.00461/) (2022)
- [Deep Adaptive Multi-Intention Inverse Reinforcement Learning](https://mlsanity.com/paper/2107.06692/) (2021)
- [Inverse Reinforce Learning with Nonparametric Behavior Clustering](https://mlsanity.com/paper/1712.05514/) (2017)

## Built on by (most cited first, 14 of 14)

- [LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discovery](https://mlsanity.com/paper/2311.02058/) (2023)
- [A Survey of Continual Reinforcement Learning](https://mlsanity.com/paper/2506.21872/) (2025)
- [Continual Learning for Generative AI: From LLMs to MLLMs and Beyond](https://mlsanity.com/paper/2506.13045/) (2025)
- [Inverse Reinforcement Learning by Estimating Expertise of Demonstrators](https://mlsanity.com/paper/2402.01886/) (2024)
- [ELEMENTAL: Interactive Learning from Demonstrations and Vision-Language Models for Reward Design in Robotics](https://mlsanity.com/paper/2411.18825/) (2024)
- [Infer and Adapt: Bipedal Locomotion Reward Learning from Demonstrations via Inverse Reinforcement Learning](https://mlsanity.com/paper/2309.16074/) (2023)
- [Learning Diverse Robot Striking Motions with Diffusion Models and Kinematically Constrained Gradient Guidance](https://mlsanity.com/paper/2409.15528/) (2024)
- [Multi-Camera Asynchronous Ball Localization and Trajectory Prediction with Factor Graphs and Human Poses](https://mlsanity.com/paper/2401.17185/) (2024)
- [Towards Balancing Preference and Performance through Adaptive Personalized Explainability](https://mlsanity.com/paper/2504.13856/) (2025)
- [Automating Deformable Gasket Assembly](https://mlsanity.com/paper/2408.12593/) (2024)
- [Task-agnostic Lifelong Robot Learning with Retrieval-based Weighted Local Adaptation](https://mlsanity.com/paper/2410.02995/) (2024)
- [Stage-Transition Dense Reward Modeling for Reinforcement Learning](https://mlsanity.com/paper/2606.31377/) (2026)
- [ConTraIRL: Factorized Contrastive Abstractions for Transferable IRL](https://mlsanity.com/paper/2606.03017/) (2026)
- [Do as the Romans Do: Learning Universal Behaviors from Heterogeneous Agents](https://mlsanity.com/paper/2606.18537/) (2026)

## Abstract (arXiv)

Learning from Demonstration (LfD) approaches empower end-users to teach robots novel tasks via demonstrations of the desired behaviors, democratizing access to robotics. However, current LfD frameworks are not capable of fast adaptation to heterogeneous human demonstrations nor the large-scale deployment in ubiquitous robotics applications. In this paper, we propose a novel LfD framework, Fast Lifelong Adaptive Inverse Reinforcement learning (FLAIR). Our approach (1) leverages learned strategies to construct policy mixtures for fast adaptation to new demonstrations, allowing for quick end-user personalization, (2) distills common knowledge across demonstrations, achieving accurate task inference; and (3) expands its model only when needed in lifelong deployments, maintaining a concise set of prototypical strategies that can approximate all behaviors via policy mixtures. We empirically validate that FLAIR achieves adaptability (i.e., the robot adapts to heterogeneous, user-specific task preferences), efficiency (i.e., the robot achieves sample-efficient adaptation), and scalability (i.e., the model grows sublinearly with the number of demonstrations while maintaining high performance). FLAIR surpasses benchmarks across three control tasks with an average 57% improvement in policy returns and an average 78% fewer episodes required for demonstration modeling using policy mixtures. Finally, we demonstrate the success of FLAIR in a table tennis task and find users rate FLAIR as having higher task (p<.05) and personalization (p<.05) performance.