# Lifelong Inverse Reinforcement Learning

arXiv: https://arxiv.org/abs/2207.00461 · HTML: https://mlsanity.com/paper/2207.00461/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 4 of its 43 references that are in the map.
- First posted on arXiv 1 Jul 2022.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 4 of 4)

- [One-Shot Imitation Learning](https://mlsanity.com/paper/1703.07326/) (2017)
- [One-Shot Visual Imitation Learning via Meta-Learning](https://mlsanity.com/paper/1709.04905/) (2017)
- [Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1507.04888/) (2015)
- [Generalizing Skills with Semi-Supervised Reinforcement Learning](https://mlsanity.com/paper/1612.00429/) (2016)

## Built on by (most cited first, 9 of 9)

- [Online Continual Learning For Interactive Instruction Following Agents](https://mlsanity.com/paper/2403.07548/) (2024)
- [A Survey of Continual Reinforcement Learning](https://mlsanity.com/paper/2506.21872/) (2025)
- [Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations](https://mlsanity.com/paper/2209.11908/) (2022)
- [Continual Learning for Generative AI: From LLMs to MLLMs and Beyond](https://mlsanity.com/paper/2506.13045/) (2025)
- [Hierarchical-Task-Aware Multi-modal Mixture of Incremental LoRA Experts for Embodied Continual Learning](https://mlsanity.com/paper/2506.04595/) (2025)
- [Inversely Learning Transferable Rewards via Abstracted States](https://mlsanity.com/paper/2501.01669/) (2025)
- [ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events](https://mlsanity.com/paper/2308.14550/) (2023)
- [Inverse Reinforcement Learning with Multiple Planning Horizons](https://mlsanity.com/paper/2409.18051/) (2024)
- [Supervised Reward Inference](https://mlsanity.com/paper/2502.18447/) (2025)

## Abstract (arXiv)

Methods for learning from demonstration (LfD) have shown success in acquiring behavior policies by imitating a user. However, even for a single task, LfD may require numerous demonstrations. For versatile agents that must learn many tasks via demonstration, this process would substantially burden the user if each task were learned in isolation. To address this challenge, we introduce the novel problem of lifelong learning from demonstration, which allows the agent to continually build upon knowledge learned from previously demonstrated tasks to accelerate the learning of new tasks, reducing the amount of demonstrations required. As one solution to this problem, we propose the first lifelong learning approach to inverse reinforcement learning, which learns consecutive tasks via demonstration, continually transferring knowledge between tasks to improve performance.