# Incremental Learning of Retrievable Skills For Efficient Continual Task Adaptation

arXiv: https://arxiv.org/abs/2410.22658 · HTML: https://mlsanity.com/paper/2410.22658/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 36 of its 42 references that are in the map.
- First posted on arXiv 30 Oct 2024.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 30 of 36)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning](https://mlsanity.com/paper/2204.04799/) (2022)
- [Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning](https://mlsanity.com/paper/2208.06193/) (2022)
- [CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning](https://mlsanity.com/paper/2211.13218/) (2022)
- [Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning](https://mlsanity.com/paper/1910.11956/) (2019)
- [Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1904.00310/) (2019)
- [Accelerating Reinforcement Learning with Learned Skill Priors](https://mlsanity.com/paper/2010.11944/) (2020)
- [Multi-Game Decision Transformers](https://mlsanity.com/paper/2205.15241/) (2022)
- [Continual Unsupervised Representation Learning](https://mlsanity.com/paper/1910.14481/) (2019)
- [Continual Learning and Private Unlearning](https://mlsanity.com/paper/2203.12817/) (2022)
- [A Unified Continual Learning Framework with General Parameter-Efficient Tuning](https://mlsanity.com/paper/2303.10070/) (2023)
- [Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation](https://mlsanity.com/paper/2109.01115/) (2021)
- [Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability](https://mlsanity.com/paper/2107.06277/) (2021)
- [Learning and Retrieval from Prior Data for Skill-based Imitation Learning](https://mlsanity.com/paper/2210.11435/) (2022)
- [Data Quality in Imitation Learning](https://mlsanity.com/paper/2306.02437/) (2023)
- [LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discovery](https://mlsanity.com/paper/2311.02058/) (2023)
- [TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models](https://mlsanity.com/paper/2310.05905/) (2023)
- [Controllability-Aware Unsupervised Skill Discovery](https://mlsanity.com/paper/2302.05103/) (2023)
- [Hierarchical Few-Shot Imitation with Skill Transition Models](https://mlsanity.com/paper/2107.08981/) (2021)
- [LISA: Learning Interpretable Skill Abstractions from Language](https://mlsanity.com/paper/2203.00054/) (2022)
- [Continual Learning with Low Rank Adaptation](https://mlsanity.com/paper/2311.17601/) (2023)

## Built on by (most cited first, 19 of 19)

- [AtomicVLA: Unlocking the Potential of Atomic Skill Learning in Robots](https://mlsanity.com/paper/2603.07648/) (2026)
- [NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domains](https://mlsanity.com/paper/2503.00870/) (2025)
- [Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuning](https://mlsanity.com/paper/2602.10503/) (2026)
- [Continual Learning for Generative AI: From LLMs to MLLMs and Beyond](https://mlsanity.com/paper/2506.13045/) (2025)
- [When Continue Learning Meets Multimodal Large Language Model: A Survey](https://mlsanity.com/paper/2503.01887/) (2025)
- [SPECI: Skill Prompts based Hierarchical Continual Imitation Learning for Robot Manipulation](https://mlsanity.com/paper/2504.15561/) (2025)
- [Dynamic Mixture of Progressive Parameter-Efficient Expert Library for Lifelong Robot Learning](https://mlsanity.com/paper/2506.05985/) (2025)
- [Temporal Distance-aware Transition Augmentation for Offline Model-based Reinforcement Learning](https://mlsanity.com/paper/2505.13144/) (2025)
- [Do Not Mimic My Voice: Speaker Identity Unlearning for Zero-Shot Text-to-Speech](https://mlsanity.com/paper/2507.20140/) (2025)
- [DreamProver: Evolving Transferable Lemma Libraries via a Wake-Sleep Theorem-Proving Agent](https://mlsanity.com/paper/2604.26311/) (2026)
- [Policy Compatible Skill Incremental Learning via Lazy Learning Interface](https://mlsanity.com/paper/2509.20612/) (2025)
- [In Praise of Stubbornness: An Empirical Case for Cognitive-Dissonance Aware Continual Update of Knowledge in LLMs](https://mlsanity.com/paper/2502.04390/) (2025)
- [Lifelong Imitation Learning with Multimodal Latent Replay and Incremental Adjustment](https://mlsanity.com/paper/2603.10929/) (2026)
- [Learning New Tasks via Reusable Skills: Skill-Compositional Experts for Embodied Continual Learning](https://mlsanity.com/paper/2606.15685/) (2026)
- [LiMoDE: Rethinking Lifelong Robot Manipulation from a Mixture-of-Dynamic-Experts Perspective](https://mlsanity.com/paper/2606.26183/) (2026)
- [Joint Flashback Adaptation for Forgetting-Resistant Instruction Tuning](https://mlsanity.com/paper/2505.15467/) (2025)
- [ContiGuard: A Framework for Continual Toxicity Detection Against Evolving Evasive Perturbations](https://mlsanity.com/paper/2603.14843/) (2026)
- [Efficient Skill Grounding via Code Refactoring with Small Language Models](https://mlsanity.com/paper/2606.07999/) (2026)
- [Does Continual Imitation Learning Remain Grounded? A Language-Perturbed Benchmark for Robotic Task Retention](https://mlsanity.com/paper/2610.00542/) (2026)

## Abstract (arXiv)

Continual Imitation Learning (CiL) involves extracting and accumulating task knowledge from demonstrations across multiple stages and tasks to achieve a multi-task policy. With recent advancements in foundation models, there has been a growing interest in adapter-based CiL approaches, where adapters are established parameter-efficiently for tasks newly demonstrated. While these approaches isolate parameters for specific tasks and tend to mitigate catastrophic forgetting, they limit knowledge sharing among different demonstrations. We introduce IsCiL, an adapter-based CiL framework that addresses this limitation of knowledge sharing by incrementally learning shareable skills from different demonstrations, thus enabling sample-efficient task adaptation using the skills particularly in non-stationary CiL environments. In IsCiL, demonstrations are mapped into the state embedding space, where proper skills can be retrieved upon input states through prototype-based memory. These retrievable skills are incrementally learned on their corresponding adapters. Our CiL experiments with complex tasks in Franka-Kitchen and Meta-World demonstrate robust performance of IsCiL in both task adaptation and sample-efficiency. We also show a simple extension of IsCiL for task unlearning scenarios.