# How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition

arXiv: https://arxiv.org/abs/2207.07730 · HTML: https://mlsanity.com/paper/2207.07730/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 179 of its 249 references that are in the map.
- First posted on arXiv 15 Jul 2022.

## Builds on (most cited first, 30 of 179)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [Overcoming catastrophic forgetting with hard attention to the task](https://mlsanity.com/paper/1801.01423/) (2018)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [The Option-Critic Architecture](https://mlsanity.com/paper/1609.05140/) (2016)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [Neural Module Networks](https://mlsanity.com/paper/1511.02799/) (2015)
- [dm_control: Software and Tasks for Continuous Control](https://mlsanity.com/paper/2006.12983/) (2020)
- [Expert Gate: Lifelong Learning with a Network of Experts](https://mlsanity.com/paper/1611.06194/) (2016)
- [PathNet: Evolution Channels Gradient Descent in Super Neural Networks](https://mlsanity.com/paper/1701.08734/) (2017)

## Built on by (most cited first, 27 of 27)

- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [PIVOT: Prompting for Video Continual Learning](https://mlsanity.com/paper/2212.04842/) (2022)
- [Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges](https://mlsanity.com/paper/2311.18044/) (2023)
- [Embodied Lifelong Learning for Task and Motion Planning](https://mlsanity.com/paper/2307.06870/) (2023)
- [A Survey on Compositional Learning of AI Models: Theoretical and Experimental Practices](https://mlsanity.com/paper/2406.08787/) (2024)
- [Federated Continual Learning: Concepts, Challenges, and Solutions](https://mlsanity.com/paper/2502.07059/) (2025)
- [Continual Task Allocation in Meta-Policy Network via Sparse Prompting](https://mlsanity.com/paper/2305.18444/) (2023)
- [Self-Composing Policies for Scalable Continual Reinforcement Learning](https://mlsanity.com/paper/2506.14811/) (2025)
- [Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications](https://mlsanity.com/paper/2410.09218/) (2024)
- [Dynamic Mixture of Progressive Parameter-Efficient Expert Library for Lifelong Robot Learning](https://mlsanity.com/paper/2506.05985/) (2025)
- [DOCTOR: A Multi-Disease Detection Continual Learning Framework Based on Wearable Medical Sensors](https://mlsanity.com/paper/2305.05738/) (2023)
- [Robotic Manipulation Datasets for Offline Compositional Reinforcement Learning](https://mlsanity.com/paper/2307.07091/) (2023)
- [How Does RL Post-training Induce Skill Composition? A Case Study on Countdown](https://mlsanity.com/paper/2512.01775/) (2025)
- [Continual Visual Reinforcement Learning with A Life-Long World Model](https://mlsanity.com/paper/2303.06572/) (2023)
- [Studying Generalization on Memory-Based Methods in Continual Learning](https://mlsanity.com/paper/2306.09890/) (2023)
- [Composing Diffusion Policies for Few-shot Learning of Movement Trajectories](https://mlsanity.com/paper/2410.17479/) (2024)
- [Continual Diffuser (CoD): Mastering Continual Offline Reinforcement Learning with Experience Rehearsal](https://mlsanity.com/paper/2409.02512/) (2024)
- [CLIMB: Language-Guided Continual Learning for Task Planning with Iterative Model Building](https://mlsanity.com/paper/2410.13756/) (2024)
- [An Energy-Based Mechanism for Compositional Behavior](https://mlsanity.com/paper/2512.04745/) (2025)
- [MedCL-Bench: Benchmarking stability-efficiency trade-offs and scaling in biomedical continual learning](https://mlsanity.com/paper/2603.16738/) (2026)
- [Sparse Adapter Fusion for Continual Learning in NLP](https://mlsanity.com/paper/2602.02502/) (2026)
- [Modularity Trumps Invariance for Compositional Robustness](https://mlsanity.com/paper/2306.09005/) (2023)
- [PAGE: Domain-Incremental Adaptation with Past-Agnostic Generative Replay for Smart Healthcare](https://mlsanity.com/paper/2403.08197/) (2024)
- [Distributed Continual Learning](https://mlsanity.com/paper/2405.17466/) (2024)
- [Compositional Instruction Following with Language Models and Reinforcement Learning](https://mlsanity.com/paper/2501.12539/) (2025)
- [Neural network task specialization via domain constraining](https://mlsanity.com/paper/2504.19592/) (2025)

## Abstract (arXiv)

A major goal of artificial intelligence (AI) is to create an agent capable of acquiring a general understanding of the world. Such an agent would require the ability to continually accumulate and build upon its knowledge as it encounters new experiences. Lifelong or continual learning addresses this setting, whereby an agent faces a continual stream of problems and must strive to capture the knowledge necessary for solving each new task it encounters. If the agent is capable of accumulating knowledge in some form of compositional representation, it could then selectively reuse and combine relevant pieces of knowledge to construct novel solutions. Despite the intuitive appeal of this simple idea, the literatures on lifelong learning and compositional learning have proceeded largely separately. In an effort to promote developments that bridge between the two fields, this article surveys their respective research landscapes and discusses existing and future connections between them.