Lifelong Learning Metrics
Not on the map. Why?
Abstract (arXiv)
The DARPA Lifelong Learning Machines (L2M) program seeks to yield advances in artificial intelligence (AI) systems so that they are capable of learning (and improving) continuously, leveraging data on one task to improve performance on another, and doing so in a computationally sustainable way. Performers on this program developed systems capable of performing a diverse range of functions, including autonomous driving, real-time strategy, and drone simulation. These systems featured a diverse range of characteristics (e.g., task structure, lifetime duration), and an immediate challenge faced by the program's testing and evaluation team was measuring system performance across these different settings. This document, developed in close collaboration with DARPA and the program performers, outlines a formalism for constructing and characterizing the performance of agents performing lifelong learning scenarios.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 19 of its 28 references that are in the map. First posted on arXiv 20 Jan 2022.
Builds on
19 of 28 references.
References · 19
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
- 2016Progressive Neural Networkscited by 1,332
- 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
- 2018Experience Replay for Continual Learningcited by 1,074
- 2019Three scenarios for continual learningcited by 610
- 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
- 2017CORe50: a New Dataset and Benchmark for Continuous Object Recognitioncited by 287
- 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
- 2018Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselinescited by 228
- 2018Towards Robust Evaluations of Continual Learningcited by 148
- 2021Avalanche: an End-to-End Library for Continual Learningcited by 124
7 more
- 2020Continual Deep Learning by Functional Regularisation of Memorable Pastcited by 104
- 2019Uncertainty-guided Continual Learning with Bayesian Neural Networkscited by 100
- 2021Continual Learning of a Mixed Sequence of Similar and Dissimilar Taskscited by 80
- 2020La-MAML: Look-ahead Meta Learning for Continual Learningcited by 51
- 2020Meta-Consolidation for Continual Learningcited by 40
- 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 36
- 2020Lifelong Policy Gradient Learning of Factored Policies for Faster Training Without Forgettingcited by 29
Built on by
Citing papers per year: 2022: 7; 2023: 3; 2024: 1; 2025: 1; 2026: 1.
13 papers.
Sort:
- 2024Continual Learning and Catastrophic Forgettingcited by 73
- 2022Lifelong Reinforcement Learning with Modulating Maskscited by 20
- 2022A Theory for Knowledge Transfer in Continual Learningcited by 16
- 2023A Domain-Agnostic Approach for Characterization of Lifelong Learning Systemscited by 13
- 2022Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-2cited by 11
- 2023Sharing Lifelong Reinforcement Learning Knowledge via Modulating Maskscited by 10
- 2023Design Principles for Lifelong Learning AI Acceleratorscited by 9
- 2022L2Explorer: A Lifelong Reinforcement Learning Assessment Environmentcited by 6
- 2022Exploring the Intersection between Neural Architecture Search and Continual Learningcited by 4
- 2022Latent Properties of Lifelong Learning Systemscited by 1
- 2022Continual Reinforcement Learning with TELLAcited by 1
- 2025Noise-Tolerant Coreset-Based Class Incremental Continual Learningcited by 0
- 2026Shortcut Solutions Learned by Transformers Impair Continual Compositional Reasoning6 May 2026
- 2025Noise-Tolerant Coreset-Based Class Incremental Continual Learning23 Apr 2025
- 2024Continual Learning and Catastrophic Forgetting8 Mar 2024
- 2023Design Principles for Lifelong Learning AI Accelerators5 Oct 2023
- 2023Sharing Lifelong Reinforcement Learning Knowledge via Modulating Masks18 May 2023
- 2023A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems18 Jan 2023
- 2022Lifelong Reinforcement Learning with Modulating Masks21 Dec 2022
- 2022A Theory for Knowledge Transfer in Continual Learning14 Aug 2022
- 2022Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-29 Aug 2022
- 2022Continual Reinforcement Learning with TELLA8 Aug 2022
- 2022Latent Properties of Lifelong Learning Systems28 Jul 2022
- 2022Exploring the Intersection between Neural Architecture Search and Continual Learning11 Jun 2022
1 more
- 2024Continual Learning and Catastrophic Forgettingcited by 73
- 2022Lifelong Reinforcement Learning with Modulating Maskscited by 20
- 2022A Theory for Knowledge Transfer in Continual Learningcited by 16
- 2023A Domain-Agnostic Approach for Characterization of Lifelong Learning Systemscited by 13
- 2022Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-2cited by 11
- 2023Sharing Lifelong Reinforcement Learning Knowledge via Modulating Maskscited by 10
- 2023Design Principles for Lifelong Learning AI Acceleratorscited by 9
- 2022L2Explorer: A Lifelong Reinforcement Learning Assessment Environmentcited by 6
- 2022Exploring the Intersection between Neural Architecture Search and Continual Learningcited by 4
- 2022Latent Properties of Lifelong Learning Systemscited by 1
- 2022Continual Reinforcement Learning with TELLAcited by 1
- 2025Noise-Tolerant Coreset-Based Class Incremental Continual Learningcited by 0
12 most cited of 13 All 13 (JSON)
- 2026Shortcut Solutions Learned by Transformers Impair Continual Compositional Reasoning6 May 2026
- 2025Noise-Tolerant Coreset-Based Class Incremental Continual Learning23 Apr 2025
- 2024Continual Learning and Catastrophic Forgetting8 Mar 2024
- 2023Design Principles for Lifelong Learning AI Accelerators5 Oct 2023
- 2023Sharing Lifelong Reinforcement Learning Knowledge via Modulating Masks18 May 2023
- 2023A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems18 Jan 2023
- 2022Lifelong Reinforcement Learning with Modulating Masks21 Dec 2022
- 2022A Theory for Knowledge Transfer in Continual Learning14 Aug 2022
- 2022Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-29 Aug 2022
- 2022Continual Reinforcement Learning with TELLA8 Aug 2022
- 2022Latent Properties of Lifelong Learning Systems28 Jul 2022
- 2022Exploring the Intersection between Neural Architecture Search and Continual Learning11 Jun 2022
12 newest of 13 All 13 (JSON)