# Don't forget, there is more than forgetting: new metrics for Continual Learning

arXiv: https://arxiv.org/abs/1810.13166 · HTML: https://mlsanity.com/paper/1810.13166/

## Key facts

- Cited by 115 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2018.

## Built on by (most cited first, 30 of 115)

- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Avalanche: an End-to-End Library for Continual Learning](https://mlsanity.com/paper/2104.00405/) (2021)
- [Continual World: A Robotic Benchmark For Continual Reinforcement Learning](https://mlsanity.com/paper/2105.10919/) (2021)
- [The CLEAR Benchmark: Continual LEArning on Real-World Imagery](https://mlsanity.com/paper/2201.06289/) (2022)
- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [AFEC: Active Forgetting of Negative Transfer in Continual Learning](https://mlsanity.com/paper/2110.12187/) (2021)
- [LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discovery](https://mlsanity.com/paper/2311.02058/) (2023)
- [Model Zoo: A Growing "Brain" That Learns Continually](https://mlsanity.com/paper/2106.03027/) (2021)
- [Create Your World: Lifelong Text-to-Image Diffusion](https://mlsanity.com/paper/2309.04430/) (2023)
- [CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents](https://mlsanity.com/paper/2110.10067/) (2021)
- [TiC-CLIP: Continual Training of CLIP Models](https://mlsanity.com/paper/2310.16226/) (2023)
- [Test-Time Training on Video Streams](https://mlsanity.com/paper/2307.05014/) (2023)
- [Continual Learning of Knowledge Graph Embeddings](https://mlsanity.com/paper/2101.05850/) (2021)
- [LeanAgent: Lifelong Learning for Formal Theorem Proving](https://mlsanity.com/paper/2410.06209/) (2024)
- [Continuum: Simple Management of Complex Continual Learning Scenarios](https://mlsanity.com/paper/2102.06253/) (2021)
- [CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparability](https://mlsanity.com/paper/2110.03331/) (2021)
- [Is Class-Incremental Enough for Continual Learning?](https://mlsanity.com/paper/2112.02925/) (2021)
- [Unsupervised Continual Semantic Adaptation through Neural Rendering](https://mlsanity.com/paper/2211.13969/) (2022)
- [A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application](https://mlsanity.com/paper/2310.14277/) (2023)
- [Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition](https://mlsanity.com/paper/2103.08637/) (2021)
- [CLARE: Continual Learning for Vision-Language-Action Models via Autonomous Adapter Routing and Expansion](https://mlsanity.com/paper/2601.09512/) (2026)
- [Practical Recommendations for Replay-based Continual Learning Methods](https://mlsanity.com/paper/2203.10317/) (2022)
- [Kaizen: Practical Self-supervised Continual Learning with Continual Fine-tuning](https://mlsanity.com/paper/2303.17235/) (2023)
- [Continually Evolving Skill Knowledge in Vision Language Action Model](https://mlsanity.com/paper/2511.18085/) (2025)
- [A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems](https://mlsanity.com/paper/2301.07799/) (2023)
- [Federated Continual Learning: Concepts, Challenges, and Solutions](https://mlsanity.com/paper/2502.07059/) (2025)
- [Selective Replay Enhances Learning in Online Continual Analogical Reasoning](https://mlsanity.com/paper/2103.03987/) (2021)
- [Continual Learning from Demonstration of Robotics Skills](https://mlsanity.com/paper/2202.06843/) (2022)
- [When Meta-Learning Meets Online and Continual Learning: A Survey](https://mlsanity.com/paper/2311.05241/) (2023)
- [Continual Learning on Graphs: A Survey](https://mlsanity.com/paper/2402.06330/) (2024)

## Abstract (arXiv)

Continual learning consists of algorithms that learn from a stream of data/tasks continuously and adaptively thought time, enabling the incremental development of ever more complex knowledge and skills. The lack of consensus in evaluating continual learning algorithms and the almost exclusive focus on forgetting motivate us to propose a more comprehensive set of implementation independent metrics accounting for several factors we believe have practical implications worth considering in the deployment of real AI systems that learn continually: accuracy or performance over time, backward and forward knowledge transfer, memory overhead as well as computational efficiency. Drawing inspiration from the standard Multi-Attribute Value Theory (MAVT) we further propose to fuse these metrics into a single score for ranking purposes and we evaluate our proposal with five continual learning strategies on the iCIFAR-100 continual learning benchmark.