# Continual evaluation for lifelong learning: Identifying the stability gap

arXiv: https://arxiv.org/abs/2205.13452 · HTML: https://mlsanity.com/paper/2205.13452/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 57 references that are in the map.
- First posted on arXiv 26 May 2022.

## Builds on (most cited first, 30 of 37)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Overcoming catastrophic forgetting with hard attention to the task](https://mlsanity.com/paper/1801.01423/) (2018)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [On Tiny Episodic Memories in Continual Learning](https://mlsanity.com/paper/1902.10486/) (2019)
- [Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines](https://mlsanity.com/paper/1810.12488/) (2018)
- [New Insights on Reducing Abrupt Representation Change in Online Continual Learning](https://mlsanity.com/paper/2104.05025/) (2021)
- [Task2Vec: Task Embedding for Meta-Learning](https://mlsanity.com/paper/1902.03545/) (2019)
- [Domain Adaptive Ensemble Learning](https://mlsanity.com/paper/2003.07325/) (2020)
- [Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams](https://mlsanity.com/paper/2009.00919/) (2020)
- [Memory Efficient Experience Replay for Streaming Learning](https://mlsanity.com/paper/1809.05922/) (2018)

## Built on by (most cited first, 30 of 56)

- [Continual Pre-Training of Large Language Models: How to (re)warm your model?](https://mlsanity.com/paper/2308.04014/) (2023)
- [Simple and Scalable Strategies to Continually Pre-train Large Language Models](https://mlsanity.com/paper/2403.08763/) (2024)
- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learning](https://mlsanity.com/paper/2404.00781/) (2024)
- [LeanAgent: Lifelong Learning for Formal Theorem Proving](https://mlsanity.com/paper/2410.06209/) (2024)
- [Online Continual Learning Without the Storage Constraint](https://mlsanity.com/paper/2305.09253/) (2023)
- [CLAP4CLIP: Continual Learning with Probabilistic Finetuning for Vision-Language Models](https://mlsanity.com/paper/2403.19137/) (2024)
- [Efficient Continual Pre-training by Mitigating the Stability Gap](https://mlsanity.com/paper/2406.14833/) (2024)
- [A Comprehensive Empirical Evaluation on Online Continual Learning](https://mlsanity.com/paper/2308.10328/) (2023)
- [A Practitioner's Guide to Continual Multimodal Pretraining](https://mlsanity.com/paper/2408.14471/) (2024)
- [Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments](https://mlsanity.com/paper/2606.05661/) (2026)
- [Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How](https://mlsanity.com/paper/2311.04898/) (2023)
- [Grounding Multilingual Multimodal LLMs With Cultural Knowledge](https://mlsanity.com/paper/2508.07414/) (2025)
- [Improving Online Continual Learning Performance and Stability with Temporal Ensembles](https://mlsanity.com/paper/2306.16817/) (2023)
- [Overcoming the Stability Gap in Continual Learning](https://mlsanity.com/paper/2306.01904/) (2023)
- [Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning](https://mlsanity.com/paper/2402.09542/) (2024)
- [Learn the Time to Learn: Replay Scheduling in Continual Learning](https://mlsanity.com/paper/2209.08660/) (2022)
- [SLCA++: Unleash the Power of Sequential Fine-tuning for Continual Learning with Pre-training](https://mlsanity.com/paper/2408.08295/) (2024)
- [Design Principles for Lifelong Learning AI Accelerators](https://mlsanity.com/paper/2310.04467/) (2023)
- [PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learning](https://mlsanity.com/paper/2407.14029/) (2024)
- [Online Curvature-Aware Replay: Leveraging mathbf2nd Order Information for Online Continual Learning](https://mlsanity.com/paper/2502.01866/) (2025)
- [Early Preparation Pays Off: New Classifier Pre-tuning for Class Incremental Semantic Segmentation](https://mlsanity.com/paper/2407.14142/) (2024)
- [Bridging Streaming Continual Learning via In-Context Large Tabular Models](https://mlsanity.com/paper/2512.11668/) (2025)
- [Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training](https://mlsanity.com/paper/2502.06589/) (2025)
- [Learning Dynamics in Continual Pre-Training for Large Language Models](https://mlsanity.com/paper/2505.07796/) (2025)
- [Emergent Abilities of Large Language Models under Continued Pretraining for Language Adaptation](https://mlsanity.com/paper/2506.00288/) (2025)
- [Implicit biases in multitask and continual learning from a backward error analysis perspective](https://mlsanity.com/paper/2311.00235/) (2023)
- [Continual Learning on a Diet: Learning from Sparsely Labeled Streams Under Constrained Computation](https://mlsanity.com/paper/2404.12766/) (2024)
- [Exploring the Stability Gap in Continual Learning: The Role of the Classification Head](https://mlsanity.com/paper/2411.04723/) (2024)
- [Putting a Face to Forgetting: Continual Learning meets Mechanistic Interpretability](https://mlsanity.com/paper/2601.22012/) (2026)

## Abstract (arXiv)

Time-dependent data-generating distributions have proven to be difficult for gradient-based training of neural networks, as the greedy updates result in catastrophic forgetting of previously learned knowledge. Despite the progress in the field of continual learning to overcome this forgetting, we show that a set of common state-of-the-art methods still suffers from substantial forgetting upon starting to learn new tasks, except that this forgetting is temporary and followed by a phase of performance recovery. We refer to this intriguing but potentially problematic phenomenon as the stability gap. The stability gap had likely remained under the radar due to standard practice in the field of evaluating continual learning models only after each task. Instead, we establish a framework for continual evaluation that uses per-iteration evaluation and we define a new set of metrics to quantify worst-case performance. Empirically we show that experience replay, constraint-based replay, knowledge-distillation, and parameter regularization methods are all prone to the stability gap; and that the stability gap can be observed in class-, task-, and domain-incremental learning benchmarks. Additionally, a controlled experiment shows that the stability gap increases when tasks are more dissimilar. Finally, by disentangling gradients into plasticity and stability components, we propose a conceptual explanation for the stability gap.