# AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning

arXiv: https://arxiv.org/abs/2308.03810 · HTML: https://mlsanity.com/paper/2308.03810/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 52 references that are in the map.
- First posted on arXiv 7 Aug 2023.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [Variational Continual Learning](https://mlsanity.com/paper/1710.10628/) (2017)
- [Using Hindsight to Anchor Past Knowledge in Continual Learning](https://mlsanity.com/paper/2002.08165/) (2020)
- [Towards Robust Evaluations of Continual Learning](https://mlsanity.com/paper/1805.09733/) (2018)
- [Generalized Federated Learning via Sharpness Aware Minimization](https://mlsanity.com/paper/2206.02618/) (2022)
- [Class-Incremental Continual Learning into the eXtended DER-verse](https://mlsanity.com/paper/2201.00766/) (2022)
- [LoMar: A Local Defense Against Poisoning Attack on Federated Learning](https://mlsanity.com/paper/2201.02873/) (2022)
- [SATS: Self-Attention Transfer for Continual Semantic Segmentation](https://mlsanity.com/paper/2203.07667/) (2022)
- [Continual Learning via Inter-Task Synaptic Mapping](https://mlsanity.com/paper/2106.13954/) (2021)
- [FoCL: Feature-Oriented Continual Learning for Generative Models](https://mlsanity.com/paper/2003.03877/) (2020)

## Built on by (most cited first, 10 of 10)

- [Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcare](https://mlsanity.com/paper/2405.06784/) (2024)
- [Advancing continual lifelong learning in neural information retrieval: definition, dataset, framework, and empirical evaluation](https://mlsanity.com/paper/2308.08378/) (2023)
- [CFTS-GAN: Continual Few-Shot Teacher Student for Generative Adversarial Networks](https://mlsanity.com/paper/2410.14749/) (2024)
- [Distribution Shift Aware Neural Tabular Learning](https://mlsanity.com/paper/2508.19486/) (2025)
- [Continual Learning from Simulated Interactions via Multitask Prospective Rehearsal for Bionic Limb Behavior Modeling](https://mlsanity.com/paper/2405.01114/) (2024)
- [Audio-Visual Class-Incremental Learning for Fish Feeding intensity Assessment in Aquaculture](https://mlsanity.com/paper/2504.15171/) (2025)
- [Unsupervised Continual Learning for Amortized Bayesian Inference](https://mlsanity.com/paper/2602.22884/) (2026)
- [In Two Minds about Lifelong Learning: Exploring Hemispheric Redundancy and Specialisation in Neural Models](https://mlsanity.com/paper/2608.19514/) (2026)
- [Reliable Replay through Spatial Coherence in Online Continual Learning](https://mlsanity.com/paper/2609.33725/) (2026)
- [Making LoRA Identifiable: Orthogonal Alignment for Continual Learning](https://mlsanity.com/paper/2610.16664/) (2026)

## Abstract (arXiv)

Continual lifelong learning is an machine learning framework inspired by human learning, where learners are trained to continuously acquire new knowledge in a sequential manner. However, the non-stationary nature of streaming training data poses a significant challenge known as catastrophic forgetting, which refers to the rapid forgetting of previously learned knowledge when new tasks are introduced. While some approaches, such as experience replay (ER), have been proposed to mitigate this issue, their performance remains limited, particularly in the class-incremental scenario which is considered natural and highly challenging. In this paper, we present a novel algorithm, called adaptive-experience replay (AdaER), to address the challenge of continual lifelong learning. AdaER consists of two stages: memory replay and memory update. In the memory replay stage, AdaER introduces a contextually-cued memory recall (C-CMR) strategy, which selectively replays memories that are most conflicting with the current input data in terms of both data and task. Additionally, AdaER incorporates an entropy-balanced reservoir sampling (E-BRS) strategy to enhance the performance of the memory buffer by maximizing information entropy. To evaluate the effectiveness of AdaER, we conduct experiments on established supervised continual lifelong learning benchmarks, specifically focusing on class-incremental learning scenarios. The results demonstrate that AdaER outperforms existing continual lifelong learning baselines, highlighting its efficacy in mitigating catastrophic forgetting and improving learning performance.