# Class-Incremental Experience Replay for Continual Learning under Concept Drift

arXiv: https://arxiv.org/abs/2104.11861 · HTML: https://mlsanity.com/paper/2104.11861/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 24 Apr 2021.

## Builds on (most cited first, 19 of 19)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1904.00310/) (2019)
- [Continual learning with hypernetworks](https://mlsanity.com/paper/1906.00695/) (2019)
- [Semantic Drift Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2004.00440/) (2020)
- [Task-Free Continual Learning](https://mlsanity.com/paper/1812.03596/) (2018)
- [Memory Efficient Experience Replay for Streaming Learning](https://mlsanity.com/paper/1809.05922/) (2018)
- [Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1702.08887/) (2017)
- [Online Deep Learning: Learning Deep Neural Networks on the Fly](https://mlsanity.com/paper/1711.03705/) (2017)
- [Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles](https://mlsanity.com/paper/1606.07839/) (2016)
- [Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering](https://mlsanity.com/paper/1909.06356/) (2019)
- [SpaceNet: Make Free Space For Continual Learning](https://mlsanity.com/paper/2007.07617/) (2020)
- [Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replay](https://mlsanity.com/paper/1903.04566/) (2019)
- [An Equivalence between Loss Functions and Non-Uniform Sampling in Experience Replay](https://mlsanity.com/paper/2007.06049/) (2020)

## Built on by (most cited first, 17 of 17)

- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data](https://mlsanity.com/paper/2105.02340/) (2021)
- [A survey on learning from imbalanced data streams: taxonomy, challenges, empirical study, and reproducible experimental framework](https://mlsanity.com/paper/2204.03719/) (2022)
- [Dealing with Cross-Task Class Discrimination in Online Continual Learning](https://mlsanity.com/paper/2305.14657/) (2023)
- [Continual Learning with Weight Interpolation](https://mlsanity.com/paper/2404.04002/) (2024)
- [Towards A Holistic View of Bias in Machine Learning: Bridging Algorithmic Fairness and Imbalanced Learning](https://mlsanity.com/paper/2207.06084/) (2022)
- [Replay-enhanced Continual Reinforcement Learning](https://mlsanity.com/paper/2311.11557/) (2023)
- [Efficient Augmentation for Imbalanced Deep Learning](https://mlsanity.com/paper/2207.06080/) (2022)
- [Solving Continual Offline RL through Selective Weights Activation on Aligned Spaces](https://mlsanity.com/paper/2410.15698/) (2024)
- [History Repeats: Overcoming Catastrophic Forgetting For Event-Centric Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2305.18675/) (2023)
- [Class-Incremental Mixture of Gaussians for Deep Continual Learning](https://mlsanity.com/paper/2307.04094/) (2023)
- [On the combined effect of class imbalance and concept complexity in deep learning](https://mlsanity.com/paper/2107.14194/) (2021)
- [Continual Diffuser (CoD): Mastering Continual Offline Reinforcement Learning with Experience Rehearsal](https://mlsanity.com/paper/2409.02512/) (2024)
- [Holistic Continual Learning under Concept Drift with Adaptive Memory Realignment](https://mlsanity.com/paper/2507.02310/) (2025)
- [A Comprehensive Review of Machine Learning Advances on Data Change: A Cross-Field Perspective](https://mlsanity.com/paper/2402.12627/) (2024)
- [Balanced Gradient Sample Retrieval for Enhanced Knowledge Retention in Proxy-based Continual Learning](https://mlsanity.com/paper/2412.14430/) (2024)
- [MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound](https://mlsanity.com/paper/2605.11617/) (2026)

## Abstract (arXiv)

Modern machine learning systems need to be able to cope with constantly arriving and changing data. Two main areas of research dealing with such scenarios are continual learning and data stream mining. Continual learning focuses on accumulating knowledge and avoiding forgetting, assuming information once learned should be stored. Data stream mining focuses on adaptation to concept drift and discarding outdated information, assuming that only the most recent data is relevant. While these two areas are mainly being developed in separation, they offer complementary views on the problem of learning from dynamic data. There is a need for unifying them, by offering architectures capable of both learning and storing new information, as well as revisiting and adapting to changes in previously seen concepts. We propose a novel continual learning approach that can handle both tasks. Our experience replay method is fueled by a centroid-driven memory storing diverse instances of incrementally arriving classes. This is enhanced with a reactive subspace buffer that tracks concept drift occurrences in previously seen classes and adapts clusters accordingly. The proposed architecture is thus capable of both remembering valid and forgetting outdated information, offering a holistic framework for continual learning under concept drift.