# Online Coreset Selection for Rehearsal-based Continual Learning

arXiv: https://arxiv.org/abs/2106.01085 · HTML: https://mlsanity.com/paper/2106.01085/

## Key facts

- Cited by 117 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 49 references that are in the map.
- First posted on arXiv 2 Jun 2021.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Overcoming catastrophic forgetting with hard attention to the task](https://mlsanity.com/paper/1801.01423/) (2018)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [Variational Continual Learning](https://mlsanity.com/paper/1710.10628/) (2017)
- [Not All Samples Are Created Equal: Deep Learning with Importance Sampling](https://mlsanity.com/paper/1803.00942/) (2018)
- [On Tiny Episodic Memories in Continual Learning](https://mlsanity.com/paper/1902.10486/) (2019)
- [Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1904.00310/) (2019)
- [Overcoming Catastrophic Forgetting by Incremental Moment Matching](https://mlsanity.com/paper/1703.08475/) (2017)
- [Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement](https://mlsanity.com/paper/1903.06059/) (2019)
- [Coresets via Bilevel Optimization for Continual Learning and Streaming](https://mlsanity.com/paper/2006.03875/) (2020)
- [Reinforced Continual Learning](https://mlsanity.com/paper/1805.12369/) (2018)
- [Understanding the Role of Training Regimes in Continual Learning](https://mlsanity.com/paper/2006.06958/) (2020)
- [Federated Continual Learning with Weighted Inter-client Transfer](https://mlsanity.com/paper/2003.03196/) (2020)
- [Continual Lifelong Learning in Natural Language Processing: A Survey](https://mlsanity.com/paper/2012.09823/) (2020)
- [Linear Mode Connectivity in Multitask and Continual Learning](https://mlsanity.com/paper/2010.04495/) (2020)
- [Scalable and Order-robust Continual Learning with Additive Parameter Decomposition](https://mlsanity.com/paper/1902.09432/) (2019)
- [Functional Regularisation for Continual Learning with Gaussian Processes](https://mlsanity.com/paper/1901.11356/) (2019)
- [Lifelong GAN: Continual Learning for Conditional Image Generation](https://mlsanity.com/paper/1907.10107/) (2019)
- [Automated Scalable Bayesian Inference via Hilbert Coresets](https://mlsanity.com/paper/1710.05053/) (2017)

## Built on by (most cited first, 30 of 117)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning](https://mlsanity.com/paper/2204.08499/) (2022)
- [Improved Distribution Matching for Dataset Condensation](https://mlsanity.com/paper/2307.09742/) (2023)
- [GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning](https://mlsanity.com/paper/2111.11210/) (2021)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [Data Pruning via Moving-one-Sample-out](https://mlsanity.com/paper/2310.14664/) (2023)
- [Computationally Budgeted Continual Learning: What Does Matter?](https://mlsanity.com/paper/2303.11165/) (2023)
- [SparCL: Sparse Continual Learning on the Edge](https://mlsanity.com/paper/2209.09476/) (2022)
- [Consistent Prompting for Rehearsal-Free Continual Learning](https://mlsanity.com/paper/2403.08568/) (2024)
- [Real-Time Evaluation in Online Continual Learning: A New Hope](https://mlsanity.com/paper/2302.01047/) (2023)
- [FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clients](https://mlsanity.com/paper/2201.01601/) (2022)
- [Information-theoretic Online Memory Selection for Continual Learning](https://mlsanity.com/paper/2204.04763/) (2022)
- [Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learning](https://mlsanity.com/paper/2303.09483/) (2023)
- [On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning](https://mlsanity.com/paper/2210.06443/) (2022)
- [Continual Semantic Segmentation with Automatic Memory Sample Selection](https://mlsanity.com/paper/2304.05015/) (2023)
- [Recent Advances of Multimodal Continual Learning: A Comprehensive Survey](https://mlsanity.com/paper/2410.05352/) (2024)
- [On the Soft-Subnetwork for Few-shot Class Incremental Learning](https://mlsanity.com/paper/2209.07529/) (2022)
- [Continual Learning as Computationally Constrained Reinforcement Learning](https://mlsanity.com/paper/2307.04345/) (2023)
- [CLAD: A realistic Continual Learning benchmark for Autonomous Driving](https://mlsanity.com/paper/2210.03482/) (2022)
- [Online Continual Learning on a Contaminated Data Stream with Blurry Task Boundaries](https://mlsanity.com/paper/2203.15355/) (2022)
- [Online Continual Learning Without the Storage Constraint](https://mlsanity.com/paper/2305.09253/) (2023)
- [A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advances](https://mlsanity.com/paper/2505.17799/) (2025)
- [Rank-based Decomposable Losses in Machine Learning: A Survey](https://mlsanity.com/paper/2207.08768/) (2022)
- [Dataset Distillation by Automatic Training Trajectories](https://mlsanity.com/paper/2407.14245/) (2024)
- [WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models](https://mlsanity.com/paper/2410.17509/) (2024)
- [Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy](https://mlsanity.com/paper/2312.10549/) (2023)
- [Task-Free Continual Learning via Online Discrepancy Distance Learning](https://mlsanity.com/paper/2210.06579/) (2022)
- [Regularizing Second-Order Influences for Continual Learning](https://mlsanity.com/paper/2304.10177/) (2023)
- [Carpe Diem: On the Evaluation of World Knowledge in Lifelong Language Models](https://mlsanity.com/paper/2311.08106/) (2023)
- [LoRA Subtraction for Drift-Resistant Space in Exemplar-Free Continual Learning](https://mlsanity.com/paper/2503.18985/) (2025)

## Abstract (arXiv)

A dataset is a shred of crucial evidence to describe a task. However, each data point in the dataset does not have the same potential, as some of the data points can be more representative or informative than others. This unequal importance among the data points may have a large impact in rehearsal-based continual learning, where we store a subset of the training examples (coreset) to be replayed later to alleviate catastrophic forgetting. In continual learning, the quality of the samples stored in the coreset directly affects the model's effectiveness and efficiency. The coreset selection problem becomes even more important under realistic settings, such as imbalanced continual learning or noisy data scenarios. To tackle this problem, we propose Online Coreset Selection (OCS), a simple yet effective method that selects the most representative and informative coreset at each iteration and trains them in an online manner. Our proposed method maximizes the model's adaptation to a current dataset while selecting high-affinity samples to past tasks, which directly inhibits catastrophic forgetting. We validate the effectiveness of our coreset selection mechanism over various standard, imbalanced, and noisy datasets against strong continual learning baselines, demonstrating that it improves task adaptation and prevents catastrophic forgetting in a sample-efficient manner.