# GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning

arXiv: https://arxiv.org/abs/2111.11210 · HTML: https://mlsanity.com/paper/2111.11210/

## Key facts

- Cited by 95 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 18 Nov 2021.

## Builds on (most cited first, 29 of 29)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Overcoming catastrophic forgetting with hard attention to the task](https://mlsanity.com/paper/1801.01423/) (2018)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training](https://mlsanity.com/paper/2103.00123/) (2021)
- [Overcoming Catastrophic Forgetting by Incremental Moment Matching](https://mlsanity.com/paper/1703.08475/) (2017)
- [GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning](https://mlsanity.com/paper/2012.10630/) (2020)
- [Compacting, Picking and Growing for Unforgetting Continual Learning](https://mlsanity.com/paper/1910.06562/) (2019)
- [Coresets via Bilevel Optimization for Continual Learning and Streaming](https://mlsanity.com/paper/2006.03875/) (2020)
- [Online Coreset Selection for Rehearsal-based Continual Learning](https://mlsanity.com/paper/2106.01085/) (2021)
- [Coresets for Scalable Bayesian Logistic Regression](https://mlsanity.com/paper/1605.06423/) (2016)
- [RETRIEVE: Coreset Selection for Efficient and Robust Semi-Supervised Learning](https://mlsanity.com/paper/2106.07760/) (2021)
- [Bayesian Coreset Construction via Greedy Iterative Geodesic Ascent](https://mlsanity.com/paper/1802.01737/) (2018)
- [CPR: Classifier-Projection Regularization for Continual Learning](https://mlsanity.com/paper/2006.07326/) (2020)
- [Automated Scalable Bayesian Inference via Hilbert Coresets](https://mlsanity.com/paper/1710.05053/) (2017)
- [SpaceNet: Make Free Space For Continual Learning](https://mlsanity.com/paper/2007.07617/) (2020)
- [Continual Learning with Adaptive Weights (CLAW)](https://mlsanity.com/paper/1911.09514/) (2019)
- [Sparse Variational Inference: Bayesian Coresets from Scratch](https://mlsanity.com/paper/1906.03329/) (2019)
- [Continual Learning with Gated Incremental Memories for sequential data processing](https://mlsanity.com/paper/2004.04077/) (2020)

## Built on by (most cited first, 30 of 95)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Robust Test-Time Adaptation in Dynamic Scenarios](https://mlsanity.com/paper/2303.13899/) (2023)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning](https://mlsanity.com/paper/2501.13198/) (2025)
- [Recent Advances of Foundation Language Models-based Continual Learning: A Survey](https://mlsanity.com/paper/2405.18653/) (2024)
- [When Prompt-based Incremental Learning Does Not Meet Strong Pretraining](https://mlsanity.com/paper/2308.10445/) (2023)
- [Continual Semantic Segmentation with Automatic Memory Sample Selection](https://mlsanity.com/paper/2304.05015/) (2023)
- [Continual Self-supervised Learning: Towards Universal Multi-modal Medical Data Representation Learning](https://mlsanity.com/paper/2311.17597/) (2023)
- [Regularizing Second-Order Influences for Continual Learning](https://mlsanity.com/paper/2304.10177/) (2023)
- [Summarizing Stream Data for Memory-Constrained Online Continual Learning](https://mlsanity.com/paper/2305.16645/) (2023)
- [Adapter Merging with Centroid Prototype Mapping for Scalable Class-Incremental Learning](https://mlsanity.com/paper/2412.18219/) (2024)
- [SMART: Submodular Data Mixture Strategy for Instruction Tuning](https://mlsanity.com/paper/2403.08370/) (2024)
- [Revisiting Neural Networks for Continual Learning: An Architectural Perspective](https://mlsanity.com/paper/2404.14829/) (2024)
- [SCALE: Online Self-Supervised Lifelong Learning without Prior Knowledge](https://mlsanity.com/paper/2208.11266/) (2022)
- [Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting](https://mlsanity.com/paper/2502.02797/) (2025)
- [Heterogeneous Forgetting Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2308.03374/) (2023)
- [Task-Agnostic Guided Feature Expansion for Class-Incremental Learning](https://mlsanity.com/paper/2503.00823/) (2025)
- [MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning](https://mlsanity.com/paper/2407.20999/) (2024)
- [Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure](https://mlsanity.com/paper/2403.07292/) (2024)
- [Provable Contrastive Continual Learning](https://mlsanity.com/paper/2405.18756/) (2024)
- [Optimizing Class Distribution in Memory for Multi-Label Online Continual Learning](https://mlsanity.com/paper/2209.11469/) (2022)
- [Continual Learning by Modeling Intra-Class Variation](https://mlsanity.com/paper/2210.05398/) (2022)
- [Coreset Sampling from Open-Set for Fine-Grained Self-Supervised Learning](https://mlsanity.com/paper/2303.11101/) (2023)
- [Contrastive Continual Learning with Importance Sampling and Prototype-Instance Relation Distillation](https://mlsanity.com/paper/2403.04599/) (2024)
- [D3Former: Debiased Dual Distilled Transformer for Incremental Learning](https://mlsanity.com/paper/2208.00777/) (2022)
- [Continual Learning for Predictive Maintenance: Overview and Challenges](https://mlsanity.com/paper/2301.12467/) (2023)
- [Never-Ending Behavior-Cloning Agent for Robotic Manipulation](https://mlsanity.com/paper/2403.00336/) (2024)
- [Memory Population in Continual Learning via Outlier Elimination](https://mlsanity.com/paper/2207.01145/) (2022)
- [MILO: Model-Agnostic Subset Selection Framework for Efficient Model Training and Tuning](https://mlsanity.com/paper/2301.13287/) (2023)

## Abstract (arXiv)

Continual learning (CL) aims to develop techniques by which a single model adapts to an increasing number of tasks encountered sequentially, thereby potentially leveraging learnings across tasks in a resource-efficient manner. A major challenge for CL systems is catastrophic forgetting, where earlier tasks are forgotten while learning a new task. To address this, replay-based CL approaches maintain and repeatedly retrain on a small buffer of data selected across encountered tasks. We propose Gradient Coreset Replay (GCR), a novel strategy for replay buffer selection and update using a carefully designed optimization criterion. Specifically, we select and maintain a "coreset" that closely approximates the gradient of all the data seen so far with respect to current model parameters, and discuss key strategies needed for its effective application to the continual learning setting. We show significant gains (2%-4% absolute) over the state-of-the-art in the well-studied offline continual learning setting. Our findings also effectively transfer to online / streaming CL settings, showing upto 5% gains over existing approaches. Finally, we demonstrate the value of supervised contrastive loss for continual learning, which yields a cumulative gain of up to 5% accuracy when combined with our subset selection strategy.