mlsanityData to 7 Oct 2026

Gradient-matching coresets for continual learning

arXiv 2112.05025 ↗First posted 9 Dec 2021cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

We devise a coreset selection method based on the idea of gradient matching: The gradients induced by the coreset should match, as closely as possible, those induced by the original training dataset. We evaluate the method in the context of continual learning, where it can be used to curate a rehearsal memory. Our method performs strong competitors such as reservoir sampling across a range of memory sizes.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 12 of its 20 references that are in the map. First posted on arXiv 9 Dec 2021.

Builds on

12 of 20 references.

References · 12

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  5. 2016Learning without Forgettingcited by 2,465
  6. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  7. 2019Gradient based sample selection for online continual learningcited by 568
  8. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 529
  9. 2020Dataset Condensation with Gradient Matchingcited by 510
  10. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 376
  11. 2020Coresets via Bilevel Optimization for Continual Learning and Streamingcited by 191
  12. 2018Bayesian Coreset Construction via Greedy Iterative Geodesic Ascentcited by 68

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1.

2 papers.

  1. 2023Regularizing Second-Order Influences for Continual Learningcited by 21
  2. 2022Black-box Coreset Variational Inferencecited by 4