mlsanityData to 7 Oct 2026

Coresets via Bilevel Optimization for Continual Learning and Streaming

arXiv 2006.03875 ↗First posted 6 Jun 2020cs.LG stat.MLcited by 191

Not on the map. Why?

Abstract (arXiv)

Coresets are small data summaries that are sufficient for model training. They can be maintained online, enabling efficient handling of large data streams under resource constraints. However, existing constructions are limited to simple models such as k-means and logistic regression. In this work, we propose a novel coreset construction via cardinality-constrained bilevel optimization. We show how our framework can efficiently generate coresets for deep neural networks, and demonstrate its empirical benefits in continual learning and in streaming settings.

Key facts. Cited by 191 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 38; 2023: 43; 2024: 39; 2025: 28; 2026: 15.

191 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2022Dataset Distillation by Matching Training Trajectoriescited by 437
  3. 2021Dataset Condensation with Distribution Matchingcited by 362
  4. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  5. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  6. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 233
  7. 2021New Insights on Reducing Abrupt Representation Change in Online Continual Learningcited by 215
  8. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 215
  9. 2023Model Sparsity Can Simplify Machine Unlearningcited by 206
  10. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  11. 2022DeepCore: A Comprehensive Library for Coreset Selection in Deep Learningcited by 146
  12. 2022Training Data Influence Analysis and Estimation: A Surveycited by 134
38 more
  1. 2022Dataset Pruning: Reducing Training Data by Examining Generalization Influencecited by 132
  2. 2023Dataset Distillation: A Comprehensive Reviewcited by 126
  3. 2023Generalizing Dataset Distillation via Deep Generative Priorcited by 125
  4. 2021Online Coreset Selection for Rehearsal-based Continual Learningcited by 114
  5. 2022Efficient Dataset Distillation Using Random Feature Approximationcited by 111
  6. 2021GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learningcited by 95
  7. 2022Constrained Few-shot Class-incremental Learningcited by 93
  8. 2024How to Train Data-Efficient LLMscited by 91
  9. 2023Data Distillation: A Surveycited by 86
  10. 2023Anchor Points: Benchmarking Models with Much Fewer Examplescited by 84
  11. 2024D4M: Dataset Distillation via Disentangled Diffusion Modelcited by 76
  12. 2021A Single-Timescale Method for Stochastic Bilevel Optimizationcited by 69
  13. 2022FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learningcited by 65
  14. 2023Large-scale Dataset Pruning with Dynamic Uncertaintycited by 63
  15. 2022Graph Condensation via Receptive Field Distribution Matchingcited by 60
  16. 2021Recent Advances of Continual Learning in Computer Vision: An Overviewcited by 55
  17. 2022SparCL: Sparse Continual Learning on the Edgecited by 55
  18. 2022A Framework and Benchmark for Deep Batch Active Learning for Regressioncited by 47
  19. 2023Dataset Distillation with Convexified Implicit Gradientscited by 46
  20. 2021Posterior Meta-Replay for Continual Learningcited by 45
  21. 2024Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearningcited by 44
  22. 2023Model Agnostic Sample Reweighting for Out-of-Distribution Learningcited by 43
  23. 2024Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptionscited by 43
  24. 2022Information-theoretic Online Memory Selection for Continual Learningcited by 42
  25. 2024Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHFcited by 41
  26. 2022Delving into Effective Gradient Matching for Dataset Condensationcited by 40
  27. 2023Continual Semantic Segmentation with Automatic Memory Sample Selectioncited by 39
  28. 2023An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learningcited by 39
  29. 2022Submodularity In Machine Learning and Artificial Intelligencecited by 37
  30. 2022Dataset Condensation with Latent Space Knowledge Factorization and Sharingcited by 34
  31. 2023The Ideal Continual Learner: An Agent That Never Forgetscited by 34
  32. 2023Loss-Curvature Matching for Dataset Selection and Condensationcited by 32
  33. 2024Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMscited by 32
  34. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
  35. 2024TAGCOS: Task-agnostic Gradient Clustered Coreset Selection for Instruction Tuning Datacited by 28
  36. 2025A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advancescited by 28
  37. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  38. 2022Rank-based Decomposable Losses in Machine Learning: A Surveycited by 26

50 most cited of 191 All 191 (JSON)