mlsanityData to 14 Oct 2026

Sample Compression for Self Certified Continual Learning

arXiv 2503.10503 ↗First posted 13 Mar 2025cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Continual learning algorithms aim to learn from a sequence of tasks. In order to avoid catastrophic forgetting, most existing approaches rely on heuristics and do not provide computable learning guarantees. In this paper, we introduce Continual Pick-to-Learn (CoP2L), a method grounded in sample compression theory that retains representative samples for each task in a principled and efficient way. This allows us to derive non-vacuous, numerically computable upper bounds on the generalization loss of the learned predictors after each task. We evaluate CoP2L on standard continual learning benchmarks under Class-Incremental and Task-Incremental settings, showing that it effectively mitigates catastrophic forgetting. It turns out that CoP2L is empirically competitive with baseline methods while certifying predictor reliability in continual learning with a non-vacuous bound.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 49 of its 84 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

49 of 84 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  6. 2016Learning without Forgettingcited by 2,496
  7. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  8. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  9. 2016Progressive Neural Networkscited by 1,361
  10. 2017Continual Learning with Deep Generative Replaycited by 1,158
  11. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  12. 2018Experience Replay for Continual Learningcited by 1,095
37 more
  1. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  2. 2021Learning to Prompt for Continual Learningcited by 944
  3. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
  4. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  5. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  6. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  7. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 379
  8. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
  9. 2019Orthogonal Gradient Descent for Continual Learningcited by 369
  10. 2020Coresets via Bilevel Optimization for Continual Learning and Streamingcited by 192
  11. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 184
  12. 2023A Unified Continual Learning Framework with General Parameter-Efficient Tuningcited by 125
  13. 2021Avalanche: an End-to-End Library for Continual Learningcited by 124
  14. 2021Online Coreset Selection for Rehearsal-based Continual Learningcited by 117
  15. 2024Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learningcited by 117
  16. 2020Tighter risk certificates for neural networkscited by 112
  17. 2020A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrixcited by 103
  18. 2022How catastrophic can catastrophic forgetting be in linear regression?cited by 81
  19. 2023Theory on Forgetting and Generalization of Continual Learningcited by 72
  20. 2020Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descentcited by 68
  21. 2021Reproducibility Report: La-MAML: Look-ahead Meta Learning for Continual Learningcited by 42
  22. 2023Avalanche: A PyTorch Library for Deep Continual Learningcited by 33
  23. 2022Analysis of Catastrophic Forgetting for Random Orthogonal Transformation Tasks in the Overparameterized Regimecited by 29
  24. 2017Less-forgetful Learning for Domain Expansion in Deep Neural Networkscited by 27
  25. 2023Compression, Generalization and Learningcited by 21
  26. 2023CL-MASR: A Continual Learning Benchmark for Multilingual ASRcited by 17
  27. 2021Provable Lifelong Learning of Representationscited by 15
  28. 2022Provable and Efficient Continual Representation Learningcited by 14
  29. 2024Contrastive Continual Learning with Importance Sampling and Prototype-Instance Relation Distillationcited by 11
  30. 2024Theoretical Insights into Overparameterized Models in Multi-Task and Replay-Based Continual Learningcited by 11
  31. 2020Batch-level Experience Replay with Review for Continual Learningcited by 10
  32. 2021Self-Bounding Majority Vote Learning Algorithms by the Direct Minimization of a Tight PAC-Bayesian C-Boundcited by 8
  33. 2022A Note on the Chernoff Bound for Random Variables in the Unit Intervalcited by 7
  34. 2022Navigating Memory Construction by Global Pseudo-Task Simulation for Continual Learningcited by 4
  35. 2024Sample Compression Unleashed: New Generalization Bounds for Real Valued Lossescited by 4
  36. 2018On the Perceptron's Compressioncited by 3
  37. 2024Data-dependent and Oracle Bounds on Forgetting in Continual Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Bound to Disagree: Generalization Bounds via Certifiable Surrogatescited by 2