mlsanityData to 7 Oct 2026

Information-theoretic Online Memory Selection for Continual Learning

arXiv 2204.04763 ↗First posted 10 Apr 2022cs.LG stat.MLcited by 42

Not on the map. Why?

Abstract (arXiv)

A challenging problem in task-free continual learning is the online selection of a representative replay memory from data streams. In this work, we investigate the online memory selection problem from an information-theoretic perspective. To gather the most information, we propose the textitsurprise and the textitlearnability criteria to pick informative points and to avoid outliers. We present a Bayesian model to compute the criteria efficiently by exploiting rank-one matrix structures. We demonstrate that these criteria encourage selecting informative points in a greedy algorithm for online memory selection. Furthermore, by identifying the importance of textitthe timing to update the memory, we introduce a stochastic information-theoretic reservoir sampler (InfoRS), which conducts sampling among selective points with high information. Compared to reservoir sampling, InfoRS demonstrates improved robustness against data imbalance. Finally, empirical performances over continual learning benchmarks manifest its efficiency and efficacy.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 29 of its 50 references that are in the map. First posted on arXiv 10 Apr 2022.

Builds on

29 of 50 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  6. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  7. 2018Experience Replay for Continual Learningcited by 1,074
  8. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  9. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  10. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  11. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  12. 2019Gradient based sample selection for online continual learningcited by 568
17 more
  1. 2020Dataset Condensation with Gradient Matchingcited by 510
  2. 2017Variational Continual Learningcited by 335
  3. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  4. 2017Automated Curriculum Learning for Neural Networkscited by 251
  5. 2018Selective Experience Replay for Lifelong Learningcited by 239
  6. 2018Task-Free Continual Learningcited by 219
  7. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 215
  8. 2020Coresets via Bilevel Optimization for Continual Learning and Streamingcited by 191
  9. 2019Functional Regularisation for Continual Learning with Gaussian Processescited by 121
  10. 2021Online Coreset Selection for Rehearsal-based Continual Learningcited by 114
  11. 2020Continual Deep Learning by Functional Regularisation of Memorable Pastcited by 104
  12. 2020Imbalanced Continual Learning with Partitioning Reservoir Samplingcited by 71
  13. 2017Automated Scalable Bayesian Inference via Hilbert Coresetscited by 55
  14. 2021Lossy Compression for Lossless Predictioncited by 51
  15. 2017Streaming Sparse Gaussian Process Approximationscited by 41
  16. 2020The Effectiveness of Memory Replay in Large Scale Continual Learningcited by 26
  17. 2015Online Submodular Maximization with Preemptioncited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 17; 2024: 6; 2025: 9; 2026: 4.

42 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  3. 2023Computationally Budgeted Continual Learning: What Does Matter?cited by 61
  4. 2023Real-Time Evaluation in Online Continual Learning: A New Hopecited by 47
  5. 2023The Ideal Continual Learner: An Agent That Never Forgetscited by 34
  6. 2023Online Continual Learning Without the Storage Constraintcited by 29
  7. 2022Unifying Approaches in Active Learning and Active Sampling via Fisher Information and Information-Theoretic Quantitiescited by 28
  8. 2023Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomycited by 24
  9. 2022Task-Free Continual Learning via Online Discrepancy Distance Learningcited by 22
  10. 2023Streaming Active Learning with Deep Neural Networkscited by 20
  11. 2023Summarizing Stream Data for Memory-Constrained Online Continual Learningcited by 19
  12. 2023Kalman Filter for Online Classification of Non-Stationary Datacited by 13
30 more
  1. 2022Optimizing Class Distribution in Memory for Multi-Label Online Continual Learningcited by 12
  2. 2022Learn the Time to Learn: Replay Scheduling in Continual Learningcited by 11
  3. 2022Memory Population in Continual Learning via Outlier Eliminationcited by 9
  4. 2023Towards Anytime Fine-tuning: Continually Pre-trained Language Models with Hypernetwork Promptcited by 8
  5. 2025Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspectivecited by 8
  6. 2023Knowledge Restore and Transfer for Multi-label Class-Incremental Learningcited by 7
  7. 2023Improving Replay Sample Selection and Storage for Less Forgetting in Continual Learningcited by 7
  8. 2024Advancing Deep Active Learning & Data Subset Selection: Unifying Principles with Information-Theory Intuitionscited by 7
  9. 2022How Relevant is Selective Memory Population in Lifelong Language Learning?cited by 6
  10. 2025Bridging Streaming Continual Learning via In-Context Large Tabular Modelscited by 6
  11. 2023Approximate Bayesian Class-Conditional Models under Continuous Representation Shiftcited by 5
  12. 2023Continual Learning in Predictive Autoscalingcited by 4
  13. 2023Primal Dual Continual Learning: Balancing Stability and Plasticity through Adaptive Memory Allocationcited by 4
  14. 2024How to Leverage Predictive Uncertainty Estimates for Reducing Catastrophic Forgetting in Online Continual Learningcited by 3
  15. 2023Analysis of the Memorization and Generalization Capabilities of AI Agents: Are Continual Learners Robust?cited by 1
  16. 2024Bridging Neuroscience and AI: Environmental Enrichment as a Model for Forward Knowledge Transfercited by 1
  17. 2024Task-agnostic Lifelong Robot Learning with Retrieval-based Weighted Local Adaptationcited by 1
  18. 2025Information-Theoretic Dual Memory System for Continual Learningcited by 1
  19. 2025Improvements of Dark Experience Replay and Reservoir Sampling towards Better Balance between Consolidation and Plasticitycited by 1
  20. 2025The Bayesian Approach to Continual Learning: An Overviewcited by 1
  21. 2025Compact Memory for Continual Logistic Regressioncited by 1
  22. 2026TopoPrune: Robust Data Pruning via Unified Latent Space Topologycited by 1
  23. 2024Retrospective Feature Estimation for Continual Learningcited by 0
  24. 2024Balanced Gradient Sample Retrieval for Enhanced Knowledge Retention in Proxy-based Continual Learningcited by 0
  25. 2025Prediction-Oriented Subsampling from Data Streamscited by 0
  26. 2025Rehearsal-free and Task-free Online Continual Learning With Contrastive Promptcited by 0
  27. 2025MODE: Multi-Objective Adaptive Coreset Selectioncited by 0
  28. 2026Recovery Guarantees for Continual Learning of Dependent Tasks: Memory, Data-Dependent Regularization, and Data-Dependent Weightscited by 0
  29. 2026Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learningcited by 0
  30. 2026Beyond Class Marginals: Bounding Rehearsal Gaps without Freezing Class Co-occurrencecited by 0