mlsanityData to 7 Oct 2026

Rethinking Experience Replay: a Bag of Tricks for Continual Learning

arXiv 2010.05595 ↗First posted 12 Oct 2020cs.LG stat.MLcited by 94

Not on the map. Why?

Abstract (arXiv)

In Continual Learning, a Neural Network is trained on a stream of data whose distribution shifts over time. Under these assumptions, it is especially challenging to improve on classes appearing later in the stream while remaining accurate on previous ones. This is due to the infamous problem of catastrophic forgetting, which causes a quick performance degradation when the classifier focuses on learning new categories. Recent literature proposed various approaches to tackle this issue, often resorting to very sophisticated techniques. In this work, we show that naive rehearsal can be patched to achieve similar performance. We point out some shortcomings that restrain Experience Replay (ER) and propose five tricks to mitigate them. Experiments show that ER, thus enhanced, displays an accuracy gain of 51.2 and 26.9 percentage points on the CIFAR-10 and CIFAR-100 datasets respectively (memory buffer size 1000). As a result, it surpasses current state-of-the-art rehearsal-based methods.

Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 20; 2023: 21; 2024: 25; 2025: 17; 2026: 6.

94 papers.

Sort:

By area:

  1. 2022Dataset Distillation using Neural Feature Regressioncited by 177
  2. 2023Dataset Distillation: A Comprehensive Reviewcited by 126
  3. 2022Class-Incremental Continual Learning into the eXtended DER-versecited by 121
  4. 2022Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning Systemcited by 110
  5. 2021Recent Advances of Continual Learning in Computer Vision: An Overviewcited by 55
  6. 2022SparCL: Sparse Continual Learning on the Edgecited by 55
  7. 2023PCR: Proxy-based Contrastive Replay for Online Class-Incremental Continual Learningcited by 50
  8. 2022Transfer without Forgettingcited by 44
  9. 2022Exploring Example Influence in Continual Learningcited by 44
  10. 2022On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learningcited by 39
  11. 2024Convolutional Prompting meets Language Models for Continual Learningcited by 32
  12. 2022Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learningcited by 27
38 more
  1. 2021Continual Semi-Supervised Learning through Contrastive Interpolation Consistencycited by 25
  2. 2023Tangent Model Composition for Ensembling and Continual Fine-tuningcited by 25
  3. 2024Efficient Continual Pre-training by Mitigating the Stability Gapcited by 25
  4. 2024Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problemcited by 23
  5. 2022Revisiting a kNN-based Image Classification System with High-capacity Storagecited by 22
  6. 2022Online Continual Learning with Contrastive Vision Transformercited by 21
  7. 2023Regularizing Second-Order Influences for Continual Learningcited by 21
  8. 2024Adaptive Memory Replay for Continual Learningcited by 21
  9. 2022A soft nearest-neighbor framework for continual semi-supervised learningcited by 17
  10. 2022Practical Recommendations for Replay-based Continual Learning Methodscited by 16
  11. 2023Curious Replay for Model-based Adaptationcited by 16
  12. 2022Provable and Efficient Continual Representation Learningcited by 14
  13. 2023Exploring Data Geometry for Continual Learningcited by 14
  14. 2022Task Agnostic Representation Consolidation: a Self-supervised based Continual Learning Approachcited by 13
  15. 2024Unveiling the Anomalies in an Ever-Changing World: A Benchmark for Pixel-Level Anomaly Detection in Continual Learningcited by 13
  16. 2021Complementary Calibration: Boosting General Continual Learning with Collaborative Distillation and Self-Supervisioncited by 12
  17. 2021One Pass ImageNetcited by 11
  18. 2022Memory Population in Continual Learning via Outlier Eliminationcited by 9
  19. 2022Continual Learning Approaches for Anomaly Detectioncited by 9
  20. 2024What Will My Model Forget? Forecasting Forgotten Examples in Language Model Refinementcited by 9
  21. 2023Lifelong Continual Learning for Anomaly Detection: New Challenges, Perspectives, and Insightscited by 8
  22. 2024Integrating Present and Past in Unsupervised Continual Learningcited by 8
  23. 2023Continual Learning with Strong Experience Replaycited by 7
  24. 2024Predicting the Susceptibility of Examples to Catastrophic Forgettingcited by 7
  25. 2023Latent Spectral Regularization for Continual Learningcited by 6
  26. 2023Preserving Linear Separability in Continual Learning by Backward Feature Projectioncited by 6
  27. 2023Exemplar-Free Continual Transformer with Convolutionscited by 6
  28. 2023TriRE: A Multi-Mechanism Learning Paradigm for Continual Knowledge Retention and Promotioncited by 6
  29. 2024Gradual Divergence for Seamless Adaptation: A Novel Domain Incremental Learning Methodcited by 6
  30. 2024Replay Consolidation with Label Propagation for Continual Object Detectioncited by 6
  31. 2023UER: A Heuristic Bias Addressing Approach for Online Continual Learningcited by 5
  32. 2023Continual Learning of Unsupervised Monocular Depth from Videoscited by 5
  33. 2022Rethinking Task-Incremental Learning Baselinescited by 4
  34. 2022A Multi-label Continual Learning Framework to Scale Deep Learning Approaches for Packaging Equipment Monitoringcited by 4
  35. 2022Exemplar-free Continual Learning of Vision Transformers via Gated Class-Attention and Cascaded Feature Drift Compensationcited by 4
  36. 2023Visually Grounded Continual Language Learning with Selective Specializationcited by 4
  37. 2025Memory Efficient Continual Learning for Edge-Based Visual Anomaly Detectioncited by 4
  38. 2026MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuningcited by 4

50 most cited of 94 All 94 (JSON)