mlsanityData to 7 Oct 2026

A Benchmark and Empirical Analysis for Replay Strategies in Continual Learning

arXiv 2208.02660 ↗First posted 4 Aug 2022cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

With the capacity of continual learning, humans can continuously acquire knowledge throughout their lifespan. However, computational systems are not, in general, capable of learning tasks sequentially. This long-standing challenge for deep neural networks (DNNs) is called catastrophic forgetting. Multiple solutions have been proposed to overcome this limitation. This paper makes an in-depth evaluation of the memory replay methods, exploring the efficiency, performance, and scalability of various sampling strategies when selecting replay data. All experiments are conducted on multiple datasets under various domains. Finally, a practical solution for selecting replay methods for various data distributions is provided.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 18 of its 29 references that are in the map. First posted on arXiv 4 Aug 2022.

Builds on

18 of 29 references.

References · 18

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2022Generative Adversarial Networkscited by 5,603
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2016Learning without Forgettingcited by 2,465
  7. 2016Progressive Neural Networkscited by 1,332
  8. 2017Continual Learning with Deep Generative Replaycited by 1,152
  9. 2017Deep Bayesian Active Learning with Image Datacited by 812
  10. 2019Three scenarios for continual learningcited by 610
  11. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
  12. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 376
6 more
  1. 2017CORe50: a New Dataset and Benchmark for Continuous Object Recognitioncited by 287
  2. 2020Coresets via Bilevel Optimization for Continual Learning and Streamingcited by 191
  3. 2020Optimal Continual Learning has Perfect Memory and is NP-hardcited by 86
  4. 2018Generative Models from the perspective of Continual Learningcited by 69
  5. 2019Sparse Variational Inference: Bayesian Coresets from Scratchcited by 28
  6. 2019OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learningcited by 28

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 5; 2025: 1.

8 papers.

  1. 2024Unveiling the Anomalies in an Ever-Changing World: A Benchmark for Pixel-Level Anomaly Detection in Continual Learningcited by 13
  2. 2022Continual Learning Approaches for Anomaly Detectioncited by 9
  3. 2024Multi-Label Continual Learning for the Medical Domain: A Novel Benchmarkcited by 6
  4. 2024Replay Consolidation with Label Propagation for Continual Object Detectioncited by 6
  5. 2025Memory Efficient Continual Learning for Edge-Based Visual Anomaly Detectioncited by 4
  6. 2023Streaming LifeLong Learning With Any-Time Inferencecited by 3
  7. 2024A Contrastive Symmetric Forward-Forward Algorithm (SFFA) for Continual Learning Taskscited by 2
  8. 2024TiROD: Tiny Robotics Dataset and Benchmark for Continual Object Detectioncited by 2