mlsanityData to 7 Oct 2026

Replay in Deep Learning: Current Approaches and Missing Biological Elements

arXiv 2104.04132 ↗First posted 1 Apr 2021q-bio.NC cs.AI cs.LGcited by 71

Not on the map. Why?

Abstract (arXiv)

Replay is the reactivation of one or more neural patterns, which are similar to the activation patterns experienced during past waking experiences. Replay was first observed in biological neural networks during sleep, and it is now thought to play a critical role in memory formation, retrieval, and consolidation. Replay-like mechanisms have been incorporated into deep artificial neural networks that learn over time to avoid catastrophic forgetting of previous knowledge. Replay algorithms have been successfully used in a wide range of deep learning methods within supervised, unsupervised, and reinforcement learning paradigms. In this paper, we provide the first comprehensive comparison between replay in the mammalian brain and replay in artificial neural networks. We identify multiple aspects of biological replay that are missing in deep learning systems and hypothesize how they could be utilized to improve artificial neural networks.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Builds on 76 of its 313 references that are in the map. First posted on arXiv 1 Apr 2021.

Builds on

76 of 313 references.

References · 76

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  5. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  6. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  7. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  8. 2016Progressive Neural Networkscited by 1,332
  9. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  10. 2017Continual Learning with Deep Generative Replaycited by 1,152
  11. 2017Hindsight Experience Replaycited by 1,096
  12. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
38 more
  1. 2015Prioritized Experience Replaycited by 1,070
  2. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  3. 2017A Survey on Multi-Task Learningcited by 838
  4. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  5. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  6. 2019Large Scale Incremental Learningcited by 727
  7. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  8. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  9. 2018End-to-End Incremental Learningcited by 596
  10. 2019Gradient based sample selection for online continual learningcited by 568
  11. 2018End-to-End Multi-Task Learning with Attentioncited by 533
  12. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  13. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 376
  14. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
  15. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  16. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 359
  17. 2017Variational Continual Learningcited by 335
  18. 2019Learning Loss for Active Learningcited by 335
  19. 2019Maintaining Discrimination and Fairness in Class Incremental Learningcited by 281
  20. 2019Continual learning with hypernetworkscited by 267
  21. 2017Automated Curriculum Learning for Neural Networkscited by 251
  22. 2018Learning without Memorizingcited by 249
  23. 2018Selective Experience Replay for Lifelong Learningcited by 239
  24. 2020Mnemonics Training: Multi-Class Incremental Learning without Forgettingcited by 220
  25. 2018Online Structured Laplace Approximations For Overcoming Catastrophic Forgettingcited by 197
  26. 2019REMIND Your Neural Network to Prevent Catastrophic Forgettingcited by 197
  27. 2020Modeling the Background for Incremental Learning in Semantic Segmentationcited by 189
  28. 2019Episodic Memory in Lifelong Language Learningcited by 188
  29. 2019Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learningcited by 181
  30. 2017FearNet: Brain-Inspired Model for Incremental Learningcited by 177
  31. 2017Cost-Effective Active Learning for Deep Image Classificationcited by 165
  32. 2017Neural Episodic Controlcited by 148
  33. 2018Memory Efficient Experience Replay for Streaming Learningcited by 128
  34. 2019Functional Regularisation for Continual Learning with Gaussian Processescited by 121
  35. 2020Conditional Channel Gated Networks for Task-Aware Continual Learningcited by 115
  36. 2017Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learningcited by 114
  37. 2019Overcoming Catastrophic Forgetting with Unlabeled Data in the Wildcited by 111
  38. 2020Memory-Efficient Incremental Learning Through Feature Adaptationcited by 107

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 20; 2023: 17; 2024: 15; 2025: 7; 2026: 10.

71 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2022Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning Systemcited by 110
  3. 2022Continual Learning of Natural Language Processing Tasks: A Surveycited by 83
  4. 2024Consistent Prompting for Rehearsal-Free Continual Learningcited by 52
  5. 2024Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learningcited by 49
  6. 2022Online Continual Learning for Embedded Devicescited by 48
  7. 2024Recent Advances of Foundation Language Models-based Continual Learning: A Surveycited by 46
  8. 2022NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networkscited by 38
  9. 2022Large Language Models and the Reverse Turing Testcited by 38
  10. 2023Error Sensitivity Modulation based Experience Replay: Mitigating Abrupt Representation Drift in Continual Learningcited by 27
  11. 2022Continual Object Detection: A review of definitions, strategies, and challengescited by 25
  12. 2023SIESTA: Efficient Online Continual Learning with Sleepcited by 24
38 more
  1. 2024Streaming Deep Reinforcement Learning Finally Workscited by 24
  2. 2021Continual Learning at the Edge: Real-Time Training on Smartphone Devicescited by 20
  3. 2022Semantic Segmentation with Active Semi-Supervised Learningcited by 16
  4. 2023Overcoming the Stability Gap in Continual Learningcited by 13
  5. 2023GRASP: A Rehearsal Policy for Efficient Online Continual Learningcited by 13
  6. 2021Ex-Model: Continual Learning from a Stream of Trained Modelscited by 11
  7. 2022Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-2cited by 11
  8. 2022SYNERgy between SYNaptic consolidation and Experience Replay for general continual learningcited by 10
  9. 2023Class-Incremental Learning with Repetitioncited by 9
  10. 2023Design Principles for Lifelong Learning AI Acceleratorscited by 9
  11. 2022Entropy-based Stability-Plasticity for Lifelong Learningcited by 8
  12. 2024Where is the Truth? The Risk of Getting Confounded in a Continual Worldcited by 8
  13. 2023Improving Replay Sample Selection and Storage for Less Forgetting in Continual Learningcited by 7
  14. 2026Position: Modular Memory is the Key to Continual Learning Agentscited by 7
  15. 2022How Relevant is Selective Memory Population in Lifelong Language Learning?cited by 6
  16. 2024Interactive Continual Learning Architecture for Long-Term Personalization of Home Service Robotscited by 6
  17. 2024Calibration of Continual Learning Modelscited by 6
  18. 2024Replay Consolidation with Label Propagation for Continual Object Detectioncited by 6
  19. 2022An Investigation of the Combination of Rehearsal and Knowledge Distillation in Continual Learning for Spoken Language Understandingcited by 5
  20. 2025Sequence Transferability and Task Order Selection in Continual Learningcited by 5
  21. 2022Generative Negative Replay for Continual Learningcited by 4
  22. 2022Exploring the Intersection between Neural Architecture Search and Continual Learningcited by 4
  23. 2023Continual Learning through Human-Robot Interaction: Human Perceptions of a Continual Learning Robot in Repeated Interactionscited by 4
  24. 2022Cold Start Streaming Learning for Deep Networkscited by 3
  25. 2023Streaming LifeLong Learning With Any-Time Inferencecited by 3
  26. 2023Adiabatic replay for continual learningcited by 3
  27. 2023Class-Incremental Mixture of Gaussians for Deep Continual Learningcited by 3
  28. 2024Sleep-Like Unsupervised Replay Improves Performance when Data are Limited or Unbalancedcited by 3
  29. 2025Scalable Strategies for Continual Learning with Replaycited by 3
  30. 2022Continual Learning for Human State Monitoringcited by 2
  31. 2022LLEDA -- Lifelong Self-Supervised Domain Adaptationcited by 2
  32. 2024TiROD: Tiny Robotics Dataset and Benchmark for Continual Object Detectioncited by 2
  33. 2025Flashbacks to Harmonize Stability and Plasticity in Continual Learningcited by 2
  34. 2026A Practical Guide to Streaming Continual Learningcited by 2
  35. 2026Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Taskscited by 2
  36. 2022Can I see an Example? Active Learning the Long Tail of Attributes and Relationscited by 1
  37. 2022Continual learning benefits from multiple sleep mechanisms: NREM, REM, and Synaptic Downscalingcited by 1
  38. 2023Self-recovery of memory via generative replaycited by 1

50 most cited of 71 All 71 (JSON)