Data to 7 Oct 2026

Selective Experience Replay for Lifelong Learning

arXiv 1802.10269 ↗First posted 28 Feb 2018cs.AIcited by 239

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning has emerged as a powerful tool for a variety of learning tasks, however deep nets typically exhibit forgetting when learning multiple tasks in sequence. To mitigate forgetting, we propose an experience replay process that augments the standard FIFO buffer and selectively stores experiences in a long-term memory. We explore four strategies for selecting which experiences will be stored: favoring surprise, favoring reward, matching the global training distribution, and maximizing coverage of the state space. We show that distribution matching successfully prevents catastrophic forgetting, and is consistently the best approach on all domains tested. While distribution matching has better and more consistent performance, we identify one case in which coverage maximization is beneficial - when tasks that receive less trained are more important. Overall, our results show that selective experience replay, when suitable selection algorithms are employed, can prevent catastrophic forgetting.

Key facts. Cited by 239 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 36; 2022: 50; 2023: 51; 2024: 52; 2025: 33; 2026: 17.

239 papers.

Sort:

By area:

2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
2023Class-Incremental Learning: A Surveycited by 209
2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 152
2022NOTE: Robust Continual Test-time Adaptation Against Temporal Correlationcited by 150
2023Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Modelscited by 112
2022Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning Systemcited by 110
2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 71
2023LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discoverycited by 63
2021Continual Learning via Local Module Compositioncited by 61
2022Don't Stop Learning: Towards Continual Learning for the CLIP Modelcited by 61
2022An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditionscited by 54

38 more

2024Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learningcited by 49
2023Real-Time Evaluation in Online Continual Learning: A New Hopecited by 47
2023Few-Shot Class-Incremental Learning via Training-Free Prototype Calibrationcited by 44
2021Same State, Different Task: Continual Reinforcement Learning without Interferencecited by 42
2022Information-theoretic Online Memory Selection for Continual Learningcited by 42
2023Traj-MAE: Masked Autoencoders for Trajectory Predictioncited by 42
2024SDDGR: Stable Diffusion-based Deep Generative Replay for Class Incremental Object Detectioncited by 42
2022Modular Lifelong Reinforcement Learning via Neural Compositioncited by 39
2023Continual Semantic Segmentation with Automatic Memory Sample Selectioncited by 39
2022NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networkscited by 38
2024SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Modelscited by 37
2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 36
2024Scaling Laws for Forgetting When Fine-Tuning Large Language Modelscited by 36
2021Lifelong Robotic Reinforcement Learning by Retaining Experiencescited by 34
2021Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connectorcited by 34
2023The Ideal Continual Learner: An Agent That Never Forgetscited by 34
2024Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Modelscited by 34
2021Continual Learning via Bit-Level Information Preservingcited by 32
2024Convolutional Prompting meets Language Models for Continual Learningcited by 32
2021Continuous Coordination As a Realistic Scenario for Lifelong Learningcited by 31
2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
2022Meta-attention for ViT-backed Continual Learningcited by 29
2022Balancing Stability and Plasticity through Advanced Null Space in Continual Learningcited by 29
2022Building a Subspace of Policies for Scalable Continual Learningcited by 28
2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
2021An Investigation of Replay-based Approaches for Continual Learningcited by 26
2023Dynamic Residual Classifier for Class Incremental Learningcited by 26
2022Reinforced Continual Learning for Graphscited by 24
2024T4P: Test-Time Training of Trajectory Prediction via Masked Autoencoder and Actor-specific Token Memorycited by 20
2021A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directionscited by 19
2022Actor Prioritized Experience Replaycited by 19
2021Encoders and Ensembles for Task-Free Continual Learningcited by 17
2023Continual Graph Learning: A Surveycited by 17
2022Sparse Coding in a Dual Memory System for Lifelong Learningcited by 16
2023Ensemble Reinforcement Learning: A Surveycited by 16
2023Kaizen: Practical Self-supervised Continual Learning with Continual Fine-tuningcited by 16
2023ConPET: Continual Parameter-Efficient Tuning for Large Language Modelscited by 16
2024Dual Consolidation for Pre-Trained Model-Based Domain-Incremental Learningcited by 16

50 most cited of 239 All 239 (JSON)