mlsanityData to 7 Oct 2026

Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgetting

arXiv 1812.02464 ↗First posted 6 Dec 2018cs.LG cs.AI cs.CVcited by 46

Not on the map. Why?

Abstract (arXiv)

Neural networks can achieve excellent results in a wide variety of applications. However, when they attempt to sequentially learn, they tend to learn the new task while catastrophically forgetting previous ones. We propose a model that overcomes catastrophic forgetting in sequential reinforcement learning by combining ideas from continual learning in both the image classification domain and the reinforcement learning domain. This model features a dual memory system which separates continual learning from reinforcement learning and a pseudo-rehearsal system that "recalls" items representative of previous tasks via a deep generative network. Our model sequentially learns Atari 2600 games without demonstrating catastrophic forgetting and continues to perform above human level on all three games. This result is achieved without: demanding additional storage requirements as the number of tasks increases, storing raw data or revisiting past tasks. In comparison, previous state-of-the-art solutions are substantially more vulnerable to forgetting on these complex deep reinforcement learning tasks.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 11; 2023: 10; 2024: 12; 2025: 7; 2026: 2.

46 papers.

Sort:

By area:

  1. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 71
  2. 2021Continual Neural Mapping: Learning An Implicit Scene Representation from Sequential Observationscited by 30
  3. 2022Building a Subspace of Policies for Scalable Continual Learningcited by 28
  4. 2022Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learningcited by 23
  5. 2024Learn and Don't Forget: Adding a New Language to ASR Foundation Modelscited by 16
  6. 2025A Survey of Continual Reinforcement Learningcited by 16
  7. 2021Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillationcited by 15
  8. 2021CoMPS: Continual Meta Policy Searchcited by 15
  9. 2022Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualizationcited by 13
  10. 2024Unveiling the Anomalies in an Ever-Changing World: A Benchmark for Pixel-Level Anomaly Detection in Continual Learningcited by 13
  11. 2022Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-2cited by 11
  12. 2022Beyond Supervised Continual Learning: a Reviewcited by 11
34 more
  1. 2024Continual Learning on Graphs: A Surveycited by 11
  2. 2022Memory-efficient Reinforcement Learning with Value-based Knowledge Consolidationcited by 9
  3. 2022Continual Learning Approaches for Anomaly Detectioncited by 9
  4. 2024Dynamic Transformer Architecture for Continual Learning of Multimodal Taskscited by 8
  5. 2024Active Fine-Tuning of Multi-Task Policiescited by 8
  6. 2025MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learningcited by 8
  7. 2023Online Reinforcement Learning in Non-Stationary Context-Driven Environmentscited by 6
  8. 2023Replay-enhanced Continual Reinforcement Learningcited by 6
  9. 2024Stable Continual Reinforcement Learning via Diffusion-based Trajectory Replaycited by 6
  10. 2024Certified Continual Learning for Neural Network Regressioncited by 5
  11. 2024Overcoming Catastrophic Forgetting in Tabular Data Classification: A Pseudorehearsal-based approachcited by 5
  12. 2024ACL-QL: Adaptive Conservative Level in Q-Learning for Offline Reinforcement Learningcited by 5
  13. 2022Avalanche RL: a Continual Reinforcement Learning Librarycited by 4
  14. 2023Adiabatic replay for continual learningcited by 3
  15. 2023CLVOS23: A Long Video Object Segmentation Dataset for Continual Learningcited by 3
  16. 2022Demystifying Reinforcement Learning in Time-Varying Systemscited by 2
  17. 2024Elastic Multi-Gradient Descent for Parallel Continual Learningcited by 2
  18. 2024Continual Diffuser (CoD): Mastering Continual Offline Reinforcement Learning with Experience Rehearsalcited by 2
  19. 2025Clustering-Based Weight Orthogonalization for Stabilizing Deep Reinforcement Learningcited by 2
  20. 2023Hawk: An Industrial-strength Multi-label Document Classifiercited by 1
  21. 2023H-InDex: Visual Reinforcement Learning with Hand-Informed Representations for Dexterous Manipulationcited by 1
  22. 2023Uncertainty-aware transfer across tasks using hybrid model-based successor feature reinforcement learningcited by 1
  23. 2024Sequential Learning in the Dense Associative Memorycited by 1
  24. 2022Improving generalization to new environments and removing catastrophic forgetting in Reinforcement Learning by using an eco-system of agentscited by 0
  25. 2022ImpressLearn: Continual Learning via Combined Task Impressionscited by 0
  26. 2023Catastrophic Forgetting in the Context of Model Updatescited by 0
  27. 2023Memory-Efficient Continual Learning Object Segmentation for Long Videocited by 0
  28. 2023All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimizationcited by 0
  29. 2025Neuron-level Balance between Stability and Plasticity in Deep Reinforcement Learningcited by 0
  30. 2025Optimisation Is Not What You Needcited by 0
  31. 2025Efficient Reinforcement Learning by Reducing Forgetting with Elephant Activation Functionscited by 0
  32. 2025Forgetting is Everywherecited by 0
  33. 2026Stagnant Neuron: Towards Understanding the Plasticity Loss in Multi-Agent Reinforcement Learning Value Factorization Methodscited by 0
  34. 2026Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearningcited by 0