mlsanityData to 7 Oct 2026

Policy Consolidation for Continual Reinforcement Learning

arXiv 1902.00255 ↗First posted 1 Feb 2019cs.LG stat.MLcited by 37

Not on the map. Why?

Abstract (arXiv)

We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is textitagnostic to the timescale of changes in the distribution of experiences, does not require knowledge of task boundaries, and can adapt in textitcontinuously changing environments. In our textitpolicy consolidation model, the policy network interacts with a cascade of hidden networks that simultaneously remember the agent's policy at a range of timescales and regularise the current policy by its own history, thereby improving its ability to learn without forgetting. We find that the model improves continual learning relative to baselines on a number of continuous control tasks in single-task, alternating two-task, and multi-agent competitive self-play settings.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 7; 2023: 7; 2024: 6; 2025: 8; 2026: 5.

37 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 112
  3. 2021Continual Learning of a Mixed Sequence of Similar and Dissimilar Taskscited by 80
  4. 2024Online DPO: Online Direct Preference Optimization with Fast-Slow Chasingcited by 38
  5. 2021Continuous Coordination As a Realistic Scenario for Lifelong Learningcited by 31
  6. 2022Building a Subspace of Policies for Scalable Continual Learningcited by 28
  7. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  8. 2022Lifelong Reinforcement Learning with Modulating Maskscited by 20
  9. 2025Robust Finetuning of Vision-Language-Action Robot Policies via Parameter Mergingcited by 19
  10. 2023Structure in Deep Reinforcement Learning: A Survey and Open Problemscited by 16
  11. 2025A Survey of Continual Reinforcement Learningcited by 16
  12. 2022A Dirichlet Process Mixture of Robust Task Models for Scalable Lifelong Reinforcement Learningcited by 15
25 more
  1. 2026Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learningcited by 13
  2. 2023System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learningcited by 12
  3. 2022Beyond Supervised Continual Learning: a Reviewcited by 11
  4. 2023Continual Task Allocation in Meta-Policy Network via Sparse Promptingcited by 11
  5. 2023Multi-agent Continual Coordination via Progressive Task Contextualizationcited by 10
  6. 2022Memory-efficient Reinforcement Learning with Value-based Knowledge Consolidationcited by 9
  7. 2024Continual Offline Reinforcement Learning via Diffusion-based Dual Generative Replaycited by 6
  8. 2024Solving Continual Offline RL through Selective Weights Activation on Aligned Spacescited by 6
  9. 2025Efficient Continual Adaptation of Pretrained Robotic Policy with Online Meta-Learned Adapterscited by 6
  10. 2022Self-Activating Neural Ensembles for Continual Reinforcement Learningcited by 5
  11. 2024Measuring Policy Distance for Multi-Agent Reinforcement Learningcited by 4
  12. 2023OER: Offline Experience Replay for Continual Offline Reinforcement Learningcited by 3
  13. 2024Debiased Offline Representation Learning for Fast Online Adaptation in Non-stationary Dynamicscited by 2
  14. 2024Continual Diffuser (CoD): Mastering Continual Offline Reinforcement Learning with Experience Rehearsalcited by 2
  15. 2025Overcoming Non-stationary Dynamics with Evidential Proximal Policy Optimizationcited by 2
  16. 2026WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Modelscited by 2
  17. 2021Sequence Adaptation via Reinforcement Learning in Recommender Systemscited by 1
  18. 2023Using Curiosity for an Even Representation of Tasks in Continual Offline Reinforcement Learningcited by 1
  19. 2025LaMOuR: Leveraging Language Models for Out-of-Distribution Recovery in Reinforcement Learningcited by 1
  20. 2025Action-Adaptive Continual Learning: Enabling Policy Generalization under Dynamic Action Spacescited by 1
  21. 2025Lifelong Learning with Behavior Consolidation for Vehicle Routingcited by 1
  22. 2026Continual Policy Consolidation for Lifelong Robot Learningcited by 1
  23. 2025Transformer World Model for Sample Efficient Multi-Agent Reinforcement Learningcited by 0
  24. 2026Continual Learning of Achieving Forgetting-free and Positive Knowledge Transfercited by 0
  25. 2026Principled Fast and Meta Knowledge Learners for Continual Reinforcement Learningcited by 0