mlsanityData to 14 Oct 2026

Continual Knowledge Adaptation for Reinforcement Learning

arXiv 2510.19314 ↗First posted 22 Oct 2025cs.AIcited by 5

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reinforcement Learning enables agents to learn optimal behaviors through interactions with environments. However, real-world environments are typically non-stationary, requiring agents to continuously adapt to new tasks and changing conditions. Although Continual Reinforcement Learning facilitates learning across multiple tasks, existing methods often suffer from catastrophic forgetting and inefficient knowledge utilization. To address these challenges, we propose Continual Knowledge Adaptation for Reinforcement Learning (CKA-RL), which enables the accumulation and effective utilization of historical knowledge. Specifically, we introduce a Continual Knowledge Adaptation strategy, which involves maintaining a task-specific knowledge vector pool and dynamically using historical knowledge to adapt the agent to new tasks. This process mitigates catastrophic forgetting and enables efficient knowledge transfer across tasks by preserving and adapting critical model parameters. Additionally, we propose an Adaptive Knowledge Merging mechanism that combines similar knowledge vectors to address scalability challenges, reducing memory requirements while ensuring the retention of essential knowledge. Experiments on three benchmarks demonstrate that the proposed CKA-RL outperforms state-of-the-art methods, achieving an improvement of 4.20% in overall performance and 8.02% in forward transfer. The source code is available at https://github.com/Fhujinwu/CKA-RL.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 42 of its 62 references that are in the map. First posted on arXiv 22 Oct 2025.

Builds on

42 of 62 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Self-Composing Policies for Scalable Continual Reinforcement Learning4 Jun
  2. 2025Mastering Continual Reinforcement Learning through Fine-Grained Sparse Network Allocation and Dormant Neuron Exploration7 Mar
  3. 2025Knowledge Retention for Continual Model-Based Reinforcement Learning6 Mar
  4. 2024Continual Task Learning through Adaptive Policy Self-Composition18 Nov
  5. 2024Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learning26 May

Other references · 37

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2015Continuous control with deep reinforcement learningcited by 3,407
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  7. 2015Trust Region Policy Optimizationcited by 2,580
  8. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  9. 2016Progressive Neural Networkscited by 1,361
  10. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  11. 2022Editing Models with Task Arithmeticcited by 1,120
  12. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
25 more
  1. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  2. 2018Residual Reinforcement Learning for Robot Controlcited by 274
  3. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 274
  4. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 269
  5. 2021Embodied Intelligence via Learning and Evolutioncited by 163
  6. 2021Machine Unlearning via Algorithmic Stabilitycited by 130
  7. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  8. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 116
  9. 2018Model-Based Reinforcement Learning via Meta-Policy Optimizationcited by 101
  10. 2023A Definition of Continual Reinforcement Learningcited by 101
  11. 2020Transient Non-Stationarity and Generalisation in Deep Reinforcement Learningcited by 75
  12. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 69
  13. 2022Modular Lifelong Reinforcement Learning via Neural Compositioncited by 40
  14. 2022Disentangling Transfer in Continual Reinforcement Learningcited by 33
  15. 2022Building a Subspace of Policies for Scalable Continual Learningcited by 28
  16. 2022Lifelong Reinforcement Learning with Modulating Maskscited by 22
  17. 2022Learning Active Camera for Multi-Object Navigationcited by 21
  18. 2023Prediction and Control in Continual Reinforcement Learningcited by 19
  19. 2021Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillationcited by 15
  20. 2022Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-2cited by 11
  21. 2023Continual Task Allocation in Meta-Policy Network via Sparse Promptingcited by 11
  22. 2024Efficient Dynamic Ensembling for Multiple LLM Expertscited by 9
  23. 2023Replay-enhanced Continual Reinforcement Learningcited by 7
  24. 2024Open-World Drone Active Tracking with Goal-Centered Rewardscited by 6
  25. 2025Dynamic Compressing Prompts for Efficient Inference of Large Language Modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stagescited by 6
  2. 2025Beyond Fast and Slow: Cognitive-Inspired Elastic Reasoning for Large Language Modelscited by 2
  3. 2026Lost or Hidden? A Concept-Level Forgetting in Supervised Continual Learningcited by 0
  4. 2026How to Find and Reuse Policies for Continuous Adaptation in Lifelong Reinforcement Learningcited by 0
  5. 2026Continual Graph Multi-Agent Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Knowledge is Not Enough: Injecting RL Skills for Continual Adaptation
  2. 2025A Survey of Continual Reinforcement Learning
  3. 2026Principled Fast and Meta Knowledge Learners for Continual Reinforcement Learning
  4. 2025Demonstration-Guided Continual Reinforcement Learning in Dynamic Environments
  5. 2024Multi-granularity Knowledge Transfer for Continual Reinforcement Learning
  6. 2026Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning
  7. 2025Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review
  8. 2025Efficient Adaptation of Reinforcement Learning Agents to Sudden Environmental Change