mlsanityData to 7 Oct 2026

Principled Fast and Meta Knowledge Learners for Continual Reinforcement Learning

arXiv 2603.00903 ↗First posted 1 Mar 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Inspired by the human learning and memory system, particularly the interplay between the hippocampus and cerebral cortex, this study proposes a dual-learner framework comprising a fast learner and a meta learner to address continual Reinforcement Learning~(RL) problems. These two learners are coupled to perform distinct yet complementary roles: the fast learner focuses on knowledge transfer, while the meta learner ensures knowledge integration. In contrast to traditional multi-task RL approaches that share knowledge through average return maximization, our meta learner incrementally integrates new experiences by explicitly minimizing catastrophic forgetting, thereby supporting efficient cumulative knowledge transfer for the fast learner. To facilitate rapid adaptation in new environments, we introduce an adaptive meta warm-up mechanism that selectively harnesses past knowledge. We conduct experiments in various pixel-based and continuous control benchmarks, revealing the superior performance of continual learning for our proposed dual-learner approach relative to baseline methods. The code is released in https://github.com/datake/FAME.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 47 of its 70 references that are in the map. First posted on arXiv 1 Mar 2026.

Builds on

47 of 70 references.

References · 47

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2016Progressive Neural Networkscited by 1,332
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  9. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  10. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  11. 2015Policy Distillationcited by 414
  12. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
35 more
  1. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
  2. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  3. 2018Statistical Aspects of Wasserstein Distancescited by 202
  4. 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 176
  5. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  6. 2022Game-theoretic statistics and safe anytime-valid inferencecited by 145
  7. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 124
  8. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 112
  9. 2020A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrixcited by 99
  10. 2023A Definition of Continual Reinforcement Learningcited by 98
  11. 2020Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifierscited by 93
  12. 2021Same State, Different Task: Continual Reinforcement Learning without Interferencecited by 42
  13. 2019Policy Consolidation for Continual Reinforcement Learningcited by 37
  14. 2020Optimizing for the Future in Non-Stationary MDPscited by 37
  15. 2021Lifelong Robotic Reinforcement Learning by Retaining Experiencescited by 34
  16. 2022Disentangling Transfer in Continual Reinforcement Learningcited by 31
  17. 2018Continual Reinforcement Learning with Complex Synapsescited by 30
  18. 2022Building a Subspace of Policies for Scalable Continual Learningcited by 28
  19. 2025Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churncited by 22
  20. 2024Parseval Regularization for Continual Reinforcement Learningcited by 21
  21. 2015Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources in the same domaincited by 19
  22. 2023Prediction and Control in Continual Reinforcement Learningcited by 19
  23. 2018Importance Weighted Transfer of Samples in Reinforcement Learningcited by 16
  24. 2024Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learningcited by 16
  25. 2021CoMPS: Continual Meta Policy Searchcited by 15
  26. 2020REPAINT: Knowledge Transfer in Deep Reinforcement Learningcited by 14
  27. 2023Continual Task Allocation in Meta-Policy Network via Sparse Promptingcited by 11
  28. 2023Provable Representation with Efficient Planning for Partial Observable Reinforcement Learningcited by 10
  29. 2025Self-Composing Policies for Scalable Continual Reinforcement Learningcited by 10
  30. 2022Towards Evaluating Adaptivity of Model-Based Reinforcement Learning Methodscited by 9
  31. 2025Continual Reinforcement Learning by Planning with Online World Modelscited by 8
  32. 2023Replay-enhanced Continual Reinforcement Learningcited by 6
  33. 2024Solving Continual Offline RL through Selective Weights Activation on Aligned Spacescited by 6
  34. 2024Stable Continual Reinforcement Learning via Diffusion-based Trajectory Replaycited by 6
  35. 2025Knowledge Retention for Continual Model-Based Reinforcement Learningcited by 6

Built on by

Not cited yet.