mlsanityData to 14 Oct 2026

Action-Adaptive Continual Learning: Enabling Policy Generalization under Dynamic Action Spaces

arXiv 2506.05702 ↗First posted 6 Jun 2025cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Continual Learning (CL) is a powerful tool that enables agents to learn a sequence of tasks, accumulating knowledge learned in the past and using it for problem-solving or future task learning. However, existing CL methods often assume that the agent's capabilities remain static within dynamic environments, which doesn't reflect real-world scenarios where capabilities dynamically change. This paper introduces a new and realistic problem: Continual Learning with Dynamic Capabilities (CL-DC), posing a significant challenge for CL agents: How can policy generalization across different action spaces be achieved? Inspired by the cortical functions, we propose an Action-Adaptive Continual Learning framework (AACL) to address this challenge. Our framework decouples the agent's policy from the specific action space by building an action representation space. For a new action space, the encoder-decoder of action representations is adaptively fine-tuned to maintain a balance between stability and plasticity. Furthermore, we release a benchmark based on three environments to validate the effectiveness of methods for CL-DC. Experimental results demonstrate that our framework outperforms popular methods by generalizing the policy across action spaces.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 41 of its 67 references that are in the map. First posted on arXiv 6 Jun 2025.

Builds on

41 of 67 references.

References · 41

  1. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  4. 2016Progressive Neural Networkscited by 1,361
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  6. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  7. 2017Continual Learning with Deep Generative Replaycited by 1,158
  8. 2018Experience Replay for Continual Learningcited by 1,095
  9. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  10. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  11. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  12. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
29 more
  1. 2018Progress & Compress: A scalable framework for continual learningcited by 520
  2. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  3. 2020Reinforcement Learning with Augmented Datacited by 468
  4. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 449
  5. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  6. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  7. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  8. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
  9. 2019Mastering Complex Control in MOBA Games with Deep Reinforcement Learningcited by 161
  10. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 116
  11. 2018Don't forget, there is more than forgetting: new metrics for Continual Learningcited by 115
  12. 2023A Definition of Continual Reinforcement Learningcited by 101
  13. 2019Learning Action Representations for Reinforcement Learningcited by 75
  14. 2021Powerpropagation: A sparsity inducing weight reparameterisationcited by 49
  15. 2021Same State, Different Task: Continual Reinforcement Learning without Interferencecited by 43
  16. 2019Policy Consolidation for Continual Reinforcement Learningcited by 39
  17. 2022Building a Subspace of Policies for Scalable Continual Learningcited by 28
  18. 2022Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?cited by 24
  19. 2019Continual Reinforcement Learning in 3D Non-stationary Environmentscited by 22
  20. 2021Curiosity-Driven Exploration via Latent Bayesian Surprisecited by 22
  21. 2022Lifelong Reinforcement Learning with Modulating Maskscited by 22
  22. 2019Lifelong Learning with a Changing Action Setcited by 17
  23. 2023Learning to Prompt Knowledge Transfer for Open-World Continual Learningcited by 15
  24. 2022Task-Agnostic Continual Reinforcement Learning: Gaining Insights and Overcoming Challengescited by 11
  25. 2024t-DGR: A Trajectory-Based Deep Generative Replay Method for Continual Learning in Decision Makingcited by 11
  26. 2019Automated shapeshifting for function recovery in damaged robotscited by 8
  27. 2023Predictive auxiliary objectives in deep RL mimic learning in the braincited by 7
  28. 2024Statistical Context Detection for Deep Lifelong Reinforcement Learningcited by 7
  29. 2024Multi-granularity Knowledge Transfer for Continual Reinforcement Learningcited by 6

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025AbideGym: Turning Static RL Worlds into Adaptive Challengescited by 1