mlsanityData to 7 Oct 2026

Lifetime policy reuse and the importance of task capacity

arXiv 2106.01741 ↗First posted 3 Jun 2021cs.LG cs.AI cs.NEcited by 4

Not on the map. Why?

Abstract (arXiv)

A long-standing challenge in artificial intelligence is lifelong reinforcement learning, where learners are given many tasks in sequence and must transfer knowledge between tasks while avoiding catastrophic forgetting. Policy reuse and other multi-policy reinforcement learning techniques can learn multiple tasks but may generate many policies. This paper presents two novel contributions, namely 1) Lifetime Policy Reuse, a model-agnostic policy reuse algorithm that avoids generating many policies by optimising a fixed number of near-optimal policies through a combination of policy optimisation and adaptive policy selection; and 2) the task capacity, a measure for the maximal number of tasks that a policy can accurately solve. Comparing two state-of-the-art base-learners, the results demonstrate the importance of Lifetime Policy Reuse and task capacity based pre-selection on an 18-task partially observable Pacman domain and a Cartpole domain of up to 125 tasks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 37 of its 85 references that are in the map. First posted on arXiv 3 Jun 2021.

Builds on

37 of 85 references.

References · 37

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2016Learning without Forgettingcited by 2,465
  6. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  7. 2016OpenAI Gymcited by 1,684
  8. 2016Progressive Neural Networkscited by 1,332
  9. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  10. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  11. 2016Learning to learn by gradient descent by gradient descentcited by 616
  12. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
25 more
  1. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  2. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 359
  3. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
  4. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  5. 2018Selective Experience Replay for Lifelong Learningcited by 239
  6. 2014Robots that can adapt like animalscited by 203
  7. 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 159
  8. 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
  9. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  10. 2020Reinforcement Learning for Non-Stationary Markov Decision Processes: The Blessing of (More) Optimismcited by 75
  11. 2018Gotta Learn Fast: A New Benchmark for Generalization in RLcited by 59
  12. 2019Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewardscited by 38
  13. 2019Non-Stationary Markov Decision Processes, a Worst-Case Approach using Model-Based Reinforcement Learning, Extended versioncited by 36
  14. 2017Less-forgetful Learning for Domain Expansion in Deep Neural Networkscited by 27
  15. 2019Discounted Reinforcement Learning Is Not an Optimization Problemcited by 25
  16. 2019Continual Reinforcement Learning in 3D Non-stationary Environmentscited by 21
  17. 2017Using Task Descriptions in Lifelong Machine Learning for Improved Performance and Zero-Shot Transfercited by 19
  18. 2018Context-Aware Policy Reusecited by 15
  19. 2019The capacity of feedforward neural networkscited by 14
  20. 2017An Optimal Online Method of Selecting Source Policies for Reinforcement Learningcited by 13
  21. 2015Bayesian Policy Reusecited by 8
  22. 2017A Capacity Scaling Law for Artificial Neural Networkscited by 7
  23. 2018Target Transfer Q-Learning and Its Convergence Analysiscited by 6
  24. 2018ViZDoom: DRQN with Prioritized Experience Replay, Double-Q Learning, & Snapshot Ensemblingcited by 3
  25. 2013Multi-Task Policy Searchcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 1; 2025: 0; 2026: 1.

4 papers.

  1. 2022From Intelligent Agents to Trustworthy Human-Centred Multiagent Systemscited by 4
  2. 2023Agent based modelling for continuously varying supply chainscited by 0
  3. 2024CHIRPs: Change-Induced Regret Proxy metrics for Lifelong Reinforcement Learningcited by 0
  4. 2026The Parts Are Greater Than the Sum: Automated Task Sequencing for Efficient Training of Multi-Policy LLMscited by 0