mlsanityData to 7 Oct 2026

Continual Learning as Computationally Constrained Reinforcement Learning

arXiv 2307.04345 ↗First posted 10 Jul 2023cs.LG cs.AIcited by 35

Not on the map. Why?

Abstract (arXiv)

An agent that efficiently accumulates knowledge to develop increasingly sophisticated skills over a long lifetime could advance the frontier of artificial intelligence capabilities. The design of such agents, which remains a long-standing challenge of artificial intelligence, is addressed by the subject of continual learning. This monograph clarifies and formalizes concepts of continual learning, introducing a framework and set of tools to stimulate further research.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 42 of its 89 references that are in the map. First posted on arXiv 10 Jul 2023.

Builds on

42 of 89 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  4. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  5. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  6. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  7. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  8. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  9. 2019Gradient based sample selection for online continual learningcited by 568
  10. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  11. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  12. 2019On Tiny Episodic Memories in Continual Learningcited by 299
30 more
  1. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
  2. 2023Understanding plasticity in neural networkscited by 170
  3. 2021Online Coreset Selection for Rehearsal-based Continual Learningcited by 114
  4. 2022The CLEAR Benchmark: Continual LEArning on Real-World Imagerycited by 92
  5. 2018Learning to Optimize under Non-Stationaritycited by 85
  6. 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 79
  7. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 71
  8. 2021Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Datacited by 67
  9. 2021Wide Neural Networks Forget Less Catastrophicallycited by 64
  10. 2020A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environmentscited by 62
  11. 2023Computationally Budgeted Continual Learning: What Does Matter?cited by 61
  12. 2021Reinforcement Learning, Bit by Bitcited by 59
  13. 2020Online Continual Learning under Extreme Memory Constraintscited by 48
  14. 2021Bootstrapped Meta-Learningcited by 45
  15. 2021Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent Statescited by 42
  16. 2014Optimal Exploration-Exploitation in a Multi-Armed-Bandit Problem with Non-stationary Rewardscited by 41
  17. 2021Autonomous Reinforcement Learning: Formalism and Benchmarkingcited by 33
  18. 2017Taming Non-stationary Bandits: A Bayesian Approachcited by 30
  19. 2023Online Continual Learning Without the Storage Constraintcited by 29
  20. 2020Drinking from a Firehose: Continual Learning with Web-scale Natural Languagecited by 24
  21. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 23
  22. 2022Non-Stationary Bandit Learning via Predictive Samplingcited by 21
  23. 2020Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processescited by 17
  24. 2019Efficient Change-Point Detection for Tackling Piecewise-Stationary Banditscited by 15
  25. 2022Beyond Supervised Continual Learning: a Reviewcited by 11
  26. 2020A Change-Detection Based Thompson Sampling Framework for Non-Stationary Banditscited by 10
  27. 2023Rapid Adaptation in Online Continual Learning: Are We Evaluating It Right?cited by 10
  28. 2023A Definition of Non-Stationary Banditscited by 9
  29. 2022An Information-Theoretic Framework for Supervised Learningcited by 6
  30. 2021Does the Adam Optimizer Exacerbate Catastrophic Forgetting?cited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 9; 2025: 16; 2026: 7.

35 papers.

Sort:

By area:

  1. 2023A Definition of Continual Reinforcement Learningcited by 98
  2. 2024Disentangling the Causes of Plasticity Loss in Neural Networkscited by 82
  3. 2023Continual Learning: Applications and the Road Forwardcited by 59
  4. 2024Weight Clipping for Deep Continual and Reinforcement Learningcited by 29
  5. 2025Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churncited by 22
  6. 2025Optimizers Qualitatively Alter Solutions And We Should Leverage Thiscited by 17
  7. 2025Rethinking the Foundations for Continual Reinforcement Learningcited by 15
  8. 2024Three Dogmas of Reinforcement Learningcited by 14
  9. 2024Plastic Learning with Deep Fourier Featurescited by 14
  10. 2024Self-Normalized Resets for Plasticity in Continual Learningcited by 11
  11. 2025Plasticity as the Mirror of Empowermentcited by 11
  12. 2023Harnessing Discrete Representations For Continual Reinforcement Learningcited by 10
23 more
  1. 2026FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability-Plasticity Tradeoffcited by 7
  2. 2024The Need for a Big World Simulator: A Scientific Challenge for Continual Learningcited by 5
  3. 2026Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directionscited by 5
  4. 2024Prospective Learning: Learning for a Dynamic Futurecited by 4
  5. 2024Data-dependent and Oracle Bounds on Forgetting in Continual Learningcited by 3
  6. 2025Decision Making in Hybrid Environments: A Model Aggregation Approachcited by 3
  7. 2025The Cell Must Go On: Agar.io for Continual Reinforcement Learningcited by 2
  8. 2025Memory Allocation in Resource-Constrained Reinforcement Learningcited by 2
  9. 2026The Rank and Gradient Lost in Non-stationarity: Sample Weight Decay for Mitigating Plasticity Loss in Reinforcement Learningcited by 2
  10. 2026Flexible Empowerment at Reasoning with Extended Best-of-N Samplingcited by 2
  11. 2025Prospective Learning in Retrospectcited by 1
  12. 2025The World Is Bigger! A Computationally-Embedded Perspective on the Big World Hypothesiscited by 1
  13. 2026Learning to Forget: Continual Learning with Adaptive Weight Decaycited by 1
  14. 2026Position: Deployed Reinforcement Learning should be Continualcited by 1
  15. 2024Exploration Unboundcited by 0
  16. 2025Learning Model Successorscited by 0
  17. 2025Illuminating the Three Dogmas of Reinforcement Learning under Evolutionary Lightcited by 0
  18. 2025Capacity-Constrained Continual Learningcited by 0
  19. 2025Ergodic Risk Measures: Towards a Risk-Aware Foundation for Continual Reinforcement Learningcited by 0
  20. 2025Forgetting is Everywherecited by 0
  21. 2025Optimal control of the future via prospective learning with controlcited by 0
  22. 2025Dynamic Mixture of Experts Against Severe Distribution Shiftscited by 0
  23. 2026The syntax and semantics of goalscited by 0