mlsanityData to 14 Oct 2026

Position: Deployed Reinforcement Learning should be Continual

arXiv 2606.04029 ↗First posted 1 Jun 2026cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reinforcement Learning (RL) has received increasing attention and adoption in real-world use cases. Most of these systems follow a train-then-fix paradigm, where trained agents do not learn while interacting with the world until performance degrades and retraining becomes necessary. In this position paper, we argue that deploying an agent that is incapable of optimality, but receives an evaluative reward signal, is inherently a continual RL problem. We identify four sources of non-stationarity after deployment that necessitate never-ending learning, and highlight why the best deployed agents never stop adapting. We analyze successful examples of continual RL in the real world, and present the community with the advantages and measures to move away from the current train-then-fix paradigm.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 42 of its 88 references that are in the map. First posted on arXiv 1 Jun 2026.

Builds on

42 of 88 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Memory Allocation in Resource-Constrained Reinforcement Learning9 Jun
  2. 2025The Cell Must Go On: Agar.io for Continual Reinforcement Learning23 May
  3. 2025Rethinking the Foundations for Continual Reinforcement Learning10 Apr
  4. 2024Three Dogmas of Reinforcement Learning15 Jul
  5. 2024Position: Lifetime tuning is incompatible with continual reinforcement learning2 Apr

Other references · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  3. 2016Concrete Problems in AI Safetycited by 1,642
  4. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  6. 2019Solving Rubik's Cube with a Robot Handcited by 734
  7. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
  8. 2017Safe Reinforcement Learning via Shieldingcited by 449
  9. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  10. 2020Performative Predictioncited by 327
  11. 2019Challenges of Real-World Reinforcement Learningcited by 291
  12. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 274
25 more
  1. 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 174
  2. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
  3. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  4. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 116
  5. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 70
  6. 2022Safe Reinforcement Learning by Imagining the Near Futurecited by 67
  7. 2023Continual Learning: Applications and the Road Forwardcited by 59
  8. 2024MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Datacited by 52
  9. 2023Continual Learning as Computationally Constrained Reinforcement Learningcited by 37
  10. 2022Object Detection Using Sim2Real Domain Randomization for Robotic Applicationscited by 25
  11. 2023Prediction and Control in Continual Reinforcement Learningcited by 19
  12. 2022The Alberta Plan for AI Researchcited by 18
  13. 2024Deep Policy Gradient Methods Without Batch Updates, Target Networks, or Replay Bufferscited by 18
  14. 2022Goal-Space Planning with Subgoal Modelscited by 12
  15. 2023Scalable Real-Time Recurrent Learning Using Columnar-Constructive Networkscited by 11
  16. 2024Monitored Markov Decision Processescited by 10
  17. 2022On Convergence of Average-Reward Off-Policy Control Algorithms in Weakly Communicating MDPscited by 6
  18. 2024Step-size Optimization for Continual Learningcited by 6
  19. 2024The Need for a Big World Simulator: A Scientific Challenge for Continual Learningcited by 5
  20. 2025Can Context Bridge the Reality Gap? Sim-to-Real Transfer of Context-Aware Policiescited by 4
  21. 2021Learning to Be Cautiouscited by 3
  22. 2024A Black Swan Hypothesis: The Role of Human Irrationality in AI Safetycited by 3
  23. 2025Risk-sensitive Actor-Critic with Static Spectral Risk Measures for Online and Offline Reinforcement Learningcited by 2
  24. 2025Generalization in Monitored Markov Decision Processes (Mon-MDPs)cited by 1
  25. 2025In-Context Learning can Perform Continual Learning Like Humanscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Space-sampled Value Decay: Forgetting Mechanisms for Non-stationary Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey of Continual Reinforcement Learning
  2. 2025Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review
  3. 2024Reset & Distill: A Recipe for Overcoming Negative Transfer in Continual Reinforcement Learning
  4. 2025Continual Reinforcement Learning by Planning with Online World Models
  5. 2025Continual Reinforcement Learning for Cyber-Physical Systems: Lessons Learned and Open Challenges
  6. 2025Demonstration-Guided Continual Reinforcement Learning in Dynamic Environments
  7. 2026Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning
  8. 2026Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions